LSTM for Event Detection on Time Series / Sequences (Gait Analysis)

Hello!

In 3-dimensional Gait Analysis, Human Gait Event Detection is a crucial part. The Gold Standard for this is the ground reaction force (GRF) plate. But often there are only 2-3 GRF plates available and so other events throughout the trial (which are not on the GRF) have to be set manually (there are some algorithms too or autocorrelate events from nexus but in some patient groups this doesnt work very well). But espacially the Foot Off event is often set wrong. This paper is using an LSTM with ALL events set manually to detect events of (unseen) patients.

Now my question is: What if we would only use the GRF events in the LSTM to detect all other events of the trial? Is this even possible?

I have been trying some things with the LSTM but can not really make it work. When i use the classify method at the end i get no events, when i use the predict i get some peaks where an event should be, but only in the middle (where the GRF is) not for other steps.

The input is the whole trial of like 700-1000 data points (of 6 different markers --> 18 x 700/1000 matrix) with 5 events. (The event for the Training data is ONLY set if a GRF plate is hit. But for the Test data is should also be set on all other steps where no GRF plate is)

• 0 is no event
• 1 is Left Foot Strike
• 2 is Left Foot Off
• 3 ist Right Foot Strike
• 4 is Right Foot Off

Pleas contact me if someone has an idea or needs more information,

Thanks.

[Mohammad Sami]

In Matlab there is a signal labelling example using EEG. That would be a good starting point.

https://www.mathworks.com/help/signal/examples/waveform-segmentation-using-deep-learning.html

I assume your training data is timeseries tracking the markers in 3D space. And your target would be the 5 labels you listed.

[Barry]

Thank you for the quick answer Mohammad.

The problem with this is that all data points are labeled like in the paper i linked. But would this be possible if only 1 event for the P/QRS/T-Wave would be available in the whole trial to predict all other events? Because (in my example) i only have 2 steps at max for like 10 setps i want to predict.

[Michael Stanley]

Hi Barry, I am working on something similar, detection of gait event with IMU data. I can get the time series data work with MATLAB toolbox. The configuration I used is laballed data for both training and testing. It is impossible to verify your test data is you don't have the truth event laballed. If you labelled the event for the step that doesn't have GRF data then you can verify your network output with these labelled events.

The thing is your data has to be prepared and trained with data that doesn't depend on the GRF data, since your network will make prediction based on the inputs for steps without GRF. If you trained with GRF data, then for the steps without GRF data, your network will not work.

Also you can have multiple networks, I notice a possible problem if you label the data as 1/2/3/4/5 for differnt event in the same network, is that when your network give you some unstability, it may float any where between 0-5 and you can be share which event it is detecting, so I find it is more accurate, to have a network for each event, so the prediction only peak for the output they trained for. For exmaple, I trained for a nework to predict 1 for initial contact, and 2 for end contact, else 0. the network will give an output around 1 around end contact, and sometimes it oscillate around 1 between end contact to initial contact (swing phase). so when the network is about half way sure of a 2, it will give a output of 1, and this may confuse the detection of 1. Later on I learnt that when tranining object classification, people using an array output such as [1,0,0], [0,1,0], [0,0,1] as the output for each object. I think it is just so such easier for the program to train that way.