Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index; 8 of the 23 sources its claims cite were checked for support.
Abstract
Introduces DeepConvLSTM, a deep learning architecture combining convolutional and LSTM recurrent units for multimodal wearable activity recognition, outperforming non-recurrent deep networks and handcrafted feature baselines by 4% to 9%.
Overview
CoChat Summary
TL;DR
The paper proposes DeepConvLSTM, a deep framework combining four convolutional layers with two LSTM recurrent layers for wearable activity recognition from raw multimodal sensor data. It reports improved F1 scores over a baseline CNN and prior results on the OPPORTUNITY and Skoda datasets, including 0.915 F1 for OPPORTUNITY gestures with the Null class and 0.958 F1 on Skoda.
Key findings
- On OPPORTUNITY, DeepConvLSTM achieved F1 scores of 0.930 for modes of locomotion without Null, 0.895 for modes with Null, 0.866 for gestures without Null, and 0.915 for gestures with Null; the baseline CNN scored 0.912, 0.878, 0.783, and 0.883, respectively. The paper reports a 6% average improvement over the best OPPORTUNITY challenge submissions and more than 9% improvement on gesture recognition without Null.
- On Skoda, DeepConvLSTM reached 0.958 F1, compared with 0.884 for the baseline CNN, 0.893 for CNN [43], and 0.861 for CNN [23], improving the best reported result by 6%.
- Multimodal fusion improved gesture recognition on OPPORTUNITY without Null: 0.689 F1 with accelerometers only, 0.611 with gyroscopes only, 0.745 with accelerometers + gyroscopes, 0.839 with accelerometers + gyroscopes + magnetic sensors, and 0.864 with the full OPPORTUNITY sensor set.
- DeepConvLSTM had 999,122 parameters versus 7,445,458 for the baseline CNN, a 600% increase for the baseline; training took 340.3 min versus 282.2 min, and classification took 6.68 s versus 5.43 s, with DeepConvLSTM classifying almost 15 min of data per second and achieving 900× real-time on the GPU used.
Methods
- DeepConvLSTM uses four convolutional layers with 64 feature maps each, followed by two dense LSTM layers with 128 cells and a softmax output; the baseline CNN shares the convolutional layers but uses two non-recurrent dense layers with 128 units. Models were trained supervised with cross-entropy loss, mini-batch gradient descent with RMSProp, learning rate 10e-3, decay factor 0.9, mini-batches of 100 segments, and dropout p=0.5 on dense-layer inputs.
- Inputs were sliding windows of 500 ms with 250 ms step size, using raw sensor data with minimal preprocessing: linear interpolation for missing values and per-channel normalization to [0,1]. Evaluation used weighted F1 to address class imbalance.
- Experiments used the OPPORTUNITY dataset (on-body sensors, 113 channels, 30 Hz; Task A: 5-class modes of locomotion/postures; Task B: 18-class sporadic gestures) and the Skoda dataset (one subject, 10 right-arm 3D accelerometers, decimated to 30 Hz, 10 manipulative gestures, about 3 h and about 70 repetitions per gesture). Analyses included comparison to baseline CNN and published results, confusion matrices, sensor-modality fusion, and hyperparameter evaluations of sequence length and number of convolutional layers.
Limitations
- The paper states that DeepConvLSTM has not yet been implemented on a wearable device; the GPU used clearly outperforms current high-end wearable computational power, and training is best envisioned server-side. It also states that pooling was omitted because the sliding-window setup limits downsampling, though pooling could be useful without that requirement, and that training time increases with the number of layers, creating a performance–computational budget trade-off.
- Evident from the text: Skoda evaluation uses one subject and only right-arm sensors, and OPPORTUNITY testing is limited to ADL4 and ADL5 for Subjects 2 and 3, which constrains generalizability. The text does not report cross-validation or statistical significance testing.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



![Figure 4 Placement of on-body sensors used in the OPPORTUNITYdataset (left: inertial measurements units; right: 3-axis accelerometers) [7].](https://figures.cochat.ai/paper-figures/27c0e4b39d104fe2b5315587d736fd2d/deep-convolutional-and-lstm-recurrent-neural-networks-for-multimodal-wearable-figure-4.jpg)




Verification
Claims the paper makes
5 Valid · 5 Not valid · 13 Not checked · 99 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“The choice of the number of recurrent layers is made following the results presented in [33], where the authors showed that a depth of at least two recurrent layers is beneficial when processing sequential data.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 3. Architecture > 3.1. DeepConvLSTM
What the sources say
- Supportedcites [33] Karpathy, A., Johnson, J., and Li, F.F. (2015). Visualizing and understanding recurrent networks. Cornell Univ. Lab., arXiv:1506.02078. source
Section 4.1 compares recurrent networks with one, two, and three layers on two sequence datasets and states that a depth of at least two is consistently beneficial. This directly supports the attributed finding, within the paper’s tested settings. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This seems to indicate that although deep learning techniques are often employed with large amounts of data, (e.g., millions of frames in computer vision [22]), they may actually be applicable to problem domains where acquiring annotated data is very costly, such as in supervised activity recognition.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 5. Results and Discussion > 5.4. Discussion
What the sources say
- Supportedcites [22] Pigou, L., Oord, A.V.D., Dieleman, S., van Herreweghe, M., and Dambre, J. (2015). Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video. arXiv Preprint, arXiv:1506.01911. source
The citation is attached to the example of large-scale computer-vision data, not to the citing authors’ conclusion about costly annotation. Section 4.1 (page unavailable) reports 1,720,800 labeled video frames in the gesture-recognition dataset, supporting the cited example of data on the order of millions of frames. (LLM, conf=0.86)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The baseline CNN also offers better results than the OPPORTUNITY submissions in the recognition of models of locomotion.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 5. Results and Discussion > 5.1. Performance Comparison
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From the results, we can also see how the model is able to learn from signals obtained from accelerometers, gyroscopes and magnetometers, fusing them without requiring any specific preprocessing.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 6. Conclusions
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(iv) it is applicable even if the gestures are longer than the observation window.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 5. Results and Discussion > 5.4. Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
84 more claims
Create a free account to see all 122 claims and to verify the 2 references still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 23
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, these complex network framework suffered from low ef ciency and can hardly meet real-time requirements in practice applications.”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [19]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, due to the lack of filtration techniques applied to multimodal data, the system’s performance was low.”
From A Multimodal IoT-Based Locomotion Classification System Using Features Engineering and Recursive Neural Network (2023) · cites this paper as [15]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“and yield better performances than pure CNNs or LSTM networks [12]”
From Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors (2018) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The proposed method outperforms other methods, such as those based on deep believe networks (DBNs) [10], SVM [10], and CNNs [13].”
From Deep Recurrent Neural Networks for Human Activity Recognition (2017) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The popular DeepConvLSTM model [7] can only afford four, moderately-sized convolution layers to learn effective data representations, and two LSTM layers for sequence modeling.”
From Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables (2021) · cites this paper as [7]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 3. Architecture
“The length of the kernels is the same for every convolutional layer, being defined as Pl=5,∀l=2,⋯,5.”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 4. Experimental Setup > 4.2. Performance Measure
“Following the OPPORTUNITY challenge experimental setup, the length of the window is 500 ms, with a step size of 250 ms.”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| Total | 996,800 +(128×nc)+nc | 996,800 +(128×nc)+nc | 7,443,136+(128×nc)+nc | 7,443,136+(128×nc)+nc |”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 3. Architecture > 3.3. Model Implementation and Training
Reported 996800 · recomputed 1037888
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| 6 | Wai,Waf,Wac,Wao: 7232×128 | 942,592 | W: 57,856×128 | 7,405,696 |”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 3. Architecture > 3.3. Model Implementation and Training
Reported 942592 · recomputed 3702784
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| 7 | Wai,Waf,Wac,Wao: 128×128 | 33,280 | W: 128×128 | 16,512 |”
§ Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition > 3. Architecture > 3.3. Model Implementation and Training
Reported 33280 · recomputed 131072
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 1×
“Other publications showed much deeper networks, such as “GoogLeNet”, which is a 27-layer deep neural network applied to image classification [45].”
BackgroundReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“Networks using LSTM cells have offered better performance than standard recurrent units in speech recognition, where they gave state-of-the-art results in phoneme recognition [20].”
BackgroundConvolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185 · cited 1×
“Deep CNNs have had a major impact on fields, like content recommendation [24], speech recognition [25] and in computer vision [26–28], where they have become the de facto standard approach.”
BackgroundOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 1×
“Deep CNNs have had a major impact on fields, like content recommendation [24], speech recognition [25] and in computer vision [26–28], where they have become the de facto standard approach.”
BackgroundLee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412 · cited 1×
“Deep learning techniques have outperformed many conventional methods in computer vision [13] and audio classification [14].”
MethodA review of wearable sensors and systems with application in rehabilitation
Patel, S., Park, H., Bonato, P., Chan, L., & Rodgers, M. (2012) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-9-21 · Cited by 2,396 · cited 1×
“, from simple hand gestures to complex activities, such as “cooking a meal”) and the context in which they occur from sensor data is at the core of smart assistive technologies, such as in smart homes [1], in rehabilitation [2], in health support [3,4], in skill assessment [5] or in industrial settings [6].”
BackgroundBeyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062 · cited 2×
“Results of a comparative analysis on the Sports-1Mand UCF-101 datasets showed that LSTM cells were necessary to take full advantage of the motion information contained in the video and yielded the highest reported performance [30].”
BackgroundMethodConvolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178838 · Cited by 1,678 · cited 3×
“Nevertheless, such an approach is not strictly part of the architecture, and in the time series domain, we can see some examples of CNNs where not every convolutional layer is followed by a pooling operation [16].”
BackgroundA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 2×
“Sliding window approaches are commonly used for static and periodic activities, while sporadic activities lend themselves to template matching approaches or hidden Markov modelling [8,11].”
BackgroundIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“A number of datasets have been published for activity recognition, including the OPPORTUNITY [36], PAMAP [37], Skoda [38] and mHealth [39] datasets.”
MethodDeep content-based music recommendation
van den Oord, A., Dieleman, S., & Schrauwen, B. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 1,425
(2015) · Cited by 989
(2009) · neural information processing systems · Cited by 962
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 5×
“, a sensor signal), often a 1D kernel is used in a temporal convolution [23].”
MethodResultInfluentialVisualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · cited 1×
“The choice of the number of recurrent layers is made following the results presented in [33], where the authors showed that a depth of at least two recurrent layers is beneficial when processing sequential data.”
BackgroundCollecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Self-citationThe Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 3×
“higher than 88% on the recognition of only 17 sporadic gestures [7].”
BackgroundMethodInfluentialSelf-citationActivity identification using body-mounted sensors—a review of classification techniques
Preece, S. J., Goulermas, J. Y., Kenney, L. P. J., Howard, D., Meijer, K., & Crompton, R. (2009) · Physiological Measurement · doi:10.1088/0967-3334/30/4/r01 · Cited by 600 · cited 1×
“Sliding window approaches are commonly used for static and periodic activities, while sporadic activities lend themselves to template matching approaches or hidden Markov modelling [8,11].”
BackgroundmHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications
Banos, O., Garcia, R., Holgado-Terriza, J. A., Damas, M., Pomares, H., Rojas, I., Saez, A., & Villalonga, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-13105-4_14 · Cited by 583 · cited 1×
“A number of datasets have been published for activity recognition, including the OPPORTUNITY [36], PAMAP [37], Skoda [38] and mHealth [39] datasets.”
MethodPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 2×
“Most recognition systems select features from a pool of “engineered” features [12].”
BackgroundAvci, A., Bosch, S., Marin-Perianu, M., Marin-Perianu, R., & Havinga, P. (2010) · University of Twente Research Information · Cited by 563
Deep Convolutional Neural Networks for Large-scale Speech Tasks
Sainath, T. N., Kingsbury, B., Saon, G., Soltau, H., Mohamed, A.-R., Dahl, G., & Ramabhadran, B. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.08.005 · Cited by 496 · cited 1×
“Deep CNNs have had a major impact on fields, like content recommendation [24], speech recognition [25] and in computer vision [26–28], where they have become the de facto standard approach.”
BackgroundKeeping the Resident in the Loop: Adapting the Smart Home to the User
Rashidi, P., & Cook, D. J. (2009) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2009.2025137 · Cited by 443
Equilibrated adaptive learning rates for non-convex optimizationPreprint
Dauphin, Y., de Vries, H., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.04390 · Cited by 389
Turing computability with neural nets
Siegelmann, H. T., & Sontag, E. D. (1991) · Applied Mathematics Letters · doi:10.1016/0893-9659(91)90080-f · Cited by 377 · cited 1×
“These types of networks have Turing capabilities [18] and, thus, are in principle suited for learning sequences.”
BackgroundWearable Activity Tracking in Car Manufacturing
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tr, G. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.40 · Cited by 367 · cited 1×
“, from simple hand gestures to complex activities, such as “cooking a meal”) and the context in which they occur from sensor data is at the core of smart assistive technologies, such as in smart homes [1], in rehabilitation [2], in health support [3,4], in skill assessment [5] or in industrial settings [6].”
BackgroundSelf-citationFeature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection
Zappi, P., Lombriser, C., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Tröster, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77690-1_2 · Cited by 329 · cited 13×
“Performance changes are not significant in the case of the Skoda dataset, showing a plateau.”
BackgroundMethodInfluentialSelf-citationBeyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video
Pigou, L., van den Oord, A., Dieleman, S., Van Herreweghe, M., & Dambre, J. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-016-0957-7 · Cited by 290 · cited 4×
“, millions of frames in computer vision [22]), they may actually be applicable to problem domains where acquiring annotated data is very costly, such as in supervised activity recognition.”
BackgroundMethodInfluential(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303768966139 · Cited by 289
Analysis of CNN-based speech recognition system using raw speech as input
Palaz, D., Magimai-Doss, M., & Collobert, R. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-3 · Cited by 288 · cited 1×
“However, applying them to features derived from the raw sensor signals often leads to higher performance [21].”
BackgroundDieleman, S., Schlüter, J., Raffel, C., Olson, E. M., Sønderby, S. K., Nouri, D., Maturana, D., Thoma, M., Battenberg, E., Kelly, J., et al. (2015) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.27878 · Cited by 279 · cited 1×
“The neural networks here described are implemented in Theano using Lasagne [34], a lightweight library to build and train neural networks.”
MethodEnsemble deep learning for speech recognition
Deng, L., & Platt, J. C. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-433 · Cited by 248 · cited 1×
“Further improvements in terms of time series classification have been obtained in the speech recognition domain, by combining convolutional and recurrent layers in a unified deep framework, which contains either standard recurrent units [29] or LSTM cells [16].”
BackgroundBeyond Short Snippets: Deep Networks for Video Classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.08909 · Cited by 244
The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices
Kranz, M., Möller, A., Hammerla, N., Diewald, S., Plötz, T., Olivier, P., & Roalter, L. (2013) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2012.06.002 · Cited by 199
(2015) · arXiv (Cornell University) · Cited by 182
ActiServ: Activity Recognition Service for mobile phones
Berchtold, M., Budde, M., Gordon, D., Schmidtke, H. R., & Beigl, M. (2010) · International Symposium on Wearable Computers (ISWC) 2010 · doi:10.1109/iswc.2010.5665868 · Cited by 160 · cited 1×
“, as in [46]).”
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.04664 · Cited by 126 · cited 2×
“CNN [43] Results reported by Alsheikh et.”
MethodUsing mobile GPU for general-purpose computing – a case study of face recognition on smartphones
Cheng, K.-T., & Wang, Y.-C. (2011) · Proceedings of 2011 International Symposium on VLSI Design, Automation and Test · doi:10.1109/vdat.2011.5783575 · Cited by 91 · cited 1×
“High-end mobile platforms already contain GPUs that can be used for general purpose processing [47].”
BackgroundMazilu, S., Blanke, U., Hardegger, M., Tröster, G., Gazit, E., & Hausdorff, J. M. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557278 · Cited by 83 · cited 1×
“, from simple hand gestures to complex activities, such as “cooking a meal”) and the context in which they occur from sensor data is at the core of smart assistive technologies, such as in smart homes [1], in rehabilitation [2], in health support [3,4], in skill assessment [5] or in industrial settings [6].”
BackgroundIn-Home Activity Recognition: Bayesian Inference for Hidden Markov Models
Ordóñez, F. J., Englebienne, G., de Toledo, P., van Kasteren, T., Sanchis, A., & Kröse, B. (2014) · IEEE Pervasive Computing · doi:10.1109/mprv.2014.52 · Cited by 48 · cited 1×
“Patterns corresponding to activities are then detected within the streaming sensor data using either feature extraction on sliding windows followed by classification, template matching approaches [9] or hidden Markov modelling [10].”
MethodSelf-citationLimited-Memory Warping LCSS for Real-Time Low-Power Pattern Recognition in Wireless Nodes
Roggen, D., Cuspinera, L. P., Pombo, G., Ali, F. H., & Nguyen-Dinh, L.-V. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-15582-1_10 · Cited by 23 · cited 1×
“Patterns corresponding to activities are then detected within the streaming sensor data using either feature extraction on sliding windows followed by classification, template matching approaches [9] or hidden Markov modelling [10].”
MethodSelf-citationActivity recognition for creatures of habit
Gordon, D., Czerny, J., & Beigl, M. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-013-0638-2 · Cited by 20
CrowdSignals: a call to crowdfund the community's largest mobile dataset
Welbourne, E., & Tapia, E. M. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication · doi:10.1145/2638728.2641309 · Cited by 15 · cited 1×
“, through crowd-sourcing [48]).”
Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video
Pigou, L., van den Oord, A., Dieleman, S., et al. (2015) · arXiv · doi:10.48550/arxiv.1506.01911
Research works printed but not linked: 1
- LeCun, Y., and Bengio, Y. (1998). The Handbook of Brain Theory and Neural Networks, MIT Press.
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342
“We think that our choice makes sense in a way that the goal of deep learning approaches is to remove the bias due to manually designed features (Ordón̈ez and Roggen, 2016), thus enabling the network to learn the most discriminant useful features for the classification task.”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
“For multi-modal signals, it is better to use CNN since (Bhattacharya and Lane, 2016) Body-worn, ambiemt RBM Gesture, ADL, transportation Self, D01 (Chen and Xue, 2015) Body-worn CNN ADL Self (Chen et al., 2016b) Body-worn CNN ADL D06 (Edel and K¨oppe, 2016) Body-worn RNN ADL D01, D04, Self (Fang and Hu, 2014) Object,…”
MethodInfluentialDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
“, [113], [254], [257], [263], [264], [478], and [479]) based on patterns observed in the output of the mobile camera or other sensors.”
BackgroundInfluentialA State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Metadata
- Authors
- Ordóñez, F., & Roggen, D.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Ordóñez, Francisco
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2016-01-18
- Volume
- 16
- Issue
- 1
- Pages
- 115
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC4732148
- OpenAlex
- W2270470215
- Cited by
- 2,761
- References
- 49
- Citations by year
- 2026: 284 · 2025: 256 · 2024: 284 · 2023: 270 · 2022: 319 · 2021: 379 · 2020: 347 · 2019: 288 · 2018: 219 · 2017: 93 · 2016: 21
- FWCI
- 140.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Google Faculty Research Award (Is deep learning useful for wearable activity recognition?)
- EPSRC (EP/N007816/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- human activity recognition, wearable sensors, multimodal sensor fusion, convolutional neural networks, LSTM neural network, temporal dynamics, automatic feature extraction, hyperparameter optimization
- MeSH terms
- Machine Learning, Clothing, Humans, Human Activities, Signal Processing, Computer-Assisted, Databases, Factual, Neural Networks, Computer, Monitoring, Ambulatory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.