The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
CNN-based sensor fusion techniques for multimodal human activity recognition
Abstract
Deep learning (DL) methods receive increasing attention within the field of human activity recognition (HAR) due to their success in other machine learning domains. Nonetheless, a direct transfer of these methods is often not possible due to domain specific challenges (e.g. handling of multi-modal sensor data, lack of large labeled datasets). In this paper, we address three key aspects for the future development of robust DL methods for HAR: (1) Is it beneficial to apply data specific normalization? (2) How to optimally fuse multimodal sensor data? (3) How robust are these approaches with respect to available training data? We evaluate convolutional neuronal networks (CNNs) on a new large real-world multimodal dataset (RBK) as well as the PAMAP2 dataset. Our results indicate that sensor specific normalization techniques are required. We present a novel pressure specific normalization method which increases the F1-score by ∼ 4.5 percentage points (pp) on the RBK dataset. Further, we show that late- and hybrid fusion techniques are superior compared to early fusion techniques, increasing the F1-score by up to 3.5 pp (RBK dataset). Finally, our results reveal that in particular CNNs based on a shared filter approach have a smaller dependency on the amount of available training data compared to other fusion techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
HASKELL, W. L., LEE, I.-M., PATE, R. R., POWELL, K. E., BLAIR, S. N., FRANKLIN, B. A., MACERA, C. A., HEATH, G. W., THOMPSON, P. D., & BAUMAN, A. (2007) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3180616b27 · Cited by 4,616
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Empirical Evaluation of Rectified Activations in Convolutional Network
Xu, B., Wang, N., Chen, T., & Li, M. (2015) · arXiv · doi:10.48550/arxiv.1505.00853 · Cited by 2,264
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 3×
“The presented F 1 -scores for the newly acquired RBK dataset are lower than the results reported on publicly available datasets [15, 17, 22].”
MethodResultSelf-citationConvolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection
Zappi, P., Lombriser, C., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Tröster, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77690-1_2 · Cited by 329
Learning deep physiological models of affect
Martinez, H. P., Bengio, Y., & Yannakakis, G. N. (2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2247823 · Cited by 287
Cited by
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“odel. Axis-based Fusion (AF). This manner treats signal sources in more detail by handling each sensor axis separately. In such a way, the interference between different sensor axes is gotten rid of. [100] referred this manner to Channel-based late fusion (CB-LF). Nevertheless, the sensor channel may be confused with…”
BackgroundMethodDeep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks
Reiss, A., Indlekofer, I., Schmidt, P., & Van Laerhoven, K. (2019) · Sensors · doi:10.3390/s19143079 · Cited by 377
Self-citationPlanetary gearbox fault diagnosis using bidirectional-convolutional LSTM networks
Shi, J., Peng, D., Peng, Z., Zhang, Z., Goebel, K., & Wu, D. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.107996 · Cited by 318
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“odel. Axis-based Fusion (AF). This manner treats signal sources in more detail by handling each sensor axis separately. In such a way, the interference between different sensor axes is gotten rid of. [100] referred this manner to Channel-based late fusion (CB-LF). Nevertheless, the sensor channel may be confused with…”
BackgroundMethodYadav, S. K., Tiwari, K., Pandey, H. M., & Akbar, S. A. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106970 · Cited by 311
Metadata
- Authors
- Münzner, S., Schmidt, P., Reiss, A., Hanselmann, M., Stiefelhagen, R., & Dürichen, R.
- Venue
- Proceedings of the 2017 ACM International Symposium on Wearable Computers
- Publisher
- ACM
- Published
- 2017-09-11
- Pages
- 158-165
- OpenAlex
- W2751594996
- Cited by
- 235
- References
- 11
- FWCI
- 5.34
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: CNN-based sensor fusion techniques for multimodal human activity recognition. https://app.cochat.ai/papers/10.1145/3123021.3123046
- The paper
- Münzner, S., Schmidt, P., Reiss, A., Hanselmann, M., Stiefelhagen, R., & Dürichen, R. (2017). CNN-based sensor fusion techniques for multimodal human activity recognition. Proceedings of the 2017 ACM International Symposium on Wearable Computers. https://doi.org/10.1145/3123021.3123046
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.