Exploring human activity annotation using a privacy preserving 3D model
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Annotating activity recognition datasets is a very time consuming process. Using lay annotators (e.g. using crowd-sourcing) has been suggested to speed this up. However, this requires to preserve privacy of users and may preclude relying on video for annotation. We investigate to which extent using a 3D human model animated from the data of inertial sensors placed on the limbs allows for annotation of human activities. We animate the upper body of the 3D model with the data from 5 inertial measurement sensors obtained from the OPPORTUNITY dataset. The animated model is shown to 6 people in a suite of experiments in order to understand to which extent it can be used for labelling. We present 3 experiments where we investigate the use of a 3D model for i) activity segmentation, ii) for "open-ended" annotation where users freely describe the activity they see on screen, and iii) traditional annotation, where users pick one activity among a pre-defined list of activities. In the latter case, results show that users recognise the model's activities with 56% accuracy when picking from 11 possible activities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Early work to that end showed a surprising human ability to recognise characteristic activities from such a virtual model Ciliberto et al. (2016).”
From Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-Based Human Activity Recognition (2021) · cites this paper as Ciliberto et al. (2016)
Citations
Cites
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147 · cited 1×
“Activity recognition is fundamental for context-aware computing [3].”
BackgroundA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 1×
“There are multiple approaches to annotation [5].”
Accurate activity recognition in a home setting
van Kasteren, T., Noulas, A., Englebienne, G., & Kröse, B. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409637 · Cited by 874 · cited 1×
“That can be done using short audio labels recorded by the subject of the dataset together with the inertial data [7] [12].”
BackgroundCollecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Self-citationAmazon Mechanical Turk: Gold Mine or Coal Mine?
Fort, K., Adda, G., & Cohen, K. B. (2011) · Computational Linguistics · doi:10.1162/coli_a_00057 · Cited by 761
Avci, A., Bosch, S., Marin-Perianu, M., Marin-Perianu, R., & Havinga, P. (2010) · University of Twente Research Information · Cited by 563
Vondrick, C., Patterson, D., & Ramanan, D. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0564-1 · Cited by 448 · cited 1×
“It has also been used to label natural language [14], for speech recognition [10] and for multimedia tagging [13].”
MethodThe effects of filtered video on awareness and privacy
Boyle, M., Edwards, C., & Greenberg, S. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358935 · Cited by 277 · cited 1×
“One way to protect the anonymity of the subjects in the dataset can be the application of a mask (e.g. a blur or a pixelize video filter [4]) to the elements in the video that could be considered meaningful from a privacy point of view.”
BackgroundPerspectives on crowdsourcing annotations for natural language processing
Wang, A., Hoang, C. D. V., & Kan, M. (2012) · Language Resources and Evaluation · doi:10.1007/s10579-012-9176-1 · Cited by 159
Parent, G., & Eskenazi, M. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-760 · Cited by 60 · cited 1×
“It has also been used to label natural language [14], for speech recognition [10] and for multimedia tagging [13].”
MethodDai, J., Wu, J., Saghafi, B., Konrad, J., & Ishwar, P. S. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301356 · Cited by 51 · cited 1×
“Another approach consists in using low resolution camera to preserve the privacy of the subject recorded [6].”
BackgroundTagging human activities in video by crowdsourcing
Nguyen-Dinh, L.-V., Waldburger, C., Roggen, D., & Tröster, G. (2013) · International Conference on Multimedia Retrieval · doi:10.1145/2461466.2461508 · Cited by 26
Self-citationVoiceLabel: using speech to label mobile sensor data
Harada, S., Lester, J., Patel, K., Saponas, T. S., Fogarty, J. A., Landay, J. A., & Wobbrock, J. O. (2008) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1452392.1452407 · Cited by 19 · cited 1×
“That can be done using short audio labels recorded by the subject of the dataset together with the inertial data [7] [12].”
Background
Research works printed but not linked: 1
- XSens. 2000. M Tx 3D Tracker . ( 2000 ). https://www.xsens.com/products/mtx/ XSens. 2000. MTx 3D Tracker. (2000). https://www.xsens.com/products/mtx/
Cited by
Ciliberto, M., Fortes Rey, V., Calatroni, A., Lukowicz, P., & Roggen, D. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.792065 · Cited by 35
“Early work to that end showed a surprising human ability to recognise characteristic activities from such a virtual model Ciliberto et al. (2016).”
BackgroundSelf-citationPrivacy preserving human activity recognition framework using an optimized prediction algorithm
Kumar, K. V., & Harikiran, J. (2022) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v11.i1.pp254-264 · Cited by 15
Annotation Performance for multi-channel time series HAR Dataset in Logistics
Reining, C., Rueda, F. M., Niemann, F., Fink, G. A., & ten Hompel, M. (2020) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops48775.2020.9156170 · Cited by 15
“Provided with a list of 11 activities, they reached an average accuracy of 56% [10].”
BackgroundSemantic human activity annotation tool using skeletonized surveillance videos
Lee, B., Lee, M., Zhang, P., Tessier, A., & Khan, A. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3343807 · Cited by 9
“Ciliberto et al [3] used anonymized 3D skeletons to maintain occupant’s privacy for activity annotation of a single person.”
MethodKumar, K. V., & Jonnadula, H. (2021) · Bulletin of Electrical Engineering and Informatics · doi:10.11591/eei.v10i6.3204 · Cited by 7
“[21] explored a privacy preserving 3D model for human activity recognition.”
Background
Metadata
- Authors
- Ciliberto, M., Roggen, D., & Morales, F. J. O.
- Affiliations
- University of Sussex (GB)
- Venue
- Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct
- Publisher
- ACM
- Published
- 2016-09-12
- Pages
- 803-812
- OpenAlex
- W2520111655
- Cited by
- 7
- References
- 13
- Citations by year
- 2025: 1 · 2022: 1 · 2021: 2 · 2020: 1 · 2019: 1 · 2018: 1
- FWCI
- 0.23
- Type
- conference paper
- Language
- en
- Open access
- green
- figshare.com
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- EPSRC (EP/N007816/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- 3D human body model, privacy preservation, inertial sensors, Opportunity dataset, activity recognition, activity segmentation, crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.