Berkeley MHAD: A comprehensive Multimodal Human Action Database
The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Over the years, a large number of methods have been proposed to analyze human pose and motion information from images, videos, and recently from depth data. Most methods, however, have been evaluated on datasets that were too specific to each application, limited to a particular modality, and more importantly, captured under unknown conditions. To address these issues, we introduce the Berkeley Multimodal Human Action Database (MHAD) consisting of temporally synchronized and geometrically calibrated data from an optical motion capture system, multi-baseline stereo cameras from multiple views, depth sensors, accelerometers and microphones. This controlled multimodal dataset provides researchers an inclusive testbed to develop and benchmark new algorithms across multiple modalities under known capture conditions in various research domains. To demonstrate possible use of MHAD for action recognition, we compare results using the popular Bag-of-Words algorithm adapted to each modality independently with the results of various combinations of modalities using the Multiple Kernel Learning. Our comparative results show that multimodal analysis of human motion yields better action recognition rates than unimodal analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · cited 1×
“These datasets include Hollywood2 by Marszalek et al. [16], UCF50 by Liu et al. (as an extension to UCF YouTube [15]), Olympic Sports by Niebles et al. [22] and HMDB51 by Kuehne et al. [11].”
MethodLearning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Recognizing human actions: a local SVM approach
Schuldt, C., Laptev, I., & Caputo, B. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1334462 · Cited by 3,010
Recognizing human actions: A local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Cited by 2,906
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,541
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440 · cited 1×
“More recently, Torralba and Efros analyzed several examples of popular object recognition databases and showed that uncontrolled data collection (i.e., " in the wild " ) can create biased results which limit the progress of algorithms developed and evaluated against such datasets [28].”
BackgroundHuman activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
A Survey of Computer Vision-Based Human Motion Capture
Moeslund, T. B., & Granum, E. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0897 · Cited by 1,768
Blank, M., Gorelick, L., Shechtman, E., Irani, M., & Basri, R. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.28 · Cited by 1,733
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247813 · Cited by 1,512
Action recognition based on a bag of 3D points
Li, W., Zhang, Z., & Liu, Z. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543273 · Cited by 1,428
Laptev, & Lindeberg (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238378 · Cited by 1,300
Evaluation of local spatio-temporal features for action recognition
Wang, H., Ullah, M. M., Klaser, A., Laptev, I., & Schmid, C. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.124 · Cited by 1,296
Marszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
Sigal, L., Balan, A. O., & Black, M. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,164
Recognizing realistic actions from videos “in the wild”
Liu, J., Luo, J., & Shah, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206744 · Cited by 1,075
A survey of vision-based methods for action representation, segmentation and recognition
Weinland, D., Ronfard, R., & Boyer, E. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.10.002 · Cited by 997
On feature combination for multiclass object classification
Gehler, P., & Nowozin, S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459169 · Cited by 797 · cited 6×
“We then use Multiple Kernel Learning (MKL) [8] to compute the optimal linear combination of multimodal/multi-view similarities for the task of action recognition.”
MethodInfluentialMultiple kernels for object detection
Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459183 · Cited by 757
Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification
Niebles, J. C., Chen, C.-W., & Fei-Fei, L. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15552-9_29 · Cited by 700
(2001) · Cited by 555
Action Recognition from Arbitrary Views using 3D Exemplars
Weinland, D., Boyer, E., & Ronfard, R. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408849 · Cited by 448
Segmenting motion capture data into distinct behaviors
Barbič, J., Safonova, A., Pan, J.-Y., Faloutsos, C., Hodgins, J. K., & Pollard, N. S. (2004) · Graphics Interface · Cited by 441
Documentation Mocap Database HDM05
Müller, M., Röder, T., Clausen, M. V., Eberhardt, B., Krüger, B., & Weber⋆, A. (2007) · Cited by 378
Elliott, D. (2010) · Cited by 312
RGBD-HuDaAct: A color-depth video database for human daily activity recognition
Ni, B., Wang, G., & Moulin, P. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130379 · Cited by 304
Toward Category-Level Object Recognition
(2006) · Lecture Notes in Computer Science · doi:10.1007/11957959 · Cited by 290
Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database
Torre, F. D. L., Hodgins, J. K., Bargteil, A. W., Martin, X., Macey, J., Collado, A., & Beltran, P. (2018) · KiltHub Repository · doi:10.1184/r1/6555020 · Cited by 228
Tenorth, M., Bandouch, J., & Beetz, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457583 · Cited by 214
Dataset Issues in Object Recognition
Ponce, J., Berg, T. L., Everingham, M., Forsyth, D. A., Hebert, M., Lazebnik, S., Marszalek, M., Schmid, C., Russell, B. C., Torralba, A., et al. (2006) · Lecture Notes in Computer Science · doi:10.1007/11957959_2 · Cited by 204
Recognizing realistic actions from videos “in the wild”
Liu, J., Luo, J., & Shah, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvprw.2009.5206744 · Cited by 179
The i3DPost Multi-View and 3D Human Action/Interaction Database
Gkalelis, N., Kim, H., Hilton, A., Nikolaidis, N., & Pitas, I. (2009) · 2009 Conference for Visual Media Production · doi:10.1109/cvmp.2009.19 · Cited by 161
Efficient and robust annotation of motion capture data
Müller, M., Baak, A., & Seidel, H. (2009) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/1599470.1599473 · Cited by 109
Integrating Audio Visual Data for Human Action Detection
Abdullah, L. N., & Noah, S. A. M. (2008) · International Conference on Computer Graphics, Imaging and Visualization (CGIV) · doi:10.1109/cgiv.2008.65 · Cited by 16
Cited by
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139
Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., & Xie, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10451 · Cited by 915
Towards Understanding Action Recognition
Jhuang, H., Gall, J., Zuffi, S., Schmid, C., & Black, M. J. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.396 · Cited by 879
Metadata
- Authors
- Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R.
- Venue
- 2013 IEEE Workshop on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2013-01
- Pages
- 53-60
- ISSN
- 1550-5790 (ISSN-L)
- OpenAlex
- W2086509056
- Cited by
- 484
- References
- 34
- FWCI
- 21.12
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.