A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives
Open accessAbstract
Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced our ability to predict posture directly from videos, which has quickly impacted neuroscience and biology more broadly. In this primer, we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Various user-friendly toolboxes are available to extract the 2D posture of animals from videos, while the user can define which body parts should be estimated.
“Various user-friendly toolboxes are available to extract the 2D posture of animals from videos (Fig. 1d, e), while the user can define which body parts should be estimated (reviewed in ref. 76).”
From Perspectives in machine learning for wildlife conservation (2022) · cites this paper as [76]
Modern ML has rapidly changed laboratory-based non-invasive observation of animals.
“In recent years, modern ML has rapidly changed laboratory-based non-invasive observation of animals76,105.”
From Perspectives in machine learning for wildlife conservation (2022) · cites this paper as [76]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Fiji: an open-source platform for biological-image analysis
Schindelin, J., Arganda-Carreras, I., Frise, E., Kaynig, V., Longair, M., Pietzsch, T., Preibisch, S., Rueden, C., Saalfeld, S., Schmid, B., et al. (2012) · Nature Methods · doi:10.1038/nmeth.2019 · Cited by 66,890 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,369 · see citing works
Momentum Contrast for Unsupervised Visual Representation LearningPreprint
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. B. (2019) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1911.05722 · Cited by 16,250 · see citing works
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341010 · Cited by 14,474 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 1× · see citing works
“…are (loosely) inspired by neurons in the visual pathway and are designed to be robust to changes in illumination, and translations; typical feature representations are Scale Invariant Feature Transform (SIFT; 31), Histogram of Gradients (HOG; 30) or Speeded Up Robust Features (SURF; 32).”
BackgroundMomentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117 · see citing works
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898 · see citing works
(2016) · Cited by 11,679 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582 · see citing works
Biomechanics and motor control of human movement
Winter, D. A. (1990) · Choice Reviews Online · doi:10.5860/choice.28-2135 · Cited by 8,496 · see citing works
scikit-image: image processing in Python
(2014) · doi:10.7717/peerj.453 · Cited by 6,794 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
Biomechanics and Motor Control of Human Movement
Winter, D. A. (2009) · Wiley eBooks · doi:10.1002/9780470549148 · Cited by 6,162 · see citing works
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984 · see citing works
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · see citing works
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · see citing works
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · see citing works
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · see citing works
Self-citation2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
(2018) · doi:10.1109/cvpr40276.2018 · Cited by 4,769 · see citing works
Visual perception of biological motion and a model for its analysis
Johansson, G. (1973) · Perception & Psychophysics · doi:10.3758/bf03212378 · Cited by 4,547 · see citing works
2008 IEEE Conference on Computer Vision and Pattern Recognition
(2008) · doi:10.1109/cvpr12618.2008 · Cited by 4,263 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
(2013) · Cited by 3,740 · see citing works
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664 · see citing works
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.214 · Cited by 3,288 · see citing works
Docker: lightweight Linux containers for consistent development and deployment
Merkel, D. (2014) · Linux journal · Cited by 3,277 · see citing works
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.471 · Cited by 2,699 · see citing works
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,546 · see citing works
Visualizing the Loss Landscape of Neural NetsPreprint
Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1712.09913 · Cited by 2,497 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,460 · see citing works
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393 · see citing works
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369 · see citing works
Simple Baselines for Human Pose Estimation and Tracking
Xiao, B., Wu, H., & Wei, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_29 · Cited by 2,233 · see citing works
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00982 · Cited by 2,194 · see citing works
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997 · see citing works
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., & Bengio, Y. (2018) · arXiv · doi:10.48550/arxiv.1808.06670 · Cited by 1,991 · see citing works
IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2001)
Hillman, P., Hannah, J. M., & Renshaw, D. (2001) · Cited by 1,967 · see citing works
CellProfiler 3.0: Next-generation image processing for biology
McQuin, C., Goodman, A., Chernyshev, V. S., Kamentsky, L. D., Cimini, B. A., Karhohs, K. W., Doan, M., Ding, L., Rafelski, S. M., Thirstrup, D. J., et al. (2018) · PLoS Biology · doi:10.1371/journal.pbio.2005970 · Cited by 1,835 · see citing works
Terrestrial animal tracking as an eye on life and planet
Kays, R., Crofoot, M. C., Jetz, W., & Wikelski, M. (2015) · Science · doi:10.1126/science.aaa2478 · Cited by 1,638 · see citing works
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557 · see citing works
DensePose: Dense Human Pose Estimation in the Wild
Güler, R. A., Neverova, N., & Kokkinos, I. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00762 · Cited by 1,496 · see citing works
Using DeepLabCut for 3D markerless pose estimation across species and behaviors
Nath, T., Mathis, A., Chen, A. C., Patel, A., Bethge, M., & Mathis, M. W. (2019) · Nature Protocols · doi:10.1038/s41596-019-0176-0 · Cited by 1,476 · see citing works
Self-citationA Simple Yet Effective Baseline for 3d Human Pose Estimation
Martínez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,468 · see citing works
DensePose: Dense Human Pose Estimation in the Wild.
Güler, R. A., Neverova, N., & Kokkinos, I. (2018) · UCL Discovery (University College London) · Cited by 1,421 · see citing works
Neuroscience Needs Behavior: Correcting a Reductionist Bias
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017) · Neuron · doi:10.1016/j.neuron.2016.12.041 · Cited by 1,358 · see citing works
Do Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325 · see citing works
Joint Training of a Convolutional Network and a Graphical Model for Human Pose EstimationPreprint
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 1,317 · see citing works
wav2vec: Unsupervised Pre-Training for Speech Recognition
Schneider, S., Baevski, A., Collobert, R., & Auli, M. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1873 · Cited by 1,290 · see citing works
Self-citationExploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224 · see citing works
A guide to convolution arithmetic for deep learningPreprint
Dumoulin, V., & Visin, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07285 · Cited by 1,223 · see citing works
DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model
Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., & Schiele, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_3 · Cited by 1,191 · see citing works
Rethinking ImageNet Pre-Training
He, K., Girshick, R., & Dollár, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00502 · Cited by 1,164 · see citing works
Visualizing the Loss Landscape of Neural Nets
Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2017) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000461393 · Cited by 1,148 · see citing works
Ilastik: Interactive learning and segmentation toolkit
Sommer, C., Straehle, C., Köthe, U., & Hamprecht, F. A. (2011) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2011.5872394 · Cited by 1,145 · see citing works
Taskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128 · see citing works
Recent advances in deep learning for object detection
Wu, X., Sahoo, D., & Hoi, S. C. H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.085 · Cited by 1,047 · see citing works
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T. S., & Zhang, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00543 · Cited by 1,023 · see citing works
Data-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014 · see citing works
Lu, T., & O’Connor, D. J. (1999) · Journal of Biomechanics · doi:10.1016/s0021-9290(98)00158-4 · Cited by 932 · see citing works
Vision-based human motion analysis: An overview
Poppe, R. W. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.016 · Cited by 841 · see citing works
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805 · see citing works
Mapping Sub-Second Structure in Mouse Behavior
Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., Abraira, V. E., Adams, R. P., & Datta, S. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.031 · Cited by 760 · see citing works
(2009) · Cited by 701 · see citing works
Recognition in Terra Incognita
Beery, S., Van Horn, G., & Perona, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_28 · Cited by 679 · see citing works
Through-Wall Human Pose Estimation Using Radio Signals
Zhao, M., Li, T., Alsheikh, M. A., Tian, Y., Zhao, H., Torralba, A. B., & Katabi, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00768 · Cited by 676 · see citing works
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity FieldsPreprint
Cao, Z., Hidalgo, G., Simon, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08008 · Cited by 672 · see citing works
Fast animal pose estimation using deep neural networks
Pereira, T., Aldarondo, D. E., Willmore, L., Kislin, M., Wang, S. S. ‐., Murthy, M., & Shaevitz, J. W. (2018) · Nature Methods · doi:10.1038/s41592-018-0234-5 · Cited by 659 · see citing works
JAABA: Interactive machine learning for automatic annotation of animal behavior
Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S., & Branson, K. (2013) · Nature Methods · doi:10.1038/nmeth.2281 · Cited by 644 · see citing works
Survey of spiking in the mouse visual system reveals functional hierarchy
Siegle, J. H., Jia, X., Durand, S., Gale, S. D., Bennett, C., Graddis, N., Heller, G. R., Ramirez, T. K., Choi, H., Luviano, J., et al. (2021) · Nature · doi:10.1038/s41586-020-03171-x · Cited by 637 · see citing works
Observing the unwatchable through acceleration logging of animal behavior
Brown, D. D., Kays, R., Wikelski, M. C., Wilson, R. P., & Klimley, A. P. (2013) · Animal Biotelemetry · doi:10.1186/2050-3385-1-20 · Cited by 627 · see citing works
Camera traps in animal ecology : methods and analyses
O’Connell, A. F., Nichols, J. D., & Kāranta, K. U. (2011) · Cited by 615 · see citing works
Bonsai: an event-based framework for processing and controlling data streams
Lopes, G., Bonacchi, N., Frazão, J., Neto, J. P., Atallah, B. V., Soares, S., MOREIRA, L. M. A. G., Matias, S., Itskov, P. M., Correia, P. A., et al. (2015) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2015.00007 · Cited by 602 · see citing works
Camomilla, V., Bergamini, E., Fantozzi, S., & Vannozzi, G. (2018) · Sensors · doi:10.3390/s18030873 · Cited by 601 · see citing works
Rigid body motion calculated from spatial co-ordinates of markers
Spoor, C. W., & Veldpaus, F. E. (1980) · Journal of Biomechanics · doi:10.1016/0021-9290(80)90020-2 · Cited by 585 · see citing works
PifPaf: Composite Fields for Human Pose Estimation
Kreiss, S., Bertoni, L., & Alahi, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01225 · Cited by 574 · see citing works
3D Human Pose Estimation = 2D Pose Estimation + Matching
Chen, C.-H., & Ramanan, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.610 · Cited by 569 · see citing works
Mapping the stereotyped behaviour of freely moving fruit flies
Berman, G. J., Choi, D. M., Bialek, W., & Shaevitz, J. W. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0672 · Cited by 565 · see citing works
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547 · see citing works
Using Pre-Training Can Improve Model Robustness and UncertaintyPreprint
Hendrycks, D., Lee, K., & Mazeika, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09960 · Cited by 531 · see citing works
Open Ephys: an open-source, plugin-based platform for multichannel electrophysiology
Siegle, J. H., Cuevas-López, A., Patel, Y., Abramov, K., Ohayon, S., & Voigts, J. (2017) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa5eea · Cited by 530 · see citing works
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks
Cai, Y., Ge, L., Liu, J., Cai, J., Cham, T., Yuan, J., & Magnenat‐Thalmann, N. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00236 · Cited by 522 · see citing works
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
Tomè, D., Russell, C., & Agapito, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.603 · Cited by 505 · see citing works
Pliocene footprints in the Laetolil Beds at Laetoli, northern Tanzania
Leakey, M. D., & Hay, R. L. (1979) · Nature · doi:10.1038/278317a0 · Cited by 493 · see citing works
Gleiss, A. C., Wilson, R. P., & Shepard, E. L. C. (2010) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2010.00057.x · Cited by 476 · see citing works
Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv · doi:10.48550/arxiv.1906.05849 · Cited by 465 · see citing works
Computational Neuroethology: A Call to Action
Datta, S. R., Anderson, D. J., Branson, K., Perona, P., & Leifer, A. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.038 · Cited by 453 · see citing works
Deep learning tools for the measurement of animal behavior in neuroscience
Mathis, M. W., & Mathis, A. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.10.008 · Cited by 450 · see citing works
Self-citationBiomechanical and physiological aspects of legged locomotion in humans
Saibene, F., & Minetti, A. E. (2003) · European Journal of Applied Physiology · doi:10.1007/s00421-002-0654-9 · Cited by 440 · see citing works
A computer vision for animal ecology
Weinstein, B. G. (2017) · Journal of Animal Ecology · doi:10.1111/1365-2656.12780 · Cited by 435 · see citing works
vq-wav2vec: Self-Supervised Learning of Discrete Speech RepresentationsPreprint
Baevski, A., Schneider, S., & Auli, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05453 · Cited by 431 · see citing works
Self-citationAn efficient framework for learning sentence representations
Logeswaran, L., & Lee, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02893 · Cited by 427 · see citing works
Learning effective human pose estimation from inaccurate annotation
Johnson, S., & Everingham, M. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995318 · Cited by 417 · see citing works
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020) · neural information processing systems · Cited by 386 · see citing works
3D Menagerie: Modeling the 3D Shape and Pose of Animals
Zuffi, S., Kanazawa, A., Jacobs, D. W., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.586 · Cited by 381 · see citing works
(2019) · neural information processing systems · Cited by 350 · see citing works
Estimating Three-Dimensional Orientation of Human Body Parts by Inertial/Magnetic Sensing
Sabatini, A. M. (2011) · Sensors · doi:10.3390/s110201489 · Cited by 349 · see citing works
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
Michaelis, C., Mitzkus, B., Geirhos, R., Rusak, E., Bringmann, O., Ecker, A. S., Bethge, M., & Brendel, W. (2019) · arXiv · doi:10.48550/arxiv.1907.07484 · Cited by 324 · see citing works
Anipose: A toolkit for robust markerless 3D pose estimation
Karashchuk, P., Rupp, K. L., Dickinson, E. S., Walling-Bell, S., Sanders, E., Azim, E., Brunton, B. W., & Tuthill, J. C. (2021) · Cell Reports · doi:10.1016/j.celrep.2021.109730 · Cited by 324 · see citing works
Systematic Review on Kinematic Assessments of Upper Limb Movements After Stroke
Schwarz, A., Kanzler, C. M., Lambercy, O., Luft, A. R., & Veerbeek, J. M. (2019) · Stroke · doi:10.1161/strokeaha.118.023531 · Cited by 315 · see citing works
Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs
von Marcard, T., Rosenhahn, B., Black, M. J., & Pons‐Moll, G. (2017) · Computer Graphics Forum · doi:10.1111/cgf.13131 · Cited by 309 · see citing works
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry
Kocabas, M., Karagoz, S., & Akbaş, E. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00117 · Cited by 297 · see citing works
Multi-Task Self-Supervised Learning for Robust Speech Recognition
Ravanelli, M., Zhong, J., Pascual, S., Świętojański, P., Monteiro, J., Trmal, J., & Bengio, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053569 · Cited by 276 · see citing works
Maceira-Elvira, P., Popa, T., Schmid, A., & Hummel, F. C. (2019) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-019-0612-y · Cited by 275 · see citing works
ArtTrack: Articulated Multi-Person Tracking in the Wild
Insafutdinov, E., Andriluka, M., Pishchulin, L., Tang, S., Levinkov, E., Andres, B., & Schiele, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.142 · Cited by 246 · see citing works
Sturman, O., von Ziegler, L., Schläppi, C., Akyol, F., Privitera, M., Slominski, D., Grimm, C., Thieren, L., Zerbi, V., Grewe, B. F., et al. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-0776-y · Cited by 243 · see citing works
Automated markerless pose estimation in freely moving macaques with OpenMonkeyStudio
Bala, P., Eisenreich, B. R., Yoo, S. B. M., Hayden, B. Y., Park, H. S., & Zimmermann, J. (2020) · Nature Communications · doi:10.1038/s41467-020-18441-5 · Cited by 238 · see citing works
de Chaumont, F., Ey, E., Torquet, N., Lagache, T., Dallongeville, S., Imbert, A., Legou, T., Sourd, A. L., Fauré, P., Bourgeron, T., et al. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0396-1 · Cited by 235 · see citing works
Composite Pain Biomarker Signatures for Objective Assessment and Effective Treatment
Tracey, I., Woolf, C. J., & Andrews, N. A. (2019) · Neuron · doi:10.1016/j.neuron.2019.02.019 · Cited by 231 · see citing works
Reverse-engineering the locomotion of a stem amniote
Nyakatura, J. A., Melo, K., Horvat, T., Karakasiliotis, K., Allen, V. R., Andikfar, A., Andrada, E., Arnold, P., Lauströer, J., Hutchinson, J. R., et al. (2019) · Nature · doi:10.1038/s41586-018-0851-2 · Cited by 224 · see citing works
Identifying behavioral structure from deep variational embeddings of animal motion
Luxem, K., Mocellin, P., Fuhrmann, F., Kürsch, J., Miller, S. R., Palop, J. J., Remy, S., & Bauer, P. (2022) · Communications Biology · doi:10.1038/s42003-022-04080-7 · Cited by 218 · see citing works
Ceseracciu, E., Sawacha, Z., & Cobelli, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0087640 · Cited by 210 · see citing works
Monocular Real-Time Hand Shape and Motion Capture Using Multi-Modal Data
Zhou, Y., Habermann, M., Xu, W., Habibie, I., Theobalt, C., & Xu, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00539 · Cited by 208 · see citing works
Biomechanics of predator–prey arms race in lion, zebra, cheetah and impala
Wilson, A. M., Hubel, T. Y., Wilshin, S. D., Lowe, J. C., Lorenc, M., Dewhirst, O. P., Bartlam‐Brooks, H. L. A., Diack, R., Bennitt, E., Golabek, K. A., et al. (2018) · Nature · doi:10.1038/nature25479 · Cited by 201 · see citing works
Self-supervised Learning of Motion CapturePreprint
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01337 · Cited by 198 · see citing works
Real-time, low-latency closed-loop feedback using markerless posture tracking
Kane, G. A., Lopes, G., Saunders, J. L., Mathis, A., & Mathis, M. W. (2020) · eLife · doi:10.7554/elife.61909 · Cited by 194 · see citing works
Self-citationNeurodata Without Borders: Creating a Common Data Format for Neurophysiology
Teeters, J. L., Godfrey, K. B., Young, R., Dang, C., Friedsam, C., Wark, B., Asari, H., Peron, S., Li, N., Peyrache, A., et al. (2015) · Neuron · doi:10.1016/j.neuron.2015.10.025 · Cited by 193 · see citing works
Benchmarking and Error Diagnosis in Multi-instance Pose Estimation
Ronchi, M. R., & Perona, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.48 · Cited by 187 · see citing works
All the light that we can see: a new era in miniaturized microscopy
Aharoni, D. B., Khakh, B. S., Silva, A. J., & Golshani, P. (2018) · Nature Methods · doi:10.1038/s41592-018-0266-x · Cited by 185 · see citing works
Günel, S., Rhodin, H., Morales, D., Campagnolo, J., Ramdya, P., & Fua, P. (2019) · eLife · doi:10.7554/elife.48571 · Cited by 167 · see citing works
Engineering a Less Artificial Intelligence
Sinz, F. H., Pitkow, X., Reimer, J., Bethge, M., & Tolias, A. S. (2019) · Neuron · doi:10.1016/j.neuron.2019.08.034 · Cited by 164 · see citing works
Two Distinct Types of Eye-Head Coupling in Freely Moving Mice
Meyer, A. F., O'Keefe, J. M., & Poort, J. (2020) · Current Biology · doi:10.1016/j.cub.2020.04.042 · Cited by 162 · see citing works
Pretraining boosts out-of-domain robustness for pose estimation
Mathis, A., Biasi, T., Schneider, S., Yüksekgönül, M., Rogers, B., Bethge, M., & Mathis, M. W. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00190 · Cited by 143 · see citing works
Self-citationMoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation
Jain, A., Tompson, J., LeCun, Y., & Bregler, C. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16808-1_21 · Cited by 142 · see citing works
Advanced Neurotechnologies for the Restoration of Motor Function
Micera, S., Caleo, M., Chisari, C., Hummel, F. C., & Pedrocchi, A. (2020) · Neuron · doi:10.1016/j.neuron.2020.01.039 · Cited by 140 · see citing works
Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture From Images “In the Wild”
Zuffi, S., Kanazawa, A., Berger-Wolf, T., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00546 · Cited by 139 · cited 1× · see citing works
“For some applications (like modeling soft tissue or Neuron 108, October 14, 2020 59 volume), full surface reconstructions are needed, and this area has seen tremendous progress in recent years (G€uler et al., 2018; Sanakoyeu et al., 2020; Zuffi et al., 2019).”
BackgroundStep by step: reconstruction of terrestrial animal movement paths by dead-reckoning
Bidder, O. R., Walker, J. S., Jones, M. W., Holton, M. D., Urgé, P., Scantlebury, D. M., Marks, N. J., Magowan, E. A., Maguire, I. E., & Wilson, R. P. (2015) · Movement Ecology · doi:10.1186/s40462-015-0055-4 · Cited by 138 · see citing works
Self-supervised learning of motion capture
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · neural information processing systems · Cited by 123 · see citing works
MacaquePose: A Novel “In the Wild” Macaque Monkey Pose Dataset for Markerless Motion Capture
Labuguen, R. T., Matsumoto, J., Negrete, S. B., Nishimaru, H., Nishijo, H., Takada, M., Go, Y., Inoue, K., & Shibata, T. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2020.581154 · Cited by 120 · see citing works
An IMU-to-Body Alignment Method Applied to Human Gait Analysis
Vargas-Valencia, L., Elías, A., Rocón, E., Bastos-Filho, T., & Frizera, A. (2016) · Sensors · doi:10.3390/s16122090 · Cited by 116 · see citing works
EventCap: Monocular 3D Capture of High-Speed Human Motions Using an Event Camera
Xu, L., Xu, W., Golyanik, V., Habermann, M., Fang, L., & Theobalt, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00502 · Cited by 116 · see citing works
2013 IEEE International Symposium on Biomedical Imaging: From Nano to Macro
(2012) · IEEE Pulse · doi:10.1109/mpul.2012.2210847 · Cited by 114 · see citing works
Human movement analysis: The soft tissue artefact issue
Camomilla, V., Dumas, R., & Cappozzo, A. (2017) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2017.09.001 · Cited by 108 · see citing works
Arac, A., Zhao, P., Dobkin, B. H., Carmichael, S. T., & Golshani, P. (2019) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2019.00020 · Cited by 108 · see citing works
Fan, B., Li, Q., & Liu, T. (2017) · Sensors · doi:10.3390/s18010076 · Cited by 106 · see citing works
A novel biomechanical approach for animal behaviour recognition using accelerometers
Chakravarty, P., Cozzi, G., Özgül, A., & Aminian, K. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13172 · Cited by 103 · see citing works
Rethinking ImageNet Pre-trainingPreprint
He, K., Girshick, R., & Dollár, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08883 · Cited by 97 · see citing works
Begon, M., Andersen, M. S., & Dumas, R. (2017) · Journal of Biomechanical Engineering · doi:10.1115/1.4038741 · Cited by 95 · see citing works
Video can make behavioural science more reproducible
Gilmore, R. O., & Adolph, K. E. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0128 · Cited by 81 · see citing works
Fusing Visual and Inertial Sensors with Semantics for 3D Human Pose Estimation
Gilbert, A., Trumble, M., Malleson, C., Hilton, A., & Collomosse, J. P. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1118-y · Cited by 77 · see citing works
Berger, M., Agha, N. S., & Gail, A. (2020) · eLife · doi:10.7554/elife.51322 · Cited by 71 · see citing works
Lebleu, J., Gosseye, T. P., Detrembleur, C., Mahaudens, P., Cartiaux, O., & Penta, M. (2020) · Sensors · doi:10.3390/s20030715 · Cited by 69 · see citing works
Movement science needs different pose tracking algorithms
Seethapathi, N., Wang, S., Saluja, R., Blohm, G., & Kording, K. P. (2019) · arXiv · doi:10.48550/arxiv.1907.10226 · Cited by 64 · see citing works
The Future Is Open: Open-Source Tools for Behavioral Neuroscience Research
White, S. R., Amarante, L. M., Kravitz, A. V., et al. (2019) · eNeuro · doi:10.1523/eneuro.0223-19.2019 · Cited by 59 · see citing works
List, R., Postolka, B., Schütz, P., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185952 · Cited by 58 · see citing works
A kinematic synergy for terrestrial locomotion shared by mammals and birds
Catavitello, G., Ivanenko, Y. P., & Lacquaniti, F. (2018) · eLife · doi:10.7554/elife.38190 · Cited by 58 · see citing works
Brain-wide representations of ongoing behavior: a universal principle?
Kaplan, H. S., & Zimmer, M. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.02.008 · Cited by 57 · see citing works
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces
Khan, M. H., McDonagh, J., Khan, S. H., Shahabuddin, M., Arora, A., Khan, F. S., Shao, L., & Tzimiropoulos, G. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00697 · Cited by 54 · see citing works
Self-supervised Keypoint Correspondences for Multi-person Pose Estimation and Tracking in Videos
Rafi, U., Doering, A., Leibe, B., & Gall, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_3 · Cited by 54 · see citing works
MONET: Multiview Semi-Supervised Keypoint Detection via Epipolar Divergence
Yao, Y., Jafarian, Y., & Park, H. S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00084 · Cited by 51 · see citing works
Methodological concerns using intra‐cortical pins to measure tibiofemoral kinematics
Ramsey, D. K., Wretenberg, P. F., Benoit, D. L., et al. (2003) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-003-0388-1 · Cited by 50 · see citing works
Madhav, M. S., & Cowan, N. J. (2020) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-060117-104856 · Cited by 45 · see citing works
Ebina, T., Obara, K., Watakabe, A., Masamizu, Y., Terada, S.-I., Matoba, R., Takaji, M., Hatanaka, N., Nambu, A., Mizukami, H., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903445116 · Cited by 43 · see citing works
Sparse Inertial PoserPreprint
von MarcardT., RosenhahnB., BlackM., J., & Pons-MollG. (2017) · arXiv (Cornell University) · Cited by 42 · see citing works
Understanding Human Motion: A Historic Review
Klette, R., & Tee, G. J. (2007) · Computational imaging and vision · doi:10.1007/978-1-4020-6693-1_1 · Cited by 40 · see citing works
Manual dexterity of mice during food-handling involves the thumb and a set of fast basic movements
Barrett, J. M., Tapies, M. G. R., & Shepherd, G. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0226774 · Cited by 40 · see citing works
Wireless inertial measurement of head kinematics in freely-moving rats
Pasquet, M. O., Tihy, M., Gourgeon, A., Pompili, M. N., Godsil, B. P., Léna, C., & Dugué, G. P. (2016) · Scientific Reports · doi:10.1038/srep35689 · Cited by 38 · see citing works
Scientific Community Image Forum: A discussion forum for scientific image software
Rueden, C., Ackerman, J., Arena, E. T., et al. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000340 · Cited by 37 · see citing works
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN SupervisionPreprint
Mehta, D., Rhodin, H., Casas, D., et al. (2016) · arXiv (Cornell University) · Cited by 34 · see citing works
On the inference speed and video-compression robustness of DeepLabCutPreprint
Mathis, A., & Warren, R. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/457242 · Cited by 34 · see citing works
Self-citationFreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D SpacesPreprint
Zimmermann, C., Schneider, A., Alyahyay, M., Brox, T., & Diester, I. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.02.27.967620 · Cited by 31 · see citing works
An Analysis of Pre-Training on Object DetectionPreprint
Li, H., Singh, B., Najibi, M., Wu, Z., & Davis, L. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05871 · Cited by 30 · see citing works
Mechanical principles of dynamic terrestrial self-righting using wings
Li, C., Kessens, C. C., Fearing, R. S., et al. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1372213 · Cited by 28 · see citing works
A Matlab-based toolbox for characterizing behavior of rodents engaged in string-pulling
Inayat, S., Singh, S., Ghasroddashti, A., et al. (2020) · eLife · doi:10.7554/elife.54540 · Cited by 28 · see citing works
Integrating XMALab and DeepLabCut for high-throughput XROMM
Laurence-Chasen, J. D., Manafzadeh, A. R., Hatsopoulos, N. G., Ross, C. F., & Arce‐McShane, F. I. (2020) · Journal of Experimental Biology · doi:10.1242/jeb.226720 · Cited by 28 · see citing works
Contrasting action and posture coding with hierarchical deep neural network models of proprioception
Sandbrink, K. J., Mamidanna, P., Michaelis, C., Bethge, M., Mathis, M. W., & Mathis, A. (2023) · eLife · doi:10.7554/elife.81499 · Cited by 27 · see citing works
Self-citationMoDeep: A Deep Learning Framework Using Motion Features for Human Pose EstimationPreprint
Jain, A., Tompson, J., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7963 · Cited by 26 · see citing works
Simultaneous Identification and Tracking of Multiple People Using Video and IMUs
Henschel, R. D., von Marcard, T., & Rosenhahn, B. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00106 · Cited by 26 · see citing works
OptiFlex: video-based animal pose estimation using deep learning enhanced by optical flowPreprint
Liu, X., Yu, S., Flierman, N. A., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.04.025494 · Cited by 25 · see citing works
Longo, A., Haid, T. H., Meulenbroek, R. G. J., et al. (2018) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2018.11.031 · Cited by 24 · see citing works
Monocular 3D Human Pose Estimation In The Wild Using Improved CNN SupervisionPreprint
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09813 · Cited by 22 · see citing works
Behavioral and Neural Variability of Naturalistic Arm Movements
Peterson, S. M., Singh, S. H., Wang, N. X. R., Rao, R. P. N., & Brunton, B. W. (2021) · eNeuro · doi:10.1523/eneuro.0007-21.2021 · Cited by 20 · see citing works
ArtTrack: Articulated Multi-person Tracking in the WildPreprint
Insafutdinov, E., Andriluka, M., Pishchulin, L., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01465 · Cited by 17 · see citing works
Smale, K. B., Potvin, B. M., Shourijeh, M. S., et al. (2017) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2017.06.049 · Cited by 15 · see citing works
Transferring Dense Pose to Proximal Animal Classes
Sanakoyeu, A., Khalidov, V., McCarthy, M. S., Vedaldi, A., & Neverova, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00528 · Cited by 13 · see citing works
FollowMeUp Sports: New Benchmark for 2D Human Keypoint Recognition
Huang, Y., Sun, B., Kan, H., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-31726-3_10 · Cited by 12 · see citing works
Data architecture for a large-scale neuroscience collaborationPreprint
Bonacchi, N., Chapuis, G., Churchland, A., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/827873 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- [17]17. Gary Kane, Gonçalo Lopes, Jonny L. Saunders, Alexander Mathis, and Mackenzie Mathis. Real-time deeplabcut for closed-loop feedback based on posture. bioRxiv , 2020.
- [39]39. Alexander Mathis, Mert Yüksekgönül, Byron Rogers, Matthias Bethge, and Mackenzie W Mathis. Pretraining boosts outof-domain robustness for pose estimation. arXiv preprint arXiv:1909.11229 , 2019. · arXiv
- [66]66. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, highperformance deep learning library. In Advances in neural information processing systems , pages 8026-8037, 2019.
- [95]95. Joshua H Siegle, Xiaoxuan Jia, Séverine Durand, Sam Gale, Corbett Bennett, Nile Graddis, Greggory Heller, Tamina K Ramirez, Hannah Choi, Jennifer A Luviano, et al. A survey of spiking activity reveals a functional hierarchy of mouse corticothalamic visual areas. bioRxiv , page 805010, 2019.
- [139]139. Alexander B Wiltschko, Matthew J Johnson, Giuliano Iurilli, Ralph E Peterson, Jesse M Katon, Stan L Pashkovski, Victoria E Abraira, Ryan P Adams, and Sandeep Robert Datta. Mapping subsecond structure in mouse behavior. Neuron , 88(6):1121-1135, 2015.
- [146]146. Kai J Sandbrink, Pranav Mamidanna, Claudio Michaelis, Mackenzie Weygandt Mathis, Matthias Bethge, and Alexander Mathis. Task-driven hierarchical deep neural networkmodels of the proprioceptive pathway. bioRxiv , 2020.
- [160]160. Mirco Ravanelli, Jianyuan Zhong, Santiago Pascual, Pawel Swietojanski, Joao Monteiro, Jan Trmal, and Yoshua Bengio. Multitask self-supervised learning for robust speech recognition. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 6989-6993. IEEE, 2020.
- [170]170. Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pretraining can improve model robustness and uncertainty. In ICML , 2019.
- [173]173. Cody Coleman, Deepak Narayanan, Daniel Kang, Tian Zhao, Jian Zhang, Luigi Nardi, Peter Bailis, Kunle Olukotun, Chris Ré, and Matei Zaharia. Dawnbench: An end-to-end deep learning benchmark and competition. Neural Information Processing Systems Workshops , 2017.
Cited by
Sorted by citations
Perspectives in machine learning for wildlife conservation
Tuia, D., Kellenberger, B., Beery, S., Costelloe, B. R., Zuffi, S., Risse, B., Mathis, A., Mathis, M. W., van Langevelde, F., Burghardt, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-27980-y · Cited by 740 · see citing works
Self-citationAnipose: A toolkit for robust markerless 3D pose estimation
Karashchuk, P., Rupp, K. L., Dickinson, E. S., Walling-Bell, S., Sanders, E., Azim, E., Brunton, B. W., & Tuthill, J. C. (2021) · Cell Reports · doi:10.1016/j.celrep.2021.109730 · Cited by 324 · see citing works
Perspectives on Individual Animal Identification from Biology and Computer Vision
Vidal, M., Wolf, N., Rosenberg, B., Harris, B. P., & Mathis, A. (2021) · Integrative and Comparative Biology · doi:10.1093/icb/icab107 · Cited by 104 · see citing works
Self-citationNeural circuits regulating prosocial behaviors
Walsh, J. J., Christoffel, D. J., & Malenka, R. C. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-022-01348-8 · Cited by 72 · see citing works
AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild
Joska, D., Clark, L., Muramatsu, N., Jericevich, R., Nicolls, F., Mathis, A., Mathis, M. W., & Patel, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561338 · Cited by 66 · see citing works
Self-citation
- Authors
- Mathis, A., Schneider, S., Lauer, J., & Mathis, M. W.
- Affiliations
- Harvard University (US)
- Corresponding author
- Mathis, Alexander
- Mathis, Mackenzie Weygandt
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2020-10
- Volume
- 108
- Issue
- 1
- Pages
- 44-65
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W3081605501
- Cited by
- 231
- References
- 202
- Citations by year
- 2026: 18 · 2025: 38 · 2024: 50 · 2023: 40 · 2022: 49 · 2021: 32 · 2020: 4
- FWCI
- 2.54
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- Chan Zuckerberg Initiative
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Keywords
- motion capture, deep learning, pose estimation, animal behavior
- MeSH terms
- Deep Learning, Algorithms, Animals, Humans, Motion, Motor Activity, Movement, Video Recording, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review