The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
End-to-end vision-based detection, tracking and activity analysis of earthmoving equipment filmed at ground level
Open accessAbstract
This paper presents a new benchmark dataset for validating vision-based methods that automatically identifies visually distinctive working activities of excavators and dump trucks from individual frames of a video sequence. Our dataset consists of 10 videos of interacting pairs of construction equipment filmed at ground level with accompanying ground truth annotations. These annotations consist of per-equipment and per-frame equipment bounding boxes that also have associated identities and activity labels. Our videos depict an excavator interacting with 1 or more dump trucks. We also propose a deep learning-based method for detecting and tracking objects based on Convolutional Neural Networks (CNNs). The tracking trajectories are fed into a Hidden Markov Model (HMM) that automatically discovers and assigns activity labels for any observed object. Our HMM method leverages trajectories to train a Gaussian Mixture Model (GMM) with which we estimate the probability density function of each activity using Support Vector Machine (SVM) classifiers. The proposed HMM also models activity duration and the transition between activities. We show that our method can accurately distinguish between individual equipment working activities. Results show 97.43% detection Average Precision (AP) for excavators and 75.29% AP for dump trucks, as well as cross-category tracking accuracy of 81.94% and tracking precision of 87.45%. Separate experiment results show activity analysis results of 86.8% accuracy for excavators and 88.5% for dump trucks. Our results show that our method can accurately conduct activity analysis and can be fused with methods that detect motion trajectories to scale to the needs of practical applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Object Detection from Video Tubelets with Convolutional Neural Networks
Kang, K., Ouyang, W., Li, H., & Wang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.95 · Cited by 386
Coupled Hidden Semi Markov Models for Activity Recognition
Natarajan, P., & Nevatia, R. (2007) · 2007 IEEE Workshop on Motion and Video Computing (WMVC'07) · doi:10.1109/wmvc.2007.12 · Cited by 144
Cited by
Machine learning in construction: From shallow to deep learning
Xu, Y., Zhou, Y., Sekula, P., & Ding, L. (2021) · Developments in the Built Environment · doi:10.1016/j.dibe.2021.100045 · Cited by 316
ANN-based automated scaffold builder activity recognition through wearable EMG and IMU sensors
Bangaru, S. S., Wang, C., Busam, S. A., & Aghazadeh, F. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103653 · Cited by 114
Metadata
- Authors
- Roberts, D., & Golparvar-Fard, M.
- Venue
- Automation in Construction
- Publisher
- Elsevier BV
- Published
- 2019-09
- Volume
- 105
- Pages
- 102811
- ISSN
- 0926-5805 (ISSN-L)
- OpenAlex
- W2944936691
- Cited by
- 178
- References
- 7
- FWCI
- 9.38
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.