Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Recording workers’ activities is an important, but burdensome, management task for site supervisors. The last decade has seen a growing trend toward vision‐based activity recognition. However, recognizing workers’ activities in far‐field surveillance videos is understudied. This study proposes a hierarchical statistical method for recognizing workers’ activities in far‐field surveillance videos. The method consists of two steps. First, a deep action recognition method was used to recognize workers’ actions, and a new fusion strategy was proposed to consider the characteristics of far‐field surveillance videos. The deep action recognition method with the new fusion strategy has achieved the comparable performance (0.84 average accuracy) on the far‐field surveillance data set in contrast to the original method on the public data sets. Second, a Bayesian nonparametric hidden semi‐Markov model was innovatively used to model and infer workers’ activities based on action sequences. The latent states of the fitted Bayesian model captured workers’ activities in terms of state duration distributions and state transition distributions, which are indispensable for understanding workers’ time allocation. It has been preliminarily illustrated that the activity information learned by the Bayesian model possesses the potential to implement objective work sampling, personal physical fatigue monitoring, trade‐level health risk assessment, and process‐based quality control. Also, the limitations of this study are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Simple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669 · cited 3×
“The representative methods in the first category are constructed based on hand-crafted local features such as dense trajectories (Wang and Schmid, 2011) and improved dense trajectories (Wang and Schmid, 2013).”
BackgroundMethodInfluentialAction recognition with trajectory-pooled deep-convolutional descriptors
Wang, L., Qiao, Y., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299059 · Cited by 1,194 · cited 5×
“In the original work of the TSNs in Wang et al. (2015), the temporal CNN performs slightly better than the spatial CNN (0.87 vs. 0.84) with the public data set UCF-101.”
BackgroundMethodInfluentialPsychophysiological investigation of vigilance decrement: boredom or cognitive fatigue?
Pattyn, N., Neyt, X., Henderickx, D., & Soetens, E. (2008) · Physiology & Behavior · doi:10.1016/j.physbeh.2007.09.016 · Cited by 422 · cited 1×
“The vigilance decrement, which can be described as a slowing in reaction times or an increase in error rates (Pattyn et al., 2008), takes place on workers when they become tired in prolonged construction activities.”
BackgroundLayered representations for human activity recognition
Oliver, N., Horvitz, E., & Garg, A. (2003) · Proceedings. Fourth IEEE International Conference on Multimodal Interfaces · doi:10.1109/icmi.2002.1166960 · Cited by 312 · cited 1×
“Hidden Markov models (HMMs) (Oliver et al., 2002; Nguyen et al., 2005; Zhang et al., 2006), dynamic Bayesian networks (Gong and Xiang, 2003; Dai et al., 2008), and conditional random fields (Liao et al., 2007; Sun and Nevatia, 2013; Shu et al., 2015) are frequently used in earlier studies.”
Method
Cited by
Computer vision techniques in construction: A critical review
Xu, S., Wang, J., Shou, W., Ngo, T., Sadick, A.-M., & Wang, X. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09504-3 · Cited by 409
Wang, X., & Yan, W. Q. (2020) · International Journal of Neural Systems · doi:10.1142/s0129065719500278 · Cited by 116
Deep Learning-Based Applications for Safety Management in the AEC Industry: A Review
Hou, L., Chen, H., Zhang, G. (., & Wang, X. (2021) · Applied Sciences · doi:10.3390/app11020821 · Cited by 76
Metadata
- Authors
- Luo, X., Li, H., Yang, X., Yu, Y., & Cao, D.
- Venue
- Computer-Aided Civil and Infrastructure Engineering
- Publisher
- Elsevier BV
- Published
- 2018-10-08
- Volume
- 34
- Issue
- 4
- Pages
- 333-351
- ISSN
- 1093-9687 (ISSN-L)
- 1467-8667
- OpenAlex
- W2895944074
- Cited by
- 111
- References
- 10
- FWCI
- 6.23
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.