Robust people counting system based on sensor fusion
Abstract
This paper presents a novel robust people counting system based on fusing the depth and vision data. Conventional algorithms utilize the monoscopic or stereoscopic vision data to count people. However, these vision-based people counting methods often fail due to occasional illumination change and crowded environment. In the proposed algorithm, both the top-view vision and depth images are captured by a video-plus-depth camera mounted on the ceiling. The depth image is first processed by a morphological operator to alleviate depth artifacts such as the optical noise and lost data. Then the human object is extracted using a human model from the preprocessed depth image. Finally, the trajectory of the detected object is established by applying the bidirectional matching algorithm. Experimental results show that the proposed algorithm achieves over 98% accuracy in various testing environments.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In situations where the public transport vehicle is full of passengers, the detected person skeletons from depth sensors can overlap, which can result in wrong detection results.
“Depth sensors were also utilized, like researched in [27], but in situations where the public transfer vehicle is full of passengers, the detected person skeletons can overlap, which can result in wrong detection results.”
From A Review of Passenger Counting in Public Transport Concepts with Solution Proposal Based on Image Processing and Machine Learning (2024) · cites this paper as [27]
CCTV cameras without machine learning achieve a typical precision range of 75-97%.
“CCTV Cameras (without ML) [22,27] 75-97%”
From A Review of Passenger Counting in Public Transport Concepts with Solution Proposal Based on Image Processing and Machine Learning (2024) · cites this paper as [27]
Cites
The Divergence and Bhattacharyya Distance Measures in Signal Selection
Kailath, T. (1967) · IRE Transactions on Communications Systems · doi:10.1109/tcom.1967.1089532 · Cited by 1,860 · see citing works
Real-time foreground–background segmentation using codebook model
Kim, K., Chalidabhongse, T. H., Harwood, D., & Davis, L. S. (2005) · Real-Time Imaging · doi:10.1016/j.rti.2004.12.004 · Cited by 1,421 · see citing works
Counting people in crowds with a real-time network of simple image sensors
Yang, Gonzalez-Banos, & Guibas (2003) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238325 · Cited by 245 · see citing works
A camera-based system for tracking people in real time
Segen, J. (1996) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1996.546795 · Cited by 154 · see citing works
Detecting and counting people in surveillance applications
Liu, X., Tu, P., Rittscher, J., Perera, A. G. A., & Krahnstoever, N. O. (2006) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2005.1577286 · Cited by 81 · see citing works
A method of counting the passing people by using the stereo images
Terada, K., Yoshida, D., Oe, S., & Yamaguchi, J. (1999) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.1999.822913 · Cited by 69 · see citing works
Real-time disparity estimation algorithm for stereo camera systems
Lee, S., & Sharma, S. (2011) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2011.6018850 · Cited by 42 · see citing works
Real-time people localization and tracking through fixed stereo vision
Bahadori, S., Iocchi, L., Leone, G. R., Nardi, D., & Scozzafava, L. (2006) · Applied Intelligence · doi:10.1007/s10489-006-0013-3 · Cited by 41 · see citing works
Unattended object intelligent analyzer for consumer video surveillance
Zin, T. T., Tin, P., Hama, H., & Toriu, T. (2011) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2011.5955191 · Cited by 40 · see citing works
Real-time passenger counting in buses using dense stereovision
Yahiaoui, T., Khoudour, L., & Meurie, C. (2010) · Journal of Electronic Imaging · doi:10.1117/1.3455989 · Cited by 37 · see citing works
3D scene capturing using stereoscopic cameras and a time-of-flight camera
Lee, C., Song, H., Choi, B., & Ho, Y. (2011) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2011.6018896 · Cited by 34 · see citing works
Lightweight People Counting and Localizing for Easily Deployable Indoors WSNs
Teixeira, T., & Savvides, A. (2008) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2008.2001426 · Cited by 33 · see citing works
Real-Time System for Counting the Number of Passing People Using a Single Camera
Kim, J.-W., Choi, K. S., Choi, B.-D., Lee, J. Y., & Ko, S.-J. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45243-0_60 · Cited by 26 · see citing works
Self-citationEffective bi-directional people flow counting for real time surveillance system
Yam, K.-Y., Siu, W.-C., Law, N.-F., & Chan, C.-K. (2011) · International Conference on Consumer Electronics · doi:10.1109/icce.2011.5722907 · Cited by 25 · see citing works
Image sequence analysis for counting in real time people getting in and out of a bus
Mecocci, A., Bartolini, F., & Cappellini, V. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90039-6 · Cited by 17 · see citing works
3d video generation and service based on a TOF depth sensor in MPEG-4 multimedia framework
Kim, S.-Y., Cho, J.-H., Koschan, A. F., & Abidi, M. A. (2010) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2010.5606319 · Cited by 16 · see citing works
Recognition of object contours from stereo images: an edge combination approach
Gelautz, M., & Markovic, D. (2004) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/tdpvt.2004.1335394 · Cited by 16 · see citing works
Depth video based human model reconstruction resolving self-occlusion
Jang, I. S., & Lee, K. (2010) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2010.5606349 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Robust Fusion of LiDAR and Wide-Angle Camera Data for Autonomous Mobile Robots
Silva, D., Roche, J., & KONDOZ, A. M. (2018) · Sensors · doi:10.3390/s18082730 · Cited by 128 · see citing works
Kolar, P., Benavidez, P. J., & Jamshidi, M. M. (2020) · Sensors · doi:10.3390/s20082180 · Cited by 112 · see citing works
Detecting and tracking people in real time with RGB-D camera
Liu, J., Liu, Y., Zhang, G., Zhu, P., & Chen, Y. Q. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.09.013 · Cited by 83 · see citing works
Behavioral Pedestrian Tracking Using a Camera and LiDAR Sensors on a Moving Vehicle
Dimitrievski, M., Veelaert, P., & Philips, W. R. (2019) · Sensors · doi:10.3390/s19020391 · Cited by 74 · see citing works
Crowd Counting by Using Top-k Relations: A Mixed Ground-Truth CNN Framework
Li, D., Zhang, H., Yang, K., Zhou, D., Shi, J., & Ma, J. (2022) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2022.3190384 · Cited by 72 · see citing works
Device-Free Counting via Wideband Signals
Bartoletti, S., Conti, A., & Win, M. Z. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2680978 · Cited by 68 · see citing works
Multiple human tracking in RGB‐depth data: a survey
Camplani, M., Paiement, A., Mirmehdi, M., Damen, D., Hannuna, S., Burghardt, T., & Tao, L. (2016) · IET Computer Vision · doi:10.1049/iet-cvi.2016.0178 · Cited by 55 · see citing works
Deep understanding of shopper behaviours and interactions using RGB-D vision
Paolanti, M., Pietrini, R., Mancini, A., Frontoni, E., & Zingaretti, P. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01118-w · Cited by 51 · see citing works
Robust people detection using depth information from an overhead Time-of-Flight camera
Luna, C. A., Losada-Gutiérrez, C., Fuentes-Jiménez, D., Fernandez-Rincon, A., Mazo, M., & Macías-Guarasa, J. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.11.019 · Cited by 45 · see citing works
Zheng, H.-C., Lin, Z.-J., Cen, J., Wu, Z., & Zhao, Y. (2019) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2018.2807806 · Cited by 36 · see citing works
Real-time human detection and tracking in complex environments using single RGBD camera
Liu, J., Liu, Y., Cui, Y., & Chen, Y. Q. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738636 · Cited by 35 · see citing works
The design and implementation of a vision-based people counting system in buses
Perng, J., Wang, T.-Y., Hsu, Y.-W., & Wu, B. (2016) · International Conference on System Science and Engineering · doi:10.1109/icsse.2016.7551620 · Cited by 35 · see citing works
Li, D., Zhang, H., Zhou, D., Shi, J., & Ma, J. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3274651 · Cited by 30 · see citing works
An ultra-fast human detection method for color-depth camera
Liu, J., Zhang, G., Liu, Y., Tian, L., & Chen, Y. Q. (2015) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2015.06.014 · Cited by 29 · see citing works
People-flow counting in complex environments by combining depth and color information
Gao, C., Liu, J., Feng, Q., & Lv, J. G. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3344-z · Cited by 28 · see citing works
A robust system for counting people using an infrared sensor and a camera
Erden, F., Alkar, A. Z., & Çetin, A. E. (2015) · Infrared Physics & Technology · doi:10.1016/j.infrared.2015.07.019 · Cited by 24 · see citing works
DPDnet: A robust people detector using deep learning with an overhead depth camera
Fuentes-Jiménez, D., López, R. M., Losada-Gutiérrez, C., Casillas-Pérez, D., Macías-Guarasa, J., Luna, C. A., & Pizarro, D. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.113168 · Cited by 20 · see citing works
Radovan, A., Mršić, L., Đambić, G., & Mihaljević, B. (2024) · Eng · doi:10.3390/eng5040172 · Cited by 24 · see citing works
Review of Automatic Passenger Counting Systems in Public Urban Transport
Grgurević, I., Juršić, K., & Rajič, V. (2022) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-67241-6_1 · Cited by 17 · see citing works
People counting system by using kinect sensor
Coskun, A., Kara, A., Parlaktuna, M., Özkan, M., & Parlaktuna, O. (2015) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista.2015.7276740 · Cited by 16 · see citing works
Depth-Based Human Detection Considering Postural Diversity and Depth Missing in Office Environment
Fujimoto, Y., & Fujita, K. (2019) · IEEE Access · doi:10.1109/access.2019.2892197 · Cited by 13 · see citing works
Real‐time multi‐class moving target tracking and recognition
Zhang, Q., Sun, Y., Yang, J., & Liu, H. (2016) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2014.0226 · Cited by 12 · see citing works
Testing and Evaluation of Low-Cost Sensors for Developing Open Smart Campus Systems Based on IoT
Neis, P., Warch, D., & Hoppe, M. (2023) · Sensors · doi:10.3390/s23208652 · Cited by 12 · see citing works
People Counting in Crowded Environment and Re-identification
Frontoni, E., Paolanti, M., & Pietrini, R. (2019) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-28603-3_18 · Cited by 11 · see citing works
Top-view people counting in public transportation using Kinect
Malawski, F. (2014) · Challenges of Modern Technology · Cited by 9 · see citing works
Enhancement of Missing Face Prediction Algorithm with Kalman Filter and DCF-CSR
Maharani, D. A., Machbub, C., & Rusmin, P. H. (2019) · doi:10.1109/iceei47359.2019.8988867 · Cited by 9 · see citing works
Robust People Detection and Tracking from an Overhead Time-of-Flight Camera
Fernandez-Rincon, A., Fuentes-Jiménez, D., Losada-Gutiérrez, C., Marrón-Romera, M., Luna, C. A., Macías-Guarasa, J., & Mazo, M. (2017) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0006169905560564 · Cited by 9 · see citing works
A Fast People Counting Method Based on Optical Flow
Tokta, A., & Hocaoǧlu, A. K. (2018) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap.2018.8620838 · Cited by 8 · see citing works
Improved Person Counting Performance Using Kalman Filter Based on Image Detection and Tracking
Vignarca, D., Prakash, J., Vignati, M., & Sabbioni, E. (2021) · AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE) · doi:10.23919/aeitautomotive52815.2021.9662745 · Cited by 7 · see citing works
A people counting system for business analytics
Pane, C., Gasparini, M., Prati, A., Gualdi, G., & Cucchiara, R. (2013) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2013.6636629 · Cited by 7 · see citing works
Device-Free Counting via OFDM Signals of Opportunity
Bartoletti, S., Conti, A., & Win, M. Z. (2018) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2018.8403681 · Cited by 7 · see citing works
Modal-Adaptive Spatial-Aware-Fusion and Propagation Network for Multimodal Vision Crowd Counting
Liu, K., Zou, X., Zhu, P., & Sang, J. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3571571 · Cited by 6 · see citing works
Towards counting via passive radar using OFDM waveforms
Bartoletti, S., Conti, A., & Win, M. Z. (2017) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2017.7962757 · Cited by 6 · see citing works
An Indirect Occupancy Detection and Occupant Counting System Using Motion Sensors
Luo, D., Lǚ, J., & Guo, G. (2017) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2017-01-1442 · Cited by 6 · see citing works
Towards dense people detection with deep learning and depth images
Fuentes-Jiménez, D., Losada-Gutiérrez, C., Casillas-Pérez, D., Macías-Guarasa, J., Pizarro, D., López, R. M., & Luna, C. A. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104484 · Cited by 6 · see citing works
Pedestrian Counting for a Large Scene Using a GigaPan Panorama and Exemplar-SVMs
Zhang, S., Yu, S., Ma, Q., Shang, P., Gui, P., Wang, J., & Feng, T. (2013) · International Conference on Computational Intelligence and Security (CIS) · doi:10.1109/cis.2013.55 · Cited by 5 · see citing works
Drăgulinescu, A., Marcu, I. M., Halunga, S. V., & Fratu, O. (2018) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-74935-8_14 · Cited by 5 · see citing works
Weight Estimation from an RGB-D camera in top-view configuration
Mameli, M., Paolanti, M., Conci, N., Tessaro, F., Frontoni, E., & Zingaretti, P. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412519 · Cited by 3 · see citing works
Cao, Z., Wu, Z., Yu, X., Song, C., & Xu, Z. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3305551 · Cited by 3 · see citing works
LiDAR and Camera-Based Convolutional Neural Network Detection for Autonomous Driving
Hamieh, I., Myers, R., Nimri, H., Rahman, T., Younan, A., Sato, B., El-Kadri, A., Nissan, S., & Tepe, K. E. (2020) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2020-01-0136 · Cited by 3 · see citing works
Shoppers Detection Analysis in an Intelligent Retail Environment
Sciucca, L. D., Manco, D., Contigiani, M., Pietrini, R., Bello, L. D., & Placidi, V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68790-8_42 · Cited by 3 · see citing works
Crowd-Centric Counting via Unsupervised Learning
Morselli, F., Bartoletti, S., Mazuelas, S., Win, M. Z., & Conti, A. (2019) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2019.8757112 · Cited by 2 · see citing works
Real-Time Human Detection and Tracking Using PEI Representation in a Dynamic 3D Environment
Mahalakshmi, M. S., Kanthavel, R., & Hemavathy, N. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-981-13-2182-5_19 · Cited by 1 · see citing works
Distributed Surveillance System for Business Economic and Information Management
Shbib, R., Zhou, S., Ndzi, D. L., Alkadhimi, K., & Al‐Mosawi, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-09265-2_52
Sensor Fusion-based Cooperative Human Detection Scheme using MAVs in Indoor Environment
Kim, M.-E., Bang, H. C., Lee, S., & Jeon, J. H. (2020) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc49870.2020.9289611
A Novel Approach of Human Tracking and Counting Using Overhead ToF Camera
Wang, H., Luo, H., Zhou, W., & Xie, D. (2020) · Communications in computer and information science · doi:10.1007/978-981-33-6378-6_31
Cloud-Based Detection of Forged Passport and Extraction of True Identity
Kanthavel, R. (2022) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-6684-5058-1.ch010
Oka, M., Azuma, K., Shinkuma, R., Sato, T., & Oki, E. (2023) · IEEE International Conference on Consumer Electronics - Berlin (ICCE-Berlin) · doi:10.1109/icce-berlin58801.2023.10375628
- Authors
- Dan, B.-K., Kim, Y.-S., Suryanto, Jung, J.-Y., & Ko, S.-J.
- Affiliations
- Korea University (KR)
- LG (South Korea) (KR)
- ORCID
- Ko, Sung-Jea
- Venue
- IEEE Transactions on Consumer Electronics
- Published
- 2012-08
- Volume
- 58
- Issue
- 3
- Pages
- 1013-1021
- ISSN
- 0098-3063 (ISSN-L)
- OpenAlex
- W2026332379
- Cited by
- 49
- References
- 18
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 4 · 2022: 2 · 2021: 5 · 2020: 5 · 2019: 7 · 2018: 6 · 2017: 4 · 2016: 5 · 2015: 3 · 2014: 3 · 2013: 3
- FWCI
- 2.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Consumer Electronics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- people counting, sensor fusion, depth images, ceiling-mounted camera, human detection, human models, morphological processing, bidirectional matching, trajectory tracking, crowded environments, illumination variation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
