The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
PPCA-Based Missing Data Imputation for Traffic Flow Volume: A Systematical Approach
Abstract
The missing data problem greatly affects traffic analysis. In this paper, we put forward a new reliable method called probabilistic principal component analysis (PPCA) to impute the missing flow volume data based on historical data mining. First, we review the current missing data-imputation method and why it may fail to yield acceptable results in many traffic flow applications. Second, we examine the statistical properties of traffic flow volume time series. We show that the fluctuations of traffic flow are Gaussian type and that principal component analysis (PCA) can be used to retrieve the features of traffic flow. Third, we discuss how to use a robust PCA to filter out the abnormal traffic flow data that disturb the imputation process. Finally, we recall the theories of PPCA/Bayesian PCA-based imputation algorithms and compare their performance with some conventional methods, including the nearest/mean historical imputation methods and the local interpolation/regression methods. The experiments prove that the PPCA method provides significantly better performance than the conventional methods, reducing the root-mean-square imputation error by at least 25%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
Jolliffe, I. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa501 · Cited by 14,310
de Boor, C. (1978) · Applied Mathematical Sciences · doi:10.1007/978-1-4612-6333-3 · Cited by 11,944
Analysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0071325 · Cited by 9,633
Witten, I. H. (2008) · Cited by 9,143
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012
The EM Algorithm and Extensions
Kushary, D. (1998) · Technometrics · doi:10.1080/00401706.1998.10485534 · Cited by 5,417
Probabilistic Principal Component Analysis
Tipping, M. E., & Bishop, C. M. (1999) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00196 · Cited by 3,679
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
Statistical Analysis with Missing Data
Bo., N., Little, R. J. A., & Rubin, D. B. (1988) · Population (French Edition) · doi:10.2307/1533221 · Cited by 2,095
Analysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0070888 · Cited by 1,979
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
ROBPCA: A New Approach to Robust Principal Component Analysis
Hubert, M., Rousseeuw, P. J., & Vanden Branden, K. (2005) · Technometrics · doi:10.1198/004017004000000563 · Cited by 1,052
Numerical methods and software
Kahaner, D. K., Moler, C., & Nash, S. (1989) · Cited by 1,026
(2002) · Springer Series in Statistics · doi:10.1007/b98835 · Cited by 919
A Unifying Review of Linear Gaussian Models
Roweis, S., & Ghahramani, Z. (1999) · Neural Computation · doi:10.1162/089976699300016674 · Cited by 869
A Bayesian Network Approach to Traffic Flow Forecasting
Sun, S., Zhang, C., & Yu, G. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.869623 · Cited by 795
A Bayesian missing value estimation method for gene expression profile data
Oba, S., Sato, M.-A., Takemasa, I., Monden, M., Matsubara, K.-I., & Ishii, S. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg287 · Cited by 753
Numerical Computing with Matlab
Moler, C. B. (2004) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898717952 · Cited by 646
Automatic Choice of Dimensionality for PCA
Minka, T. P. (2000) · neural information processing systems · Cited by 630
A Novel Anomaly Detection Scheme Based on Principal Component Classifier
Shyu, M., Chen, S., Sarinnapakorn, K., & Chang, L. (2003) · Cited by 602
Maximum Likelihood Estimation from Incomplete Data
Hartley, H. O. (1958) · Biometrics · doi:10.2307/2527783 · Cited by 602
(2003) · Statistics in Musicology · doi:10.1201/9780203496947.ch8 · Cited by 577
Resampling Methods: A Practical Guide to Data Analysis
Chaubey, Y. P. (2000) · Technometrics · doi:10.1080/00401706.2000.10486056 · Cited by 427
High breakdown estimators for principal components: the projection-pursuit approach revisited
Croux, C., & Ruiz-Gazen, A. (2005) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2004.08.002 · Cited by 328
Missing Observations in Multivariate Statistics I. Review of the Literature
Afifi, A. A., & Elashoff, R. M. (1966) · Journal of the American Statistical Association · doi:10.1080/01621459.1966.10480891 · Cited by 306
Detecting Errors and Imputing Missing Data for Single-Loop Surveillance Systems
Chen, C., Kwon, J., Rice, J., Skabardonis, A., & Varaiya, P. P. (2003) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/1855-20 · Cited by 298
Nearest Neighbor Imputation for Survey Data
Chen, J., & Shao, J. (2000) · Journal of Official Statistics · Cited by 262
Bishop, C. (1998) · neural information processing systems · Cited by 224
Principal component analysis with missing data and its application to polyhedral object modeling
Shum, H.-Y., Ikeuchi, K., & Reddy, R. (1995) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.406651 · Cited by 188
Reconstructing the spatio-temporal traffic dynamics from stationary detector data
Treiber, M., & Helbing, D. (2002) · Cited by 187
Exploring Imputation Techniques for Missing Data in Transportation Management Systems
Smith, B. L., Scherer, W. T., & Conklin, J. H. (2003) · Transportation Research Record: Journal of the Transportation Research Board · doi:10.3141/1836-17 · Cited by 172
Principal component analysis for data containing outliers and missing elements
Serneels, S., & Verdonck, T. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.05.024 · Cited by 171
Multiple Imputation Scheme for Overcoming the Missing Values and Variability Issues in ITS Data
Ni, D., Leonard, J. D., Guin, A., & Feng, C. (2005) · Journal of Transportation Engineering · doi:10.1061/(asce)0733-947x(2005)131:12(931 · Cited by 121
A BPCA based missing value imputing method for traffic flow volume data
Qu, L., Zhang, Y., Hu, J., Jia, L., & Li, L. (2008) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2008.4621153 · Cited by 111
Self-citationSimultaneously Prediction of Network Traffic Flow Based on PCA-SVR
Jin, X., Zhang, Y., & Yao, D. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-72393-6_121 · Cited by 71
Self-citationSpatial–Temporal Traffic Data Analysis Based on Global Data Management Using MAS
Zhang, H.-S., Zhang, Y., Li, Z., & Hu, D.-C. (2004) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2004.837816 · Cited by 58
Self-citationRobust principal component analysis by projection pursuit
Xie, Y., Wang, J., Liang, Y., Sun, L., Song, X., & Yu, R. (1993) · Journal of Chemometrics · doi:10.1002/cem.1180070606 · Cited by 56
Netlib and NA-Net: Building a Scientific Computing Community
Dongarra, J. J., Golub, G. H., Grosse, E., et al. (2008) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2008.29 · Cited by 39
Cited by
Data-Driven Intelligent Transportation Systems: A Survey
Zhang, J., Wang, F.-Y., Wang, K., Lin, W.-H., Xu, X., & Chen, C. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2158001 · Cited by 1,817
“[100] introduced probabilistic principal component analysis (PPCA)-based missing data imputation, where PPCA is used to capture the main structure, and”
BackgroundMethodShort-term traffic forecasting: where we are and where we’re going
Vlahogianni, E. I., Karlaftis, M. G., & Golias, J. C. (2014) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2014.01.005 · Cited by 1,132
Sensor Technologies for Intelligent Transportation Systems
Guerrero-Ibáñez, J., Zeadally, S., & Contreras-Castillo, J. (2018) · Sensors · doi:10.3390/s18041212 · Cited by 637
A tensor-based method for missing traffic data completion
Tan, H., Feng, G., Feng, J., Wang, W., Zhang, Y.-J., & Li, F. (2013) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2012.12.007 · Cited by 427
Self-citationA Bayesian tensor decomposition approach for spatiotemporal traffic data imputation
Chen, X., He, Z., & Sun, L. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.11.003 · Cited by 373
Metadata
- Authors
- Qu, L., Hu, J., Li, L., & Zhang, Y.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2009-09
- Volume
- 10
- Issue
- 3
- Pages
- 512-522
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2163150789
- Cited by
- 441
- References
- 42
- FWCI
- 5.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.