Improving quality of data: IoT data aggregation using device-to-device communications
Open accessAbstract
With the widespread adaptation of the Internet of things (IoT), we are already witnessing a deluge of IoT data analytics applications. IoT data analytics can be defined as a process to control and optimize decision making in real time, by analyzing huge chunks of IoT sensor data. Moreover, data analytics stands on the shoulders of sensor data aggregation that includes data pre-processing and routing. This paper envisions two open problems of data aggregation, first, raw IoT sensor data is highly uncertain, second, the traditional algorithms are not fit for processing highly uncertain sensor data. This is formally known as data veracity problem. This paper proposes a data aggregation scheme for highly uncertain raw IoT sensor data collected using the device to device communication. The approach initially reconstructs the subspace using sample data and then it iteratively tracks down the low-rank approximation of the dominant subspace in the presence of high uncertainties at the fog server. Later, the robust dominant subspace is used to estimate a more reliable true sensor data matrix from the highly uncertain raw IoT sensor data traffic matrix. Moreover, the proposed scheme achieves the aforementioned tasks while processing the raw IoT sensor data without any prior information, i.e., in a fully unsupervised fashion. The existing literature based on sampling, approximation, and data reduction either causes random data reduction or destruction of global characteristics of the raw data. However, unlike the existing solutions, the proposed method removes the uncertainties while preserving the global characteristics of the raw data. Performance evaluations conducted using both the real world sensor data and synthetic data injected with noise, outliers, and missing values. Experimental results show that the proposed approach can estimate a reliable true sensor data matrix in the presence of high uncertainties and the energy efficient device to device communication-based data delivery mechanism can accommodate a large number of IoT devices.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Three of them use reference sensors/monitors or sample data to improve sensor data accuracy.
“Three of them [27,31,33] use reference sensors/monitors or sample data to improve sensor data accuracy.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [33]
Four studies provide techniques that repair corrupt data and increase its accuracy.
“Four studies [23,27,31,33] provide techniques that repair “corrupt” data and increase its accuracy.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [33]
Only four primary studies [23,27,31,33] address online data repair.
“However, we identified only four primary studies [23,27,31,33] addressing online data repair.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [33]
Only three studies (i.e., [27,33,44]) use regression metrics to assess the performance of data quality techniques, not their impact on the performance of classification/regression models.
“As mentioned above, only three studies (i.e., [27,33,44]) use regression metrics to assess the performance of data quality techniques, not their impact on the performance of classification/regression models.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [33]
They evaluate data repair approaches predicting missing or corrupted data.
“They evaluate data repair approaches predicting missing or corrupted data.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [33]
Cites
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075 · see citing works
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,863 · see citing works
Robust principal component analysis?
Candès, E. J., Li, X., Ma, Y., & Wright, J. N. (2011) · Journal of the ACM · doi:10.1145/1970392.1970395 · Cited by 6,914 · see citing works
Ziegel, E. R., & Lohr, S. L. (2000) · Technometrics · doi:10.2307/1271491 · Cited by 2,450 · see citing works
New criteria for blind deconvolution of nonminimum phase systems (channels)
Shalvi, O., & Weinstein, E. (1990) · IEEE Transactions on Information Theory · doi:10.1109/18.52478 · Cited by 863 · see citing works
Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach
Maharjan, S., Zhu, Q., Zhang, Y., Gjessing, S., & Başar, T. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2223766 · Cited by 784 · see citing works
Data driven prediction models of energy use of appliances in a low-energy house
Candanedo, L. M., Feldheim, V., & Deramaix, D. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.01.083 · Cited by 559 · see citing works
Channel State Information Prediction for 5G Wireless Communications: A Deep Learning Approach
Luo, C., Ji, J., Wang, Q., Chen, X., & Li, P. (2018) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2018.2848960 · Cited by 467 · see citing works
Robust PCA via Outlier Pursuit
Xu, H., Caramanis, C., & Sanghavi, S. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2173156 · Cited by 445 · see citing works
Home M2M networks: Architectures, standards, and QoS improvement
Zhang, Y., Yu, R., Xie, S., Yao, W., Xiao, Y., & Guizani, M. (2011) · IEEE Communications Magazine · doi:10.1109/mcom.2011.5741145 · Cited by 432 · see citing works
ROBUST PRINCIPAL COMPONENT ANALYSIS
Alanzi, A. R. A. (2014) · OpenSIUC (Southern Illinois University Carbondale) · Cited by 419 · see citing works
Adaptive cleaning for RFID data streams
Jeffery, S. R., Garofalakis, M., & Franklin, M. J. M. (2006) · UC Berkeley · doi:10.5555/1182635.1164143 · Cited by 411 · see citing works
Cognitive machine-to-machine communications: visions and potentials for the smart grid
Zhang, Y., Yu, R., Nekovee, M., et al. (2012) · IEEE Network · doi:10.1109/mnet.2012.6201210 · Cited by 393 · see citing works
Generalized Principal Component Analysis
Vidal, R., Ma, Y., & Sastry, S. (2016) · Interdisciplinary applied mathematics · doi:10.1007/978-0-387-87811-9 · Cited by 370 · see citing works
Dynamic Trust Relationships Aware Data Privacy Protection in Mobile Crowd-Sensing
Wu, D., Si, S., Wu, S., et al. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2768073 · Cited by 190 · see citing works
Social Attribute Aware Incentive Mechanism for Device-to-Device Video Distribution
Wu, D., Yan, J., Wang, H., et al. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2692648 · Cited by 178 · see citing works
Yu, T., Wang, X., & Shami, A. A. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2756025 · Cited by 162 · see citing works
Blind calibration of sensor networks
Balzano, L., & Nowak, R. M. (2007) · doi:10.1145/1236360.1236372 · Cited by 159 · see citing works
Fast Algorithms for Robust PCA via Gradient DescentPreprint
Yi, X., Park, D., Chen, Y., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07784 · Cited by 131 · see citing works
Communication-Efficient Online Detection of Network-Wide Anomalies
Huang, L., Nguyen, X., Garofalakis, M., et al. (2007) · doi:10.1109/infcom.2007.24 · Cited by 122 · see citing works
Probabilistic Inference over RFID Streams in Mobile Environments
Tran, T. T. L., Sutton, C. A., Cocci, R., et al. (2009) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2009.33 · Cited by 113 · see citing works
Knowledge-Centric Edge Computing Based on Virtualized D2D Communication Systems
Wang, R., Yan, J., Wu, D., et al. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700876 · Cited by 103 · see citing works
Energy Efficient IoT Data Collection in Smart Cities Exploiting D2D Communications
Orsino, A., Araniti, G., Militano, L., et al. (2016) · Sensors · doi:10.3390/s16060836 · Cited by 95 · see citing works
Fast Algorithms for Robust PCA via Gradient Descent
Yi, X., Park, D., Chen, Y., et al. (2016) · neural information processing systems · Cited by 93 · see citing works
Security-oriented opportunistic data forwarding in Mobile Social Networks
Wu, D., Zhang, F., Wang, H., et al. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.07.028 · Cited by 82 · see citing works
Learning from Uncertainty for Big Data: Future Analytical Challenges and Strategies
Wang, X., & He, Y. (2016) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2016.2557479 · Cited by 80 · see citing works
ApproxIoT: Approximate Analytics for Edge Computing
Wen, Z., Quoc, D. L., Bhatotia, P., Chen, R., & Lee, M. (2018) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2018.00048 · Cited by 76 · see citing works
Blind Calibration of Sensor Networks
Balzano, L., & Nowak, R. D. (2007) · International Symposium on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2007.4379667 · Cited by 66 · see citing works
SRSM-Based Adaptive Relay Selection for D2D Communications
Zhang, Z., Zhang, P., Liu, D., et al. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2749443 · Cited by 65 · see citing works
Self-citationChallenges for Quality of Data in Smart Cities
Barnaghi, P., Bermúdez-Edo, M., & Tönjes, R. (2015) · Journal of Data and Information Quality · doi:10.1145/2747881 · Cited by 64 · see citing works
A Novel Fog Computing Enabled Temporal Data Reduction Scheme in IoT Systems
Yu, T., Wang, X., & Shami, A. A. (2017) · Globecom · doi:10.1109/glocom.2017.8253941 · Cited by 62 · see citing works
High-Accuracy Entity State Prediction Method Based on Deep Belief Network Toward IoT Search
Zhang, P., Kang, X., Wu, D., et al. (2018) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2018.2877639 · Cited by 61 · see citing works
Self-citationZhang, P., Liu, Y., Wu, F., Liu, S., & Tang, B. (2016) · IEEE Communications Letters · doi:10.1109/lcomm.2016.2521735 · Cited by 38 · see citing works
Self-citationChannel Characteristic Aware Privacy Protection Mechanism in WBAN
Zhang, P., & Ma, J. (2018) · Sensors · doi:10.3390/s18082403 · Cited by 30 · see citing works
Self-citationWidom, J., & Olston, C. (2003) · Cited by 27 · see citing works
Cooperative Willingness Aware Collaborative Caching Mechanism Towards Cellular D2D Communication
Zhang, P., Kang, X., Liu, Y., et al. (2018) · IEEE Access · doi:10.1109/access.2018.2873662 · Cited by 27 · see citing works
Self-citationA review on approximate computing and some of the associated techniques for energy reduction in IOT
Marwaha, D., & Sharma, A. (2018) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc.2018.8399087 · Cited by 9 · see citing works
Robust PCA and subspace tracking from incomplete observations using L0-surrogatesPreprint
Hage, C., & Kleinsteuber, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.0805 · Cited by 5 · see citing works
Sensor network monitors 1,400-km canal [News]
Staedter, T. (2018) · IEEE Spectrum · doi:10.1109/mspec.2018.8322033 · Cited by 5 · see citing works
Co-relative mobility based IoT data uploading using D2D communication
Sanyal, S., Wu, D., Yan, J., et al. (2017) · EAI International Conference on Mobile Multimedia Communications (MobiMedia) · doi:10.4108/eai.13-7-2017.2270111 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Data Quality Management in the Internet of Things
Zhang, L., Jeong, D., & Lee, S. (2021) · Sensors · doi:10.3390/s21175834 · Cited by 77 · see citing works
A Systematic Review of Data Quality in CPS and IoT for Industry 4.0
Goknil, A., Nguyen, P., Sen, S., Politaki, D., Niavis, H., Pedersen, K. J., Suyuthi, A., Anand, A., & Ziegenbein, A. (2023) · ACM Computing Surveys · doi:10.1145/3593043 · Cited by 68 · see citing works
- Authors
- Sanyal, S., & Zhang, P.
- Affiliations
- Chongqing University of Posts and Telecommunications (CN)
- Venue
- IEEE Access
- Published
- 2018
- Volume
- 6
- Pages
- 67830-67840
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2900147861
- Cited by
- 51
- References
- 40
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 7 · 2023: 8 · 2022: 7 · 2021: 7 · 2020: 6 · 2019: 7
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61771082)
- Program for Innovation Team Building at Institutions of Higher Education in Chongqing (CXTDX201601020)
- Chongqing Municipal Education Commission (KJQN201800615)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- device-to-device communication, data veracity, low-rank approximation, missing data, outlier robustness, fog computing, sensor data quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
