The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
On the Temporal Relations between Logging and Code
Abstract
Prior work shows that misleading logging texts (i.e., the textual descriptions in logging statements) can be counterproductive for developers during their use of logs. One of the most important types of information provided by logs is the temporal information of the recorded system behavior. For example, a logging text may use a perfective aspect to describe a fact that an important system event has finished. Although prior work has performed extensive studies on automated logging suggestions, few of these studies investigate the temporal relations between logging and code. In this work, we make the first attempt to comprehensively study the temporal relations between logging and its corresponding source code. In particular, we focus on two types of temporal relations: (1) logical temporal relations, which can be inferred from the execution order between the logging statement and the corresponding source code; and (2) semantic temporal relations, which can be inferred based on the semantic meaning of the logging text. We first perform qualitative analyses to study these two types of logging-code temporal relations and the inconsistency between them. As a result, we derive rules to detect these two types of temporal relations and their inconsistencies. Based on these rules, we propose a tool named TempoLo to automatically detect the issues of temporal inconsistencies between logging and code. Through an evaluation of four projects, we find that TempoLo can effectively detect t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Logs are a crucial source of information in software systems, capturing system runtime behavior and aiding in software engineering tasks like program comprehension [1], [2], anomaly detection [3]- [6], and failure diagnosis [7], [8].”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [2]
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015
Maintaining knowledge about temporal intervals
Allen, J. F. (1983) · Communications of the ACM · doi:10.1145/182.358434 · Cited by 7,593
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962
A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories
Mostafazadeh, N., Chambers, N., He, X., Parikh, D., Batra, D., Vanderwende, L., Kohli, P., & Allen, J. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1098 · Cited by 598
SemEval-2007 task 15: TempEval temporal relation identification
Verhagen, M., Gaizauskas, R., Schilder, F., Hepple, M., Katz, G., & Pustejovsky, J. (2007) · Proceedings of the 4th International Workshop on Semantic Evaluations - SemEval '07 · doi:10.3115/1621474.1621488 · Cited by 296
Temporal Annotation in the Clinical Domain
Styler, W., Bethard, S., Finan, S., Palmer, M., Pradhan, S., de Groen, P. D., Erickson, B., Miller, T., Lin, C., Savova, G., et al. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00172 · Cited by 285
Learning to Log: Helping Developers Make Informed Logging Decisions
Zhu, J., He, P., Fu, Q., Zhang, H., Lyu, M. R.-T., & Zhang, D. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.60 · Cited by 230
Leveraging existing instrumentation to automatically infer invariant-constrained models
Beschastnikh, I., Brun, Y., Schneider, S., Sloan, M., & Ernst, M. D. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025151 · Cited by 219
Characterizing logging practices in open-source software
Yuan, D., Park, S., & Zhou, Y. (2012) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2012.6227202 · Cited by 195
Yuan, D., Mai, H., Xiong, W., Tan, L., Zhou, Y., & Pasupathy, S. (2010) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/1736020.1736038 · Cited by 176
Temporal Information Extraction
Ling, X., & Weld, D. S. (2010) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7512 · Cited by 164
The Specification Language TimeML
Pustejovsky, J., Ingria, R., Sauri´, R., O, J. E. C., Littman, J., Gaizauskas, R., Setzer, A., Katz, G., & Mani, I. (2005) · The Language Of Time · doi:10.1093/oso/9780199268535.003.0031 · Cited by 161
CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures
Mostafazadeh, N., Grealish, A., Chambers, N., Allen, J. F., & Vanderwende, L. (2016) · doi:10.18653/v1/w16-1007 · Cited by 126
Assisting developers of Big Data Analytics Applications when deploying on Hadoop clouds
Shang, W., Jiang, Z. M., Hemmati, H., Adams, B., Hassan, A. E., & Martin, P. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606586 · Cited by 125
Self-citationChen, B., & Jiang, Z. M. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9429-5 · Cited by 124
Which log level should developers choose for a new logging statement?
Li, H., Shang, W., & Hassan, A. E. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9456-2 · Cited by 118
Self-citationCharacterizing and Detecting Anti-Patterns in the Logging Code
Chen, B., & Jiang, Z. M. (2017) · IEEE/ACM International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.15 · Cited by 114
Characterizing the natural language descriptions in software logging statements
He, P., Chen, Z., He, S., & Lyu, M. R.-T. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238193 · Cited by 103
A Qualitative Study of the Benefits and Costs of Logging From Developers’ Perspectives
Li, H., Shang, W., Adams, B., Sayagh, M., & Hassan, A. E. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2970422 · Cited by 97
Self-citationSALSA: analyzing logs as state machines
Tan, J., Pan, X., Kavulya, S. P., Gandhi, R., & Narasimhan, P. (2008) · Cited by 95
Using deep learning to generate complete log statements
Mastropaolo, A., Pascarella, L., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3511561 · Cited by 81
Studying and detecting log-related issues
Hassani, M., Shang, W., Shihab, E., & Tsantalis, N. (2018) · Empirical Software Engineering · doi:10.1007/s10664-018-9603-z · Cited by 71
Self-citationPattern-Based Mining of Opinions in Q&A Websites
Lin, B., Zampetti, F., Bavota, G., Penta, M. D., & Lanza, M. (2019) · International Conference on Software Engineering · doi:10.1109/icse.2019.00066 · Cited by 70
DLFinder: Characterizing and Detecting Duplicate Logging Code Smells
Li, Z.-H., Chen, T., Yang, J.-Q., & Shang, W.-Y. (2019) · International Conference on Software Engineering · doi:10.1109/icse.2019.00032 · Cited by 68
Self-citationDeepLV: Suggesting Log Levels Using Ordinal Based Neural Networks
Li, Z., Li, H., Chen, T.-H. P., & Shang, W. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00131 · Cited by 63
Self-citationAn Exploratory Study of Performance Regression Introducing Code Changes
Chen, J., & Shang, W. (2017) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2017.13 · Cited by 62
Self-citationUnderstanding Log Lines Using Development Knowledge
Shang, W., Nagappan, M., Hassan, A. E., & Jiang, Z. M. (2014) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2014.24 · Cited by 58
Self-citationContextual analysis of program logs for understanding system behaviors
Fu, Q., Lou, J., Lin, Q., et al. (2013) · Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624054 · Cited by 49
A pilot study on annotating temporal relations in text
Setzer, A., & Gaizauskas, R. (2001) · doi:10.3115/1118238.1118248 · Cited by 45
LogAssist: Assisting Log Analysis Through Log Summarization
Locke, S., Li, H., Chen, T.-H., Shang, W., & Liu, W. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3083715 · Cited by 45
Self-citationTowards an NLP-based log template generation algorithm for system log analysis
Kobayashi, S., Fukuda, K., & Esaki, H. (2014) · International Conference on Future Internet Technologies · doi:10.1145/2619287.2619290 · Cited by 43
Improving Performances of Log Mining for Anomaly Prediction Through NLP-Based Log Parsing
Aussel, N., Petetin, Y., & Chabridon, S. (2018) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2018.00031 · Cited by 43
Ding, Z., Li, H., Shang, W., & Chen, T.-H. P. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10118-5 · Cited by 39
Self-citationLoGenText: Automatically Generating Logging Texts Using Neural Machine Translation
Ding, Z., Li, H., & Shang, W. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00051 · Cited by 37
Self-citationStudying the use of Java logging utilities in the wild
Chen, B., & Jiang, Z. M. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380408 · Cited by 35
Towards the use of the readily available tests from the release pipeline as performance tests
Ding, Z., Chen, J., & Shang, W. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380351 · Cited by 34
Self-citationStudying Duplicate Logging Statements and Their Relationships With Code Clones
Li, Z.-H., Chen, T., Yang, J.-Q., & Shang, W.-Y. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3060918 · Cited by 33
Self-citationAutomated evolution of feature logging statement levels using Git histories and degree of interest
Tang, Y., Spektor, A., Khatchadourian, R., & Bagherzadeh, M. (2021) · Science of Computer Programming · doi:10.1016/j.scico.2021.102724 · Cited by 21
Self-citationTemporal information extraction for temporal question answering
Schilder, F., & Habel, C. U. (2003) · Cited by 14
Towards Learning Generalizable Code Embeddings Using Task-agnostic Graph Convolutional Networks
Ding, Z., Li, H., Shang, W., & Chen, T.-H. P. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3542944 · Cited by 13
Self-citationImproving software diagnosability via log enhancement
Yuan, D., Zheng, J., Park, S., Zhou, Y., & Savage, S. (2011) · ACM SIGARCH Computer Architecture News · doi:10.1145/1961295.1950369 · Cited by 11
ClearTAC: Verb Tense, Aspect, and Form Classification Using Neural Nets
Myers, S., & Palmer, M. S. (2019) · doi:10.18653/v1/w19-3315 · Cited by 8
Determining the Types of Temporal Relations in Discourse
Derczynski, L. (2013) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 7
Research works printed but not linked: 7
- li, 0, Where shall we log? studying and suggesting logging locations in code blocks, 35th IEEE/ACM International Conference on Automated Software Engineering ASE 2020, 361
- kabinna, 2016, Logging Library Migrations: A Case Study for the Apache Software Foundation Projects, 2016 IEEE/ACM 13th Conference on Mining Software Repositories (MSR), 154
- barik, 2016, The Bones of the System: A Case Study of Logging and Telemetry at Microsoft, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C) ICSE-C, 92
- honnibal, 2017, spaCy 2 Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing
- 2020, Mood & modality and dialogue sentiment to- I wards data science
- nicholas, 2017, JavaParser
- hamm, 2021, Tense and Aspect, The Stanford Encyclopedia of Philosophy
Cited by
Practitioners’ expectations on log anomaly detection
Ma, X., Li, Y., Keung, J., Yu, X., Zou, H., Yang, Z., Sarro, F., & Barr, E. T. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3586700 · Cited by 21
LogUpdater : Automated Detection and Repair of Specific Defects in Logging Statements
Zhong, R., Li, Y., Kuang, J., Gu, W., Huo, Y., & Lyu, M. R. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3731754 · Cited by 7
Self-citation
Metadata
- Authors
- Ding, Z., Tang, Y., Li, Y., Li, H., & Shang, W.
- Venue
- 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2023-05
- Pages
- 843-854
- Cited by
- 14
- References
- 43
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.