Cited by 262Information Retrieval and Search BehaviorAlgorithms and Data CompressionTopic Modeling

The DOI matches its Crossref record; 159 of the 166 references it cites were found in a registry or an index; 8 of the 137 sources its claims cite were checked for support.

Survey of Temporal Information Retrieval and Related Applications

Open access
Campos, Ricardo, Dias, Gaël, Jorge, Alípio M., Jatowt, Adam·2015·ACM Computing Surveys

Abstract

Temporal information retrieval has been a topic of great interest in recent years. Its purpose is to improve the effectiveness of information retrieval methods by exploiting temporal information in documents and queries. In this article, we present a survey of the existing literature on temporal information retrieval. In addition to giving an overview of the field, we categorize the relevant research, describe the main contributions, and compare different approaches. We organize existing research to provide a coherent view, discuss several open issues, and point out some possible future research directions in this area. Despite significant advances, the area lacks a systematic arrangement of prior efforts and an overview of state-of-the-art approaches. Moreover, an effective end-to-end temporal retrieval system that exploits temporal information to improve the quality of the presented results remains undeveloped.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This survey reviews temporal information retrieval (T-IR), organizing prior work on temporal annotation, extraction from web resources, and temporal web IR tasks such as crawling, indexing, query processing, ranking, clustering, classification, search engines, and future-related retrieval. It concludes that despite advances, the area lacks a systematic overview and an effective end-to-end temporal retrieval system remains undeveloped.

Key findings

  • T-IR combines traditional document relevance with temporal relevance; timeliness/currency is treated as one of five key aspects of document quality, alongside relevance, accuracy, objectivity, and coverage.
  • Explicit temporal queries are rare: about 1.5% of all queries [Nunes et al. 2008], reduced to 1.21% after accounting for false positives [Campos et al. 2011b]. Implicit temporal query studies report varying profiles: 54% atemporal and 46% temporally ambiguous [Jones and Diaz 2007]; only 7% implicit temporal [Metzler et al. 2009]; 10% never spiked, 47% spiked once, and 43% spiked multiple times [Kulkarni et al. 2011]; 75% atemporal and 25% implicit temporal [Campos et al. 2011c].
  • Temporal tagger evaluations: GUTime achieved F1 0.85 for recognition and 0.82 for normalization on TERN 2004; in TempEval-2, SuTime achieved the best extraction F1 of 0.92 versus HeidelTime’s 0.86, while HeidelTime achieved the best normalization F1 of 0.85 versus SuTime’s 0.82.
  • Web temporal evidence: among 62,842 web snippets, roughly 10% contain explicit temporal information; among 508 web snippets, 82.48% of future dates relate to the near future and 17.52% to a further future. The Internet Archive reportedly grows at 100 terabytes per month and has over 350 billion archived web pages, later cited as more than 357 billion.

Methods

  • Design: literature survey of existing T-IR research; it categorizes relevant research, describes main contributions, compares approaches, organizes existing research, and discusses open issues and future directions.
  • Data/sample: no primary dataset or experiment; it synthesizes prior literature and references datasets/collections such as TimeBank, TimeML, the New York Times Annotated Corpus, ClueWeb09/12, web query logs, web snippets, news archives, and Twitter.
  • Analysis: qualitative narrative synthesis and taxonomy across temporal annotation, temporal information extraction, web temporal feature extraction (metadata, content, usage), and temporal web IR stages including crawling, indexing, query processing, ranking, clustering, classification, search engines, and future-related retrieval.

Limitations

  • Stated in the paper: the area lacks a systematic arrangement of prior efforts and an overview of state-of-the-art approaches; an effective end-to-end temporal retrieval system that exploits temporal information to improve result quality remains undeveloped.
  • Stated in the paper: evaluation is difficult because the community often lacks a gold standard; most research has a rather specific scope; query log access outside industry is impeded by privacy/legal concerns; temporal taggers are often language- and domain-specific; metadata timestamps are often unavailable or unreliable and may differ from focus time.
  • Evident from the text: as a survey, it presents no new empirical experiments or primary validation; its findings and numbers are drawn from cited studies, so coverage and conclusions depend on the selected literature.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/2619088

Cites

159 of 159 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 of 137 cited sources checked · 5 supported · 0 flagged · accuracy not yet measured

Show all 75 claims

  • Probably not a substantial claim. 151
  • It may state two separate claims as one. 32
168 of 168
  • The paper relies on this without support.

    The area lacks a systematic arrangement of prior efforts and an overview of state-of-the-art approaches.

    “Despite significant advances, the area lacks a systematic arrangement of prior efforts and an overview of state-of-the-art approaches.”
  • The paper relies on this without support.

    An effective end-to-end temporal retrieval system that exploits temporal information to improve the quality of presented results remains undeveloped.

    “Moreover, an effective end-to-end temporal retrieval system that exploits temporal information to improve the quality of the presented results remain undeveloped.”
  • The paper relies on this without support.

    The value of information and its quality are intrinsically time-dependent.

    “The value of information and its quality are intrinsically time-dependent.”
  • The authors state this as an assumption.

    In web search scenarios, a document's reading time is the same as the time a search query was issued.

    “In web search scenarios, a document's reading time is assumed to be the same as the time a search query was issued since users often access search results immediately after performing a search.”
  • The paper relies on this without support.

    Events are not totally independent from one another.

    “Often events are not totally independent from one another.”
  • The paper relies on this without support.

    Newer information is more reliable than old information.

    “In addition, paying attention to the timestamp of predictions is critical because newer information is more reliable than old information.”
  • The paper relies on this without support.

    Microblog collections like tweets or Facebook posts usually include temporal information.

    “Also important is the emergence of microblog collections, like tweets or Facebook posts that usually include temporal information.”
  • The paper relies on this without support.

    Documents are full of temporal expressions and many have strong temporal characteristics.

    “However, despite the fact that documents are full of temporal expressions and many have strong temporal characteristics, current IR systems still do not sufficiently exploit this information.”
  • The paper relies on this without support.

    Traditional IR systems continue to treat temporal expressions as normal text terms.

    “As an example, when a user's information quest includes temporal aspects, traditional IR systems may fail because they continue to treat temporal expressions as normal text terms.”
  • The paper relies on this without support.

    Most researches developed so far have a rather specific scope.

    “Despite the growing importance of the area, this recent research trend is still without immediate or at least visible effects for average users since most of the researches developed so far have a rather specific scope.”
  • The paper relies on this without support.

    The community still lacks a gold standard to which most approaches can be compared.

    “A further problem is related to the difficulty of evaluating research proposals, since in many cases the community still lacks a gold standard to which most of the approaches can be compared.”
  • The paper relies on this without support.

    Image retrieval queries may sometimes have an explicit or implicit temporal intent.

    “Like in document retrieval, image retrieval queries may sometimes have an explicit or implicit temporal intent.”
  • The paper relies on this without support.

    In unstructured collections such as the web, many images do not contain explicit metadata that can be easily retrieved.

    “However, in unstructured collections such as the web, many images do not contain explicit metadata that can be easily retrieved.”
  • The paper relies on this without support.

    Predicting a query's temporal intent is a critical step to decide appropriate ranking.

    “Predicting a query's temporal intent is a critical step to decide appropriate ranking.”
14 of 14

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Section 4.2 names different preprocessing steps from Section 2.5

    Minor

    Section 4.2 says Section 2.5 already described tokenization, stopping, stemming, and information extraction. Section 2.5 instead lists tokenization, sentence extraction, part-of-speech tagging, and named-entity recognition as its four preprocessing steps.

  • YAGO2 citation year differs from its reference

    Minor

    The text cites the Hoffart et al. YAGO2 work as 2011, but the matching YAGO2 entry in the references is dated 2013.

  • Reinanda citation year differs from its reference

    Minor

    The text cites Reinanda et al. as 2013, while the listed reference for Reinanda et al. is dated 2012.

3 of 3

Citations

Cites

Flagged first, then by citations

Show all 166 references

7 of the 166 references couldn’t be matched to a known paper.

Research works printed but not linked: 19

  1. Advanced Research Projects Agency . 1993 . Software and intelligent systems technology office . In Proceedings of the 5th Conference on Message Understanding. Morgan Kaufmann. Advanced Research Projects Agency. 1993. Software and intelligent systems technology office. In Proceedings of the 5th Conference on Message Understanding. Morgan Kaufmann.
  2. O. Alonso , R. Baeza-Yates , and M. Gertz . 2007a. Exploratory search using timelines . In Proceedings of the Workshop on Exploratory Search and Computer Human Interaction Associated to CHI'07: SIGCHI. ACM. O. Alonso, R. Baeza-Yates, and M. Gertz. 2007a. Exploratory search using timelines. In Proceedings of the Workshop on Exploratory Search and Computer Human Interaction Associated to CHI'07: SIGCHI. ACM.
  3. O. Alonso , R. Baeza-Yates , and M. Gertz . 2009b. Effectiveness of temporal snippets . In Proceedings of the WSRSP'09 Workshop associated to WWW'09 . ACM. O. Alonso, R. Baeza-Yates, and M. Gertz. 2009b. Effectiveness of temporal snippets. In Proceedings of the WSRSP'09 Workshop associated to WWW'09. ACM.
  4. R. Baeza-Yates . 2005 . Searching the future . In Proceedings of the Mathematical/Formal Methods in Information Retrieval Workshop Associated to SIGIR'05 . ACM. R. Baeza-Yates. 2005. Searching the future. In Proceedings of the Mathematical/Formal Methods in Information Retrieval Workshop Associated to SIGIR'05. ACM.
  5. K. Bollacker , P. Tufts , T. Pierce , and R. Cook . 2007. A platform for scalable, collaborative, structured information integration . In Proceedings of the IIWeb'07 Workshop Associated to AAAI'07 . AAAI Press, 22--27. K. Bollacker, P. Tufts, T. Pierce, and R. Cook. 2007. A platform for scalable, collaborative, structured information integration. In Proceedings of the IIWeb'07 Workshop Associated to AAAI'07. AAAI Press, 22--27.
  6. 10.5555/297805.297827 · DOI
  7. H. Cunningham , D. Maynard , K. Bontcheva , and V. Tablan . 2002. GATE: A framework and graphical development environment for robust NLP tools and applications . In Proceedings of ACL'02 . Association for Computational Linguistics, 168--175. H. Cunningham, D. Maynard, K. Bontcheva, and V. Tablan. 2002. GATE: A framework and graphical development environment for robust NLP tools and applications. In Proceedings of ACL'02. Association for Computational Linguistics, 168--175.
  8. M. Efron . 2012 . Query-specific recency ranking: Survival analysis for improved microblog retrieval . In Proceedings of the TAIA'12 Workshop associated to SIGIR'12 . M. Efron. 2012. Query-specific recency ranking: Survival analysis for improved microblog retrieval. In Proceedings of the TAIA'12 Workshop associated to SIGIR'12.
  9. N. Kanhabua , C. Niederee , and W. Siberski . 2013. Towards concise preservation by managed forgetting: Research issues and case study . In Proceedings of the iPRES'13 . N. Kanhabua, C. Niederee, and W. Siberski. 2013. Towards concise preservation by managed forgetting: Research issues and case study. In Proceedings of the iPRES'13.
  10. H. Li , G. Xu , W. B. Croft , M. Bendersky , Z. Wang , and E. Viegas . 2012. A public dataset for promoting query representation and understanding research . In Proceedings of the WSCD'12 . H. Li, G. Xu, W. B. Croft, M. Bendersky, Z. Wang, and E. Viegas. 2012. A public dataset for promoting query representation and understanding research. In Proceedings of the WSCD'12.
  11. M. Matthews , P. Tolchinsky , R. Blanco , J. Atserias , P. Mika , and H. Zaragoza . 2010. Searching through time in the New York Times . In Proceedings of the HCIR'10 Workshop. 41--44 . M. Matthews, P. Tolchinsky, R. Blanco, J. Atserias, P. Mika, and H. Zaragoza. 2010. Searching through time in the New York Times. In Proceedings of the HCIR'10 Workshop. 41--44.
  12. 10.1002/asi.v58:13 · DOI
  13. S. Nunes , C. Ribeiro , and G. David . 2008. Use of temporal expressions in web search . In Proceedings of the ECIR'08 . Springer-Verlag. 580--584. S. Nunes, C. Ribeiro, and G. David. 2008. Use of temporal expressions in web search. In Proceedings of the ECIR'08. Springer-Verlag. 580--584.
  14. D. Odijk , G. Santucci , M. de Rijke , M. Angelini , and G. Granato . 2012. Exploring word meaning through time . In Proceedings of the TAIA'12 Workshop Associated to SIGIR'12 . D. Odijk, G. Santucci, M. de Rijke, M. Angelini, and G. Granato. 2012. Exploring word meaning through time. In Proceedings of the TAIA'12 Workshop Associated to SIGIR'12.
  15. M. Osborne , S. Petrovic , R. McCreadie , C. Macdonald , and I. Ounis . 2012. Bieber no more: First story detection using Twitter and Wikipedia . In Proceedings of the TAIA'12 Workshop Associated to SIGIR'12 . M. Osborne, S. Petrovic, R. McCreadie, C. Macdonald, and I. Ounis. 2012. Bieber no more: First story detection using Twitter and Wikipedia. In Proceedings of the TAIA'12 Workshop Associated to SIGIR'12.
  16. J. Pustejovsky , M. Verhagen , R. Sauri , J. Littman , R. J. Gaizauskas , G. Katz , I. Mani , R. Knippen , and A. Setzer . 2006. TimeBank 1.2 . In Proceedings of the Linguistic Data Consortium. J. Pustejovsky, M. Verhagen, R. Sauri, J. Littman, R. J. Gaizauskas, G. Katz, I. Mani, R. Knippen, and A. Setzer. 2006. TimeBank 1.2. In Proceedings of the Linguistic Data Consortium.
  17. B. Shaparenko , R. Caruana , J. Gehrke , and T. Joachims . 2005. Identifying temporal patterns and key players in document collections . In Proceedings of the TDM'05 Workshop Associated to ICDM'05 . IEEE Computer Society Press, 165--174. B. Shaparenko, R. Caruana, J. Gehrke, and T. Joachims. 2005. Identifying temporal patterns and key players in document collections. In Proceedings of the TDM'05 Workshop Associated to ICDM'05. IEEE Computer Society Press, 165--174.
  18. 10.1145/318898.318921 · DOI
  19. 10.5555/1620853.1620899 · DOI

Cited by

Sorted by citations

Items 201–261 of 261

Metadata

Authors
Campos, R., Dias, G., Jorge, A. M., & Jatowt, A.
Affiliations
  • University of Beira Interior (PT)
  • GREYC (FR)
  • LIAAD -- INESC TEC, DCC-FCUP, University of Porto, Porto, Portugal
  • Kyoto University (JP)
Venue
ACM Computing Surveys
Published
2014-08-25
Volume
47
Issue
2
Pages
1-41
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W2039297031
Cited by
262
References
166
Citations by year
2026: 16 · 2025: 12 · 2024: 30 · 2023: 17 · 2022: 12 · 2021: 23 · 2020: 16 · 2019: 21 · 2018: 25 · 2017: 35 · 2016: 38 · 2015: 15 · 2014: 2
FWCI
51.88
Type
journal article
Language
en
Open access
Locations
  • ACM Computing Surveys (publishedVersion)
  • Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
  • HAL (submittedVersion)
Funders
  • European Regional Development Fund (ERDF)
  • Center of Mathematics, University of Beira Interior, within project PEst-OE/MAT/UI0212/2014
  • Project NORTE-07-0124-FEDER-000059
  • JST research promotion program Presto
  • National Strategic Reference Framework (NSRF)
  • MEXT Grant-in-Aid for Young Scientists B (#22700096)
  • North Portugal Regional Operational Programme (ON.2 -- O Novo Norte)
  • National Funds through the FCT within project FCOMP-01-0124-FEDER-037281
  • Portuguese funding agency (FCT)
  • ERDF through the COMPETE Programme (operational programme for competitiveness)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Mathematics › Theoretical Computer Science › Algorithms and Data Compression
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
temporal information retrieval, temporal queries, research categorization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review