The foundation of the concept of relevance
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
In 1975 Tefko Saracevic declared “the subject knowledge view” to be the most fundamental perspective of relevance. This paper examines the assumptions in different views of relevance, including “the system's view” and “the user's view” and offers a reinterpretation of these views. The paper finds that what was regarded as the most fundamental view by Saracevic in 1975 has not since been considered (with very few exceptions). Other views, which are based on less fruitful assumptions, have dominated the discourse on relevance in information retrieval and information science. Many authors have reexamined the concept of relevance in information science, but have neglected the subject knowledge view, hence basic theoretical assumptions seem not to have been properly addressed. It is as urgent now as it was in 1975 seriously to consider “the subject knowledge view” of relevance (which may also be termed “the epistemological view”). The concept of relevance, like other basic concepts, is influenced by overall approaches to information science, such as the cognitive view and the domain‐analytic view. There is today a trend toward a social paradigm for information science. This paper offers an understanding of relevance from such a social point of view.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Structure of Scientific Revolutions
Hall, M. B., & Kuhn, T. S. (1963) · The American Historical Review · doi:10.2307/1847040 · Cited by 7,479
On understanding computers and cognition: A new foundation for design
Winograd, T., & Flores, F. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90026-9 · Cited by 3,677
Carrier, J. G., & Bloor, D. (1977) · The British Journal of Sociology · doi:10.2307/590007 · Cited by 2,355 · cited 2×
“This is a principle which has in particular been associated with the so-called " strong program " in the sociology of science and introduced as one of the four basic tenets of that program by Bloor (1976).”
BackgroundThe Scientific Consensus on Climate Change
Oreskes, N. (2004) · Science · doi:10.1126/science.1103618 · Cited by 1,814 · cited 1×
“31 There have been some studies on the scientific consensus about climate change and Al Gore based his film on Oreskes (2004).”
BackgroundON USER STUDIES AND INFORMATION NEEDS
WILSON, T. D. (1981) · Journal of Documentation · doi:10.1108/eb026702 · Cited by 1,565 · cited 1×
“2 Tom Wilson (1981) and Sundin & Johannisson (2005) also describe the need for the social dimension in the understanding of relevance and information needs within LIS.”
BackgroundRELEVANCE: A review of and a framework for the thinking on the notion in information science
Saracevic, T. (1975) · Journal of the American Society for Information Science · doi:10.1002/asi.4630260604 · Cited by 863 · cited 8×
“Saracevic (1975) is today (July 2009) the most cited paper in Social SciSearch about relevance (s relevance/ti and Sc=Library.”
BackgroundInfluentialToward a new horizon in information science: Domain-analysis
Hjørland, B., & Albrechtsen, H. (1995) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199507)46:6<400::aid-asi2>3.0.co;2-y · Cited by 759
Self-citationShaping the Web: Why the Politics of Search Engines Matters
Introna, L. D., & Nissenbaum, H. (2017) · Computer Ethics · doi:10.4324/9781315259697-19 · Cited by 708
Mizzaro, S. (1997) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199709)48:9<810::aid-asi6>3.0.co;2-u · Cited by 573
Examining the Scientific Consensus on Climate Change
Doran, P. T., & Zimmerman, M. K. (2009) · Eos, Transactions American Geophysical Union · doi:10.1029/2009eo030002 · Cited by 528
The concept of relevance in IR
Borlund, P. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10286 · Cited by 482 · cited 6×
“" There is a contradiction in the assumption that the user is seen as (Borlund, 2003, p. 916). a subjective interpreter, but the researcher studying the users is assumed to " These variables can be standardized by having the same set of be an objective investigator (positivist assumptions…”
BackgroundInfluentialLooking for Information: A Survey of Research on Information Seeking, Needs, and Behavior
(2016) · Studies in Information · doi:10.1108/s2055-53772016020 · Cited by 469
Value-Added Processes in Information Systems
Murphy, P., & Taylor, R. S. (1987) · Journal of Education for Library and Information Science · doi:10.2307/40323680 · Cited by 437
A re-examination of relevance: toward a dynamic, situational definition∗
Schamber, L., Eisenberg, M. B., & Nilan, M. S. (1990) · Information Processing & Management · doi:10.1016/0306-4573(90)90050-c · Cited by 426 · cited 6×
“Schamber et al. (1990, p. 758) claim: " Today it is generally agreed that direct topical matches can be measured.”
BackgroundInfluentialFactors determining the performance of indexing systems
Cleverdon, C. W., Mills, J., & Keen, M. (1966) · Medical Entomology and Zoology · Cited by 414
How many relevances in information retrieval?
Mizzaro, S. (1998) · Interacting with Computers · doi:10.1016/s0953-5438(98)00012-5 · Cited by 314
Epistemology and the socio‐cognitive perspective in information science
Hjørland, B. (2002) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10042 · Cited by 307 · cited 3×
“Simplified relevance criteria in four epistemological schools (from Hjørland, 2002, p. 269).”
BackgroundSelf-citationA definition of relevance for information retrieval
Cooper, W. S. (1971) · Information Storage and Retrieval · doi:10.1016/0020-0271(71)90024-6 · Cited by 304
Hjørland, B. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21082 · Cited by 249 · cited 1×
“I have argued elsewhere (e.g., Hjørland & Albrechtsen, 1995; Hjørland, 2009) that a social paradigm is a better theoretical framework than the cognitive view.”
BackgroundSelf-citationChanges in relevance criteria and problem stages in task performance
Vakkari, P., & Hakala, N. (2000) · Journal of Documentation · doi:10.1108/eum0000000007127 · Cited by 237
Saracevic, T. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20682 · Cited by 227 · cited 4×
“In this connection , it is important to consider Saracevic's statement: " The whole point of the evaluation of different algorithms is that they produce different outputs for the same query and from the same set of documents in the system " (Saracevic, 2007, p. 1930).”
BackgroundInfluentialOrganizing Information: Principles of Data Base and Retrieval Systems
Soergel, D. (1985) · International Linear Collider · Cited by 198
Wilson, P. (1973) · Information Storage and Retrieval · doi:10.1016/0020-0271(73)90096-x · Cited by 188 · cited 4×
“” Sundin & Johannisson wrote: “Despite Wilson’s [1981] argument that”
BackgroundInfluentialCleverdon, C. W., & Keen, M. (1966) · CERES (Cranfield University) · Cited by 172
The sociological turn in information science
Cronin, B. (2008) · Journal of Information Science · doi:10.1177/0165551508088944 · Cited by 169 · cited 3×
“2 This has also been argued by Cronin (2008).”
BackgroundInfluentialGenres and the WEB: Is the personal home page the first uniquely digital genre?
Dillon, A. P., & Gushrowski, B. (2000) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(2000)51:2<209::aid-asi14>3.0.co;2-b · Cited by 147
Tang, R., & Solomon, P. J. (1998) · Information Processing & Management · doi:10.1016/s0306-4573(97)00081-2 · Cited by 113 · cited 1×
“24 Tang & Solomon (1998) is a concrete study because it is about one person's specific search.”
BackgroundAnnual Review of Information Science and Technology: Volume 41
(2008) · Library Review · doi:10.1108/00242530810868797 · Cited by 108
Work tasks and socio‐cognitive relevance: A specific example
Hjørland, B., & Christensen, F. S. (2002) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10132 · Cited by 92
Self-citationIntroduction to information science
Saračević, T. (1970) · Cited by 83
The dilemma of measurement in information retrieval research
Ellis, D. (1996) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199601)47:1<23::aid-asi3>3.0.co;2-4 · Cited by 83
Do Web search engines suppress controversy?
Gerhart, S. (2004) · First Monday · doi:10.5210/fm.v9i1.1111 · Cited by 69 · cited 2×
“, by Gerhart, 2004, and by Introna & Nissenbaum, 2000). This issue is an example of the subjectivity of the system. The examination of this kind of subjectivity is only possible from a social perspective, and it is opposed to the claim made by cognitive researchers that “the systems view of relevance” is objective.7 A…”
BackgroundResultSubject Access Points in Electronic Retrieval.
Hjørland, B., & Nielsen, L. K. (2001) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 65
Self-citationDeveloping the library: Between efficiency, accountability and forms of recognition
Kann-Rasmussen, N., & Andersen, J. (2009) · Journal of Documentation · doi:10.1108/00220410910937589 · Cited by 63 · cited 1×
“• Its interest in users may be interpreted as related to commercial-like interest in customers (and New Public Management philosophy, cf. Kann-Christensen & Andersen, 2009).”
BackgroundModels of User Satisfaction: Understanding False Positives
Applegate, R. (1993) · IUScholarWorks (Indiana University) · Cited by 53
A note on the concept of “relevance”
Foskett, D. J. (1972) · Information Processing & Management · doi:10.1016/0020-0271(72)90009-5 · Cited by 51
Saracevic, T. (2008) · Library Trends · doi:10.1353/lib.0.0000 · Cited by 44 · cited 5×
“This type of testing is now relegated to history (Saracevic, 2008, p. 772).”
BackgroundInfluentialNovelty and topicality in interactive information retrieval
Xu, Y. C., & Yin, H. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20709 · Cited by 43 · cited 1×
“…" topicality " is mostly " being on the topic, " i.e., about subject relations: " Topicality measures the 'aboutness' of a document to the topic area suggested by a query " (Xu & Yin, 2008, p. 202) or " Topicality, the relation of a document to the topic of a user's query " (Janes, 1994, p. 161).”
BackgroundThe Instrumentality of Information Needs and Relevance
Sundin, O., & Johannisson, J. (2005) · Lecture notes in computer science · doi:10.1007/11495222_10 · Cited by 39 · cited 2×
“2 Tom Wilson (1981) and Sundin & Johannisson (2005) also describe the need for the social dimension in the understanding of relevance and information needs within LIS.”
BackgroundOn the evaluation of interactive information retrieval systems
Belkin, N. J. (2010) · Rutgers University eBooks · doi:10.7282/t3sf2tjk · Cited by 39 · cited 7×
“Borlund, for example, writes: “In this approach [the system-driven] queries are assumed [to be] identical to static information needs, and relevance judgments are assumed to be of an objective nature of whether or not the document is about the query, and can as such be made by any knowledgeable person” (Borlund, 2000,…”
BackgroundMethodResultInfluentialCoiera, E. W., & Vickland, V. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2663 · Cited by 10 · cited 1×
“Finally, a recent study by Coiera and Vickland (2008) found that user-provided relevance rankings seem to be of limited to no value when designing a search engine, whereas relevance rankings may have a place in situations in which experts provide rankings.”
BackgroundCompromised need and the label effect: An examination of claims and evidence
Nicolaisen, J. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21129 · Cited by 7
(2001) · doi:10.1002/1097-4571(2000)9999:9999<::aid-asi1542>3.0.co;2-t
(1966) · CERES (Cranfield University)
(1994)
Cited by
Eriksen, M. B., & Frandsen, T. F. (2018) · Journal of the Medical Library Association · doi:10.5195/jmla.2018.345 · Cited by 1,216
“, the ranking of results or a study being present in the search results) is not sufficient [20].”
BackgroundEvolution of library and information science, 1965–2005: Content analysis of journal articles
Tuomaala, O., Järvelin, K., & Vakkari, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23034 · Cited by 129
Hjørland, B. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23250 · Cited by 47
“As discussed in Hjørland (2010, note 43, p. 235), a given document may be considered relevant to a given task or may be postponed for a later task.”
BackgroundSelf-citation
Metadata
- Authors
- Hjørland, B.
- Venue
- Journal of the American Society for Information Science and Technology
- Publisher
- Wiley
- Published
- 2009-11-20
- Volume
- 61
- Issue
- 2
- Pages
- 217-237
- ISSN
- 1532-2882 (ISSN-L)
- 1532-2890
- OpenAlex
- W2319223103
- Cited by
- 160
- References
- 45
- FWCI
- 22.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.