The DOI matches its Crossref record; 241 of the 241 references it cites were found in a registry or an index.
Methods for Evaluating Interactive Information Retrieval Systems with Users
Abstract
This paper provides overview and instruction regarding the evaluation of interactive information retrieval systems with users. The primary goal of this article is to catalog and compile material related to this topic into a single source. This article (1) provides historical background on the development of user-centered approaches to the evaluation of interactive information retrieval systems; (2) describes the major components of interactive information retrieval system evaluation; (3) describes different experimental designs and sampling strategies; (4) presents core instruments and data collection techniques and measures; (5) explains basic data analysis techniques; and (4) reviews and discusses previous studies. This article also discusses validity and reliability issues with respect to both measures and methods, presents background information on research ethics and discusses some ethical issues which are specific to studies of interactive information retrieval (IIR). Finally, this article concludes with a discussion of outstanding challenges and future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
The Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,245
Basics of Qualitative Research: Grounded Theory Procedures and Techniques.
Ellis, C., Strauss, A., & Corbin, J. (1992) · Contemporary Sociology · doi:10.2307/2074814 · Cited by 29,249
Handbook of Qualitative Research
Crozier, G., Denzin, N., & Lincoln, Y. (1994) · British Journal of Educational Studies · doi:10.2307/3121684 · Cited by 26,666
(1963) · Medical Entomology and Zoology · Cited by 15,085
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
The Practice of Social Research
Neustadtl, A., & Babbie, E. (1989) · Teaching Sociology · doi:10.2307/1318433 · Cited by 14,398
Research works printed but not linked: 133
- Adar, 2007, Proceedings of the Workshop on Query Log Analysis: Social and Technological Challenges, at the 16th International World Wide Web Conference
- AgichteinBrillDumais, 2006, Proceedings of the 29th Annual ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’06), 3
- Allan, 2004, TREC2003, Proceedings of the 12th Text Retrieval Conference
- Allan, 2006, TREC2005, Proceedings of the l4th Text Retrieval Conference
- Allen, 1997, Information Seeking in Context: Proceedings of an International Conference on Research in Information Needs, Seeking and Use in Different Contexts, 111
- Amdur, 2003, Institutional Review Board Member Handbook
- American Psychological Association, 2001, Publication Manual of the American Psychological Association
- AmitayMurrayTeevan, 2007, Proceedings of the 16th International World Wide Web Conference
- Anick, 2003, Proceedings of the 26th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ‘03), 88
- ArapakisJose, 2008, Proceedings of the 31st Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ‘08), 395
- BaileyCraswellSoboroffThomasde VriesYilmaz, 2008, Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’08), 667
- BeaulieuGatfordHuangRobertsonWalkerWilliams, 1997, Proceedings of the 5th Text Retrieval Conference (TREC-5), 143
- Belkin, 1980, “Anomalous states of knowledge as a basis for information retrieval”, Canadian Journal of Information Science, 5, 133
- BelkinCabezasCoolKimNgParkPressmanRiehSavageXie, 1997, Proceedings of the 5th Text Retrieval Conference (TREC-5), 257
- BelkinKellyKimKimLeeMuresanTangYuanCool, 2003, Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 205
- BellRuthven, 2004, Proceedings of the 26th Annual International European Conference on Information Retrieval (ECIR 200fi), 57
- Bennett, 1972, “The user interface in interactive systems”, Annual Review of Information Science and Technology, 7, 159
- Bernal, 1948, “Preliminary analysis of pilot questionnaires on the use of scientific literature”, The Royal Society Scientific Information Conference, 589
- BilenkoWhite, 2008, Proceedings of the 17th International Conference on the World Wide Web (WWW ’08), 51
- Borgman, 1983, Proceedings of the 6th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’83), 162
- BorlundIngwersen, 1998, Proceedings of the 21st ACM SIGIR Conference on Research and Development of Information Retrieval (SIGIR ’98), 324
- BorlundRuthven, 2008, “Introduction to the special issue on evaluating interactive information retrieval systems”, Information Processing and Management, 44, 1
- BorlundSchneiderLaimasTombrosFeatherKellyde VriesAzzopardi, 2008, Proceedings of the 2nd International Symposium on Information Interaction in Context
- BoyceMeadowKraft, 1994, Measurement in Information Science
- BudzikHammond, 2000, Proceedings of the 5th International Conference on Intelligent User Interfaces (IUI TO), 44
- Capra, 2008, Personal Information Management, CHI 2008 Workshop
- Charmaz, 2002, Handbook of Interview Research: Context and Method
- Cleverdon, 1997, Readings in Information Retrieval, 173
- CowleyHaackLittlefieldHampson, 2006, Proceedings of the 2nd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences (CARPE ’05), 13
- CzerwinskiHorvitzCutrell, 2001, Proceedings of IHM-HCI 2001 Conference, 167
- CzerwinskiHorvitzWilhite, 2004, Proceedings of ACM Human Factors in Computing Systems Conference (CHI ’Of), 175
- DahlbäckJonssonAhrenberg, 1993, Proceedings for the 1st International Conference on Intelligent User Interfaces (IUI ’93), 193
- DangKellyLin, 2008, TREC2007, Proceedings of the 16th Text Retrieval Conference
- De Mey, 1977, CC77: International Workshop on the Cognitive Viewpoint, xvi
- Dervin, 1992, Qualitative Research in Information Management, 61
- Dervin, 1996, Information Seeking in Context: Proceedings of an International Conference on Research in Information Needs, Seeking and Use in Different Contexts, 13
- DominickPenniman, 1979, Proceedings of the 2nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’79), 2
- DumaisCutrellCadizJanckeSarinRobbins, 2003, Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR ’03), 72
- Dunlop, 1997, Proceedings of the 20th ACM Conference on Research and Development in Information Retrieval (SIGIR ’97), 206
- ElsweilerRuthven, 2007, Proceedings of the 30th ACM Conference on Research and Development in Information Retrieval (SIGIR ’03), 22
- FenstermacherGinsburg, 2003, “Client-side monitoring for Web mining”, Journal of the American Society for Information Science and Technoloqy, 54, 625, 10.1002/asi.10255 · doi:10.1002/asi.10255
- Fiske, 1982, Forms of Validity in Research, 77
- FittsJonesMilton, 1950, “Eye movements of aircraft pilots during instrument-landing approaches”, Aeronautical Engineering Review, 9, 24
- GhaniSupnickRooney, 1991, Proceedings of International Conference on Information Systems (JCIS 1991), 229
- GrimesTangRussell, 2007, Proceedings of the Workshop on Query Log Analysis: Social and Technology Challenges at the 16th International World Wide Web Conference
- GuanLeeCuddihyRamey, 2006, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 1253
- HaradaNaamanSongWangPaepcke, 2004, Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL T4), 325
- Harman, 2005, TREC: Experiment and Evaluation in Information Retrieval, 21
- HartStaveland, 1988, Human Mental Workload, 139
- HawkingBaileyCraswell, 2000, “Efficient and flexible search using text and metadata”, CSIRO Mathematical and Information Sciences Tech Report, 2000-83
- Hersh, 2006, Advances in Open Domain Question Answering, 431
- HershOver, 2001, “Introduction to a special issue on interactivity at the Text Retrieval Conference (TREC)”, Information Processing and Management, 37, 365, 10.1016/S0306-4573(00)00052-2 · doi:10.1016/s0306-4573(00)00052-2
- HershTurpinPriceChanKramerSacherekOlson, 2000, Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR TO), 17
- HershElliotHickamWolfMolnarLeichtenstein, 1995, Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’95), 164
- HirshDinkelacker, 2004, “Seeking information in order to produce information: An empirical study at Hewlett Packard Labs”, Journal of the American Society for Information Science and Technology, 55, 807, 10.1002/asi.20024 · doi:10.1002/asi.20024
- HuangWang, 2004, “The influence of document presentation order and number of documents judged on users’ judgments of relevance”, Journal of the American Society for Information Science, 55, 970, 10.1002/asi.20047 · doi:10.1002/asi.20047
- IngwersenWillett, 1995, “An introduction to algorithmic and cognitive approaches for information retrieval”, Libri, 45, 160
- International Standards Office, 1998, Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs): Part II, Guidance on Usability
- JacobKarn, 2003, The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, 573
- JansenRamadossZhangZang, 2006, Workshop on Evaluating Exploratory Search Systems at the 29th Annual International ACM SIGIR Conference on Research and Development on Information Retrieval (SIGIR ‘06)
- JärvelinKekäläinen, 2000, Proceedings of the 23rd ACM SIGIR Conference on Research and Development of Information Retrieval (SIGIR ’00), 41
- JärvelinPriceDelcambreNielsen, 2008, Proceedings of the 30th European Conference on Information Retrieval
- Jex, 1988, Human Mental Workload, 5
- JoachimsGrankaPanHembrookeGay, 2005, Proceedings of the 28th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’05), 154
- JohoHannahJose, 2008, Proceedings of the 2nd IHX Symposium on Information Interaction in Context, 89
- JonesTeevan, 2007, Personal Information Management
- KäkiAula, 2008, “Controlling the complexity in comparing search user interfaces via user studies”, Information Processing and Management, 44, 82, 10.1016/j.ipm.2007.02.006 · doi:10.1016/j.ipm.2007.02.006
- KantorKazaiMilic-FraylingWilkinson, 2008, Proceedings of the Conference on Information and Knowledge Management (CIKM ’08)
- Kantor, 1982, “Evaluation of and feedback in information storage and retrieval systems”, Annual Review of Information Science and Technology, 17, 99
- KellyFu, 2006, Proceedings of the 29th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’06), 453
- KellyKantorMorseScholtzSun, 2006, Proceedings of the Workshop on Interactive Question Answering at the Human Language Technology Conference (HLT-NAACL ’06)
- KellyLin, 2007, “Overview of the TREC 2006 ciQA task”, SIGIR Forum, 41, 107, 10.1145/1273221.1273231 · doi:10.1145/1273221.1273231
- KellyShahSugimotoBaileyClemensIrvineJohnsonKeOhPoljakovaRodriguezvan NoordZhang, 2008, Proceedings of the 2nd Symposium on Information Interaction in Context (IIiX), 75
- KellyWacholderRittmanSunKantorSmallStrzalkowski, 2007, “Using interview data to identify evaluation criteria for interactive, analytical question answering systems”, Journal of the American Society for Information Science and Technology, 58, 1032, 10.1002/asi.20575 · doi:10.1002/asi.20575
- King, 1968, “Design and evaluation of information systems”, Annual Review of Information Science and Technology, 3, 61
- LarsenMalikTombros, 2006, INEX 2005, 398
- Likert, 1932, “A technique for the measurement of attitudes”, Archives of Psychology, 140, 1
- LinQuanSinhaBakshiHuynhKatzKarger, 2003, Proceedings of the 9th IFIP TCI3 International Conference on Human-Computer Interaction (INTERACT 2003)
- LinSmucker, 2008, Proceedings of the 31st International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 08), 19
- LinBelkin, 2000, Proceedings of the Annual Conference of the American Society for Information Science (ASIS TO), 133
- Losee, 1996, “Evaluating retrieval performance given database and query characteristics: Analytical determination of performance surfaces”, Journal of the American Society for Information Science, 47, 95, 10.1002/(SICI)1097-4571(199601)47:1<95::AID-ASI9>3.0.CO;2-Y · doi:10.1002/(sici)1097-4571(199601)47:1<95::aid-asi9>3.0.co;2-y
- Luhn, 1958, H. P. Luhn: Pioneer of Infor-mation Science. Selected Works, 132
- Lund, 2001, “Measuring usability with the USE Questionnaire”, Usability and User Experience, 8
- Mackay, 1995, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 138
- Marchionini, 2008, “Toward human-Computer information retrieval”, Bulletin of the American Society for Information Science
- Michel, 1994, “What is used during cognitive processing in information retrieval and library searching? Eleven sources of search information”, Journal of the American Society for Information Science, 45, 498, 10.1002/(SICI)1097-4571(199408)45:7<498::AID-ASI5>3.0.CO;2-T · doi:10.1002/(sici)1097-4571(199408)45:7<498::aid-asi5>3.0.co;2-t
- MillettFriedmanFelten, 2001, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 46
- Morahan-Martin, 1998, Psychology and the Internet: Intrapersonal, Interpersonal, and Transpersonal Implications, 169
- MorrisHorvitz, 2007, Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST ’07), 3
- MyersWell, 2003, Research Design and Statistical Analysis
- Nielsen, 2008, “Usability 101: Introduction to usability”, Jakob Nielsen’s Alertbox
- Oard, 2000, Proceedings of the 1st Cross-Language Evaluation Forum
- O’BrienTomsKellowayKelley, 2008, Proceedings of the American Society for Information Science and Technology
- PetersKurthFlahertySandoreKaske, 1993, “An introduction to the special section on transaction log analysis”, Library Hi Tech, 11, 38, 10.1108/eb047883 · doi:10.1108/eb047883
- Petrelli, 2008, “On the role of user-centred evaluation in the advancement of interactive information retrieval”, Information Processing and Management, 44, 22, 10.1016/j.ipm.2007.01.024 · doi:10.1016/j.ipm.2007.01.024
- Robertson, 2006, Proceedings of the 15th ACM international Conference on information and Knowledge Management (CIKM’06), 78
- RoddenFu, 2007, Proceedings of the Web Information-Seeking and Interaction ( WIST) Workshop at the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’Of), 29
- Rorvig, 1988, “Psychometric measurement and information retrieval”, Annual Review of Information Science and Technology, 23, 157
- Ruthven, 2006, Proceedings of the 1st International Conference on Information Interaction in Context, 10.1145/1164820 · doi:10.1145/1164820
- Saracevic, 1967, Proceedings of the Annual Meeting of the American Documentation Institute, 4, 100
- Saracevic, 1995, Proceedings of the 18th Annual ACM SIGIR Conference on Research and Development of Information Retrieval, 138
- SaracevicKantor, 1988, “A study of information seeking and retrieving. II. Users, questions and effectiveness”, Journal of the American Society for Information Science, 39, 177, 10.1002/(SICI)1097-4571(198805)39:3<177::AID-ASI3>3.0.CO;2-F · doi:10.1002/(sici)1097-4571(198805)39:3<177::aid-asi3>3.0.co;2-f
- Savage-KnepshieldBelkin, 1999, “Interaction in information retrieval: Trends over time”, Journal of the American Society for Information Science, 50, 1067, 10.1002/(SICI)1097-4571(1999)50:12<1067::AID-ASI6>3.0.CO;2-4 · doi:10.1002/(sici)1097-4571(1999)50:12<1067::aid-asi6>3.0.co;2-4
- Siatri, 1999, “The evolution of user studies”, Libri, 49, 132, 10.1515/libr.1999.49.3.132 · doi:10.1515/libr.1999.49.3.132
- SmithKantor, 2008, Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’08), 147
- Sormunen, 2002, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’02), 324
- Spärck-Jones, 1981, Information Retrieval Experiment
- Spink, 1996, “Multiple search session model of end-user behavior: An exploratory study”, Journal of the American Society for Information Science, 47, 603, 10.1002/(SICI)1097-4571(199608)47:8<603::AID-ASI4>3.0.CO;2-X · doi:10.1002/(sici)1097-4571(199608)47:8<603::aid-asi4>3.0.co;2-x
- Spink, 1997, “Study of interactive feedback during mediated information retrieval”, Journal of the American Society for Information Science, 48, 382, 10.1002/(SICI)1097-4571(199705)48:5<382::AID-ASI2>3.0.CO;2-R · doi:10.1002/(sici)1097-4571(199705)48:5<382::aid-asi2>3.0.co;2-r
- Stern, 2003, “Encountering distressing information in online research: A consideration of legal and ethical responsibilities”, New Media and Society, 5, 249, 10.1177/1461444803005002006 · doi:10.1177/1461444803005002006
- Sugar, 1995, “User-centered perspectives of information retrieval research and analysis methods”, Annual Review of Information Science and Technology, 30, 77
- SunKantor, 2006, “Cross-evaluation: A new model for information system evaluation”, Journal of American Society for Information Science and Technology, 56, 614
- Tague, 1987, “Informativeness as an ordinal utility function for information retrieval”, SIGIR Forum, 21, 10, 10.1145/30075.30077 · doi:10.1145/30075.30077
- Tague, 1981, Information Retrieval Experiment, 59
- Tague-Sutcliffe, 1992, Proceedings of the 15th Annual AGM International Conference on Research and Development in Information Retrieval (SIGIR ’92), 23
- TeevanAlvaradoAckermanKarger, 2004, Proceedings of the AGM SIGCHI Conference on Human Factors in Computing Systems (SIGCHI ’04), 415
- TeevanDumaisHorvitz, 2005, Proceedings of the 28th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’05), 449
- ThomasHawking, 2006, Proceedings of the 15th Annual Conference on Information and Knowledge Management (CIKM ’06), 94
- TurpinHersh, 2001, Proceedings of the 24th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’01), 225
- TurpinScholer, 2006, Proceedings of the 29th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’06), 11
- TurtlePennimanHickey, 1981, “Data entry/display devices for interactive information retrieval”, Annual Review of Information Science and Technology, 16, 55
- Urquhart, 1948, “The distribution and use of scientific and technical information”, The Royal Society Scientific Information Conference, 408
- VakkariJärvelin, 2005, New Directions in Cognitive Information Retrieval, 113
- VeerasamyBelkin, 1996, Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 85
- VeerasamyHeikes, 1997, “Effectiveness of a graphical display of retrieval results”, SIGIR Forum, 31, 236, 10.1145/278459.258580 · doi:10.1145/278459.258580
- VoorheesHarman, 2005, TREC: Experiment and Evaluation in Information Retrieval
- Wang, 1999, “Methodologies and methods for user behavior research”, Annual Review of Information Science and Technology, 34, 53
- WenRuthvenBorlund, 2006, Proceedings of the 28th European Conference in Information Retrieval (ECIR 2006’)
- WhiteBilenkoCucerzan, 2007, Proceedings of the 30th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR ’07), 159
- XiongAgichtein, 2007, Proceedings of the Workshop on Query Log Analysis: Social and Technology Challenges at the 16th International World Wide Web Conference
- YuanMeadow, 1999, “A study of the use of variables in information retrieval user studies”, Journal of the American Society for Information Science, 50, 140, 10.1002/(SICI)1097-4571(1999)50:2<140::AID-ASI5>3.0.CO;2-P · doi:10.1002/(sici)1097-4571(1999)50:2<140::aid-asi5>3.0.co;2-p
- ZhangPlettenbergKlavansOardSoergel, 2007, Proceedings of the 7th ACM/IEEE Joint Conference on Digital Libraries (JCDL ’07), 117
- Zhang, 2002, “Collaborative relevance judgments: A group consensus method for evaluating user search performance”, Journal of the American Society for Information Science and Technology, 53, 220, 10.1002/asi.10036 · doi:10.1002/asi.10036
Cited by
Crowdsourcing for relevance evaluation
Alonso, O., Rose, D. E., & Stewart, B. (2008) · ACM SIGIR Forum · doi:10.1145/1480506.1480508 · Cited by 329
Bartneck, C., Duenser, A., Moltchanova, E., & Zawieska, K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0121595 · Cited by 317
Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., & Diaz, F. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3272027 · Cited by 279
Lalmas, M., O’Brien, H., & Yom-Tov, E. (2015) · Synthesis Lectures on Information Concepts, Retrieval, and Services · doi:10.1007/978-3-031-02289-0 · Cited by 192
Kelly, D., Arguello, J., Edwards, A., & Wu, W.-C. (2015) · Proceedings of the 2015 International Conference on The Theory of Information Retrieval · doi:10.1145/2808194.2809465 · Cited by 155
Self-citation
Metadata
- Authors
- Kelly, D.
- Venue
- Foundations and Trends® in Information Retrieval
- Publisher
- Emerald
- Published
- 2009-05-27
- Volume
- 3
- Issue
- 1-2
- Pages
- 1-224
- ISSN
- 1554-0669 (ISSN-L)
- 1554-0677
- OpenAlex
- W2035569891
- Cited by
- 633
- References
- 241
- FWCI
- 60.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methods for Evaluating Interactive Information Retrieval Systems with Users. https://app.cochat.ai/papers/10.1561/1500000012
- The paper
- Kelly, D. (2009). Methods for Evaluating Interactive Information Retrieval Systems with Users. Foundations and Trends® in Information Retrieval. https://doi.org/10.1561/1500000012
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.