The DOI matches its Crossref record; 125 of the 135 references it cites were found in a registry or an index.
Task-Based Information Interaction Evaluation: The Viewpoint of Program Theory
Abstract
Evaluation is central in research and development of information retrieval (IR). In addition to designing and implementing new retrieval mechanisms, one must also show through rigorous evaluation that they are effective. A major focus in IR is IR mechanisms’ capability of ranking relevant documents optimally for the users, given a query. Searching for information in practice involves searchers, however, and is highly interactive. When human searchers have been incorporated in evaluation studies, the results have often suggested that better ranking does not necessarily lead to better search task, or work task, performance. Therefore, it is not clear which system or interface features should be developed to improve the effectiveness of human task performance. In the present article, we focus on the evaluation of task-based information interaction (TBII). We give special emphasis to learning tasks to discuss TBII in more concrete terms. Information interaction is here understood as behavioral and cognitive activities related to task planning, searching information items, selecting between them, working with them, and synthesizing and reporting. These five generic activities contribute to task performance and outcome and can be supported by information systems. In an attempt toward task-based evaluation, we introduce program theory as the evaluation framework. Such evaluation can investigate whether a program consisting of TBII activities and tools works and how it works and, further, provides a causal description of program (in)effectiveness. Our goal in the present article is to structure TBII on the basis of the five generic activities and consider the evaluation of each activity using the program theory framework. Finally, we combine these activity-based program theories in an overall evaluation framework for TBII. Such an evaluation is complex due to the large number of factors affecting information interaction. Instead of presenting tested program theories, we illustrate how the evaluation of TBII should be accomplished using the program theory framework in the evaluation of systems and behaviors, and their interactions, comprehensively in context.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282 · cited 1×
“Task planning and reflective assessment also intertwines with the actor’s metacognitive process, and the task itself [Flavell 1979].”
BackgroundBloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956) · Cited by 9,342
A revision of bloom's taxonomy: an overview
Krathwohl, D. R. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4104_2 · Cited by 8,132
Plans and Situated Actions: The Problem of Human-Machine Communication
Kantowitz, B., & Suchman, L. A. (1990) · The American Journal of Psychology · doi:10.2307/1423221 · Cited by 5,299
Evaluation: A Systematic Approach
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (1979) · Cited by 2,943
Inside the search process: Information seeking from the user's perspective
Kuhlthau, C. C. (1991) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199106)42:5<361::aid-asi6>3.0.co;2-# · Cited by 2,166
Evaluation: A Systematic Approach
Wallace, D. P. (2005) · Library & Information Science Research · doi:10.1016/j.lisr.2004.09.009 · Cited by 1,922
The design of browsing and berrypicking techniques for the online search interface
Bates, M. J. (1989) · Online Review · doi:10.1108/eb024320 · Cited by 1,757
Plans and situated actions: The problem of human-machine communication
Agre, P. E. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90078-e · Cited by 1,669
Plans and Situated Actions: The Problem of Human Machine Communication.
Woolgar, S., & Suchman, L. A. (1989) · Contemporary Sociology · doi:10.2307/2073874 · Cited by 1,604
Exploratory search: from finding to understanding
Marchionini, G. (2006) · Communications of the ACM · doi:10.1145/1121949.1121979 · Cited by 1,506
Promoting General Metacognitive Awareness
Schraw, G. (2001) · Neuropsychology and Cognition · doi:10.1007/978-94-017-2243-8_1 · Cited by 1,245
Promoting general metacognitive awareness
Schraw, G. (1998) · Instructional Science · doi:10.1023/a:1003044231033 · Cited by 1,088
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025
Scriven, M. S. (1991) · Cited by 956
Task complexity affects information seeking and use
Byström, K., & Järvelin, K. (1995) · Information Processing & Management · doi:10.1016/0306-4573(94)00041-z · Cited by 885
Self-citation(2005) · The Information Retrieval Series · doi:10.1007/1-4020-3851-8 · Cited by 864
Self-citationRELEVANCE: A review of and a framework for the thinking on the notion in information science
Saracevic, T. (1975) · Journal of the American Society for Information Science · doi:10.1002/asi.4630260604 · Cited by 863
Ingwersen, P., & Järvelin, K. (2005) · Springer eBooks · Cited by 758
Self-citationAction Research: Its Nature and Validity
Checkland, P., & Holwell, S. (1998) · Systemic Practice and Action Research · doi:10.1023/a:1022908820784 · Cited by 731
A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011) · American Journal of Evaluation · doi:10.1177/1098214010389321 · Cited by 531 · cited 1×
“This includes causal description and causal explanation, as well as determining program ( )effectiveness [Coryn et al. 2011; Rossi et al. 2004].”
BackgroundTest Collection Based Evaluation of Information Retrieval Systems
Sanderson, M. (2010) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000009 · Cited by 485 · cited 1×
“This is often done through test-collection based experiments not involving users directly [Sanderson 2010].”
BackgroundThe concept of relevance in IR
Borlund, P. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10286 · Cited by 482
User performance versus precision measures for simple search tasks
Turpin, A., & Scholer, F. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148176 · Cited by 389
Task‐based information searching
Vakkari, P. (2003) · Annual Review of Information Science and Technology · doi:10.1002/aris.1440370110 · Cited by 366
Self-citationTask complexity, problem structure and information actions
Vakkari, P. (1999) · Information Processing & Management · doi:10.1016/s0306-4573(99)00028-x · Cited by 348
Self-citationWhere should the person stop and the information search interface start?
Bates, M. J. (1990) · Information Processing & Management · doi:10.1016/0306-4573(90)90103-9 · Cited by 342
The effects of domain knowledge on search tactic formulation
Wildemuth, B. M. (2004) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10367 · Cited by 341
The Constructivist Metaphor: Reading, Writing, and the Making of Meaning
Spivey, N. N. (1997) · College Composition and Communication · doi:10.2307/358470 · Cited by 338
A faceted approach to conceptualizing tasks in information seeking
Li, Y., & Belkin, N. J. (2008) · Information Processing & Management · doi:10.1016/j.ipm.2008.07.005 · Cited by 336
Experimental components for the evaluation of interactive information retrieval systems
Borlund, P. (2000) · Journal of Documentation · doi:10.1108/eum0000000007110 · Cited by 307
Vakkari, P. (2001) · Journal of Documentation · doi:10.1108/eum0000000007075 · Cited by 294
Self-citationProgram theory evaluation: Practice, promise, and problems
Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000) · New Directions for Evaluation · doi:10.1002/ev.1177 · Cited by 274
Users' criteria for relevance evaluation: A cross-situational comparison
Barry, C. L., & Schamber, L. (1998) · Information Processing & Management · doi:10.1016/s0306-4573(97)00078-2 · Cited by 272
Clinical Decision Support Systems
(2007) · Health Informatics · doi:10.1007/978-0-387-38319-4 · Cited by 265
The skills of document use: from text comprehension to Web-based learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5174 · Cited by 260
Writing from Sources, Writing from Sentences
Howard, R. M., Serviss, T., & Rodrigue, T. K. (2010) · Writing & Pedagogy · doi:10.1558/wap.v2i2.177 · Cited by 256
Cosijn, E., & Ingwersen, P. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(99)00072-2 · Cited by 245
Kuhlthau, C. C. (1999) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1999)50:5<399::aid-asi3>3.0.co;2-l · Cited by 245
Changes in relevance criteria and problem stages in task performance
Vakkari, P., & Hakala, N. (2000) · Journal of Documentation · doi:10.1108/eum0000000007127 · Cited by 237
Self-citationInformation search process of lawyers: a call for ’just for me’ information services
Kuhlthau, C. C., & Tama, S. L. (2001) · Journal of Documentation · doi:10.1108/eum0000000007076 · Cited by 234
Evaluating computer-supported cooperative work: models and frameworks
Neale, D. C., Carroll, J. M., & Rosson, M. B. (2004) · Proceedings of the 2004 ACM conference on Computer supported cooperative work · doi:10.1145/1031607.1031626 · Cited by 224
A diary study of work-related reading: design implications for digital reading devices
Adler, A., Gujar, A., Harrison, B. L., O'Hara, K., & Sellen, A. (1998) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '98 · doi:10.1145/274644.274679 · Cited by 224
Conceptual framework for tasks in information studies
Byström, K., & Hansen, P. (2005) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20197 · Cited by 220
A cognitive model of document use during a research project. Study I. Document selection
Wang, P., & Soergel, D. (1998) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1998)49:2<115::aid-asi3>3.0.co;2-1 · Cited by 213
Changes of search terms and tactics while writing a research proposal
Vakkari, P., Pennanen, M., & Serola, S. (2003) · Information Processing & Management · doi:10.1016/s0306-4573(02)00031-6 · Cited by 199
Self-citationMendeley: Creating Communities of Scholarly Inquiry Through Research Collaboration
Holt Zaugg, R. E. W., West, R. E., Tateishi, I., & Randall, D. L. (2011) · TechTrends · doi:10.1007/s11528-011-0467-y · Cited by 198
Heinström, J. (2005) · Journal of Documentation · doi:10.1108/00220410510585205 · Cited by 184
The Role of Metacognition in Enhancing Learning
Biggs, J. (1988) · Australian Journal of Education · doi:10.1177/000494418803200201 · Cited by 178
Clinical Decision Support Systems
Helmons, P. (2009) · Pharmacy Informatics · doi:10.1201/9781420071764-c15 · Cited by 176
Two first-year students’ strategies for writing from sources: Patchwriting or plagiarism?
Li, Y., & Casanave, C. P. (2012) · Journal of Second Language Writing · doi:10.1016/j.jslw.2012.03.002 · Cited by 174
Using Writing Tasks to Elicit Adolescents’ Historical Reasoning
Monte-Sano, C., & De La Paz, S. (2012) · Journal of Literacy Research · doi:10.1177/1086296x12450445 · Cited by 165
Information seeking and use by newspaper journalists
Attfield, S., & Dowell, J. D. (2003) · Journal of Documentation · doi:10.1108/00220410310463860 · Cited by 153
What is browsing - really? A model drawing from behavioural science research.
Bates, M. J. (2007) · Information Research · Cited by 152
Modeling Interaction in Information Retrieval (IR): A Review and Proposal.
Saračević, T. (1996) · Cited by 148
Information Retrieval as a Trial-And-Error Process
Swanson, D. R. (1977) · The Library Quarterly · doi:10.1086/620653 · Cited by 145
Identifying task-based sessions in search engine query logs
Lucchese, C., Orlando, S., Perego, R., Silvestri, F., & Tolomei, G. (2011) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1935826.1935875 · Cited by 144
Modeling Information Content Using Observable Behavior
Oard, D. W., & Kim, J. (2001) · Scholar Commons (University of South Carolina) · Cited by 143
Dillon, A. P., & Turnbull, D. (2009) · CRC Press eBooks · doi:10.1081/e-elis3-120017582 · Cited by 128
Mateos, M., & Solé, I. (2009) · European Journal of Psychology of Education · doi:10.1007/bf03178760 · Cited by 124 · cited 2×
“There is an obvious need to develop systems that actively retrieve and suggest semantic components from the repository of selected information items relevant in a particular writing situation.”
BackgroundWriting from Sources: The Effect of Explicit Instruction on College Students' Processes and Products
Segev-Miller, R. (2004) · L1 Educational Studies in Language and Literature · doi:10.1023/b:esll.0000033847.00732.af · Cited by 122
Distributed expert problem treatment as a model for information system analysis and design
Belkin, N. J., Seeger, T., & Wersig, G. (1982) · Journal of Information Science · doi:10.1177/016555158200500501 · Cited by 119
A task-oriented approach to information retrieval evaluation
Hersh, W., Pentecost, J. O., & Hickam, D. H. (1996) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199601)47:1<50::aid-asi5>3.0.co;2-1 · Cited by 117
Todd, R. J. (2006) · Information Research · Cited by 116
Carr, M., Kurtz‐Costes, B., Schneider, W., Turner, L. A., & Borkowski, J. G. (1989) · Developmental Psychology · doi:10.1037//0012-1649.25.5.765 · Cited by 116
Heinström, J. (2005) · Information Research · Cited by 114
BLANDFORD, A. E., & Attfield, S. (2010) · Choice Reviews Online · doi:10.5860/choice.48-1518 · Cited by 113
Working on Understanding During Collaborative Online Reading
Kiili, C., Laurinen, L. I., Marttunen, M. J., & Leu, D. J. (2012) · Journal of Literacy Research · doi:10.1177/1086296x12457166 · Cited by 112
Argumentation support: from technologies to tools
de Moor, A., & Aakhus, M. A. (2006) · Communications of the ACM · doi:10.1145/1118178.1118182 · Cited by 102
User adaptation: good results from poor systems
Smith, C. L., & Kantor, P. B. (2008) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1390334.1390362 · Cited by 93
Information retrieval: A sequential learning process
Bookstein, A. (1983) · Journal of the American Society for Information Science · doi:10.1002/asi.4630340504 · Cited by 87
Macevičiūtė, E. (2010) · Borås Academic Digital Archive (University of Borås) · Cited by 85
BLANDFORD, A. E., & Attfield, S. (2010) · Synthesis lectures on human-centered informatics · doi:10.2200/s00227ed1v01y200911hci006 · Cited by 85
Model-driven formative evaluation of exploratory search: A study under a sensemaking framework
Qu, Y., & Furnas, G. W. (2007) · Information Processing & Management · doi:10.1016/j.ipm.2007.09.006 · Cited by 83
Shah, C. S. (2012) · doi:10.1007/978-3-642-28813-5 · Cited by 79
Relevance and contributing information types of searched documents in task performance
Vakkari, P. (2000) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/345508.345512 · Cited by 78
Self-citationSummaries, ranked retrieval and sessions: a unified framework for information access evaluation
Sakai, T., & Dou, Z. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484031 · Cited by 76
Relevance and retrieval evaluation: Perspectives from medicine
Hersh, W. (1994) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199404)45:3<201::aid-asi9>3.0.co;2-w · Cited by 74
Interactive Information Retrieval in Digital Environments
Xie, I. (2008) · IGI Global eBooks · doi:10.4018/978-1-59904-240-4 · Cited by 68
Reading and writing the electronic book
Marshall, C. (1986) · Microelectronics Journal · doi:10.1016/s0026-2692(86)80020-9 · Cited by 64
Vakkari, P. (2001) · Information Retrieval · doi:10.1023/a:1016089224008 · Cited by 62
Self-citationMedical students' personal knowledge, searching proficiency, and database use in problem solving
Wildemuth, B. M., de Bliek, R., Friedman, C. P., & File, D. D. (1995) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199509)46:8<590::aid-asi7>3.0.co;2-# · Cited by 62
Modeling task-genre relationships for IR in the workplace
Freund, L., Toms, E. G., & Clarke, C. L. A. (2005) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1076034.1076110 · Cited by 58
Using Metacognitive Theories to Design User Interfaces for Computer-Based Learning.
Jones, M. G. (1995) · Educational Technology archive · Cited by 52
Search effort degrades search output but improves task outcome
Vakkari, P., & Huuskonen, S. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21683 · Cited by 52
Self-citationKekäläinen, J., & Järvelin, K. (n.d.) · Cited by 50
Self-citationClinical Decision Support Systems: Theory and Practice (Health Informatics)
Berner, E. S. (2006) · Springer eBooks · Cited by 48
Almighty Twitter, what are people asking for?
Liu, Z., & Jansen, B. J. J. (2012) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504901134 · Cited by 47
A taxonomy of adolescent metacognitive knowledge during the information search process
Bowler, L. (2009) · Library & Information Science Research · doi:10.1016/j.lisr.2009.09.005 · Cited by 46
Measuring improvement in user search performance resulting from optimal search tips
Moraveji, N., Russell, D. M., Bien, J., & Mease, D. (2011) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2009916.2009966 · Cited by 44
Information interaction in molecular medicine: integrated use of multiple channels
Kumpulainen, S., & Järvelin, K. (2010) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/1840784.1840800 · Cited by 41
Self-citationExploratory Searching As Conceptual Exploration
Vakkari, P. (2010) · Cited by 41
Self-citationBarriers to task‐based information access in molecular medicine
Kumpulainen, S., & Järvelin, K. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21672 · Cited by 40
Self-citationThe user's mental model of an information retrieval system
Borgman, C. L. (1985) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/253495.253533 · Cited by 38
Assessing learning outcomes in two information retrieval learning environments
Halttunen, K., & Järvelin, K. (2004) · Information Processing & Management · doi:10.1016/j.ipm.2004.02.004 · Cited by 37
Self-citationTask-based Information Seeking and Retrieval in the Patent Domain: Processes and Relationships
Hansen, P. (2011) · Tampere University Institutional Repository (Tampere University) · Cited by 36
Pearson, J. S., Buchanan, G. R., & Thimbleby, H. (2013) · Synthesis lectures on information concepts, retrieval, and services · doi:10.2200/s00539ed1v01y201310icr029 · Cited by 36
Students' group work strategies in source‐based writing assignments
Sormunen, E., Tanni, M., Alamettälä, T., & Heinström, J. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23032 · Cited by 34
Self-citationA subjunctive exploratory search interface to support media studies researchers
Bron, M., Gorp, J. V., Nack, F., de Rijke, M., Vishneuski, A., & de Leeuw, S. (2012) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2348283.2348342 · Cited by 33
A critical review of research on information behavior in assigned learning tasks
Tanni, M., & Sormunen, E. (2008) · Journal of Documentation · doi:10.1108/00220410810912442 · Cited by 30
Self-citationClient information system as an everyday information tool in child protection work
Huuskonen, S., & Vakkari, P. (2010) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/1840784.1840788 · Cited by 29
Self-citationExplaining User Performance in Information Retrieval: Challenges to IR Evaluation
Järvelin, K. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04417-5_28 · Cited by 28
Self-citationBig Macs and Eigenfactor scores: Don't let correlation coefficients fool you
Eschenfelder, K. R. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.v61:9 · Cited by 27
Expected reading effort in focused retrieval evaluation
Arvola, P., Kekäläinen, J., & Junkkari, M. (2010) · Information Retrieval · doi:10.1007/s10791-010-9133-9 · Cited by 27
Self-citationChen, C., & Lin, S.-T. (2014) · Computers & Education · doi:10.1016/j.compedu.2014.02.006 · Cited by 25
Searching vs. writing: Factors affecting information use task performance
Liu, J., & Belkin, N. J. (2012) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504901127 · Cited by 23
Task complexity and information searching in administrative tasks revisited
Saastamoinen, M., Kumpulainen, S., & Järvelin, K. (2012) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/2362724.2362759 · Cited by 22
Self-citationContextualization models for XML retrieval
Arvola, P., Kekäläinen, J., & Junkkari, M. (2011) · Information Processing & Management · doi:10.1016/j.ipm.2011.02.006 · Cited by 22
Self-citationSormunen, E., & Lehtiö, L. (2011) · Tampere University Institutional Repository (Tampere University) · Cited by 19
Self-citationWu, I., Liu, D., & Chang, P. (2008) · Information Processing & Management · doi:10.1016/j.ipm.2007.11.005 · Cited by 18
A method for the analysis of information use in source-based writing
Sormunen, E., Heinström, J., Romu, L., & Turunen, R. (2012) · Tampere University Institutional Repository (Tampere University) · Cited by 16
Self-citation(2013) · Online Information Review · doi:10.1108/oir-04-2013-0084 · Cited by 16
Kumpulainen, S. (2013) · Tampere University Institutional Repository (Tampere University) · doi:10.13140/rg.2.1.3352.8808 · Cited by 15
Self-citationKumpulainen, S. (2014) · Journal of Documentation · doi:10.1108/jd-06-2013-0082 · Cited by 15
Self-citationSome measures and procedures for evaluation of the user interface in an information retrieval system
Tague, J. M., & Schultz, R. (1988) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/62437.62477 · Cited by 15
Interactive Information Retrieval in Digital Environments
Borlund, P. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21110 · Cited by 13
Pearson, J. S., Buchanan, G. R., & Thimbleby, H. (2014) · Synthesis lectures on information concepts, retrieval, and services · doi:10.1007/978-3-031-02330-9 · Cited by 11
Video needs at the different stages of television program making process
Markkula, M., & Sormunen, E. (2006) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/1164820.1164844 · Cited by 10
Self-citationButcher, K. R., Davies, S. R., Crockett, A., Dewald, A., & Zheng, R. Z. (2011) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/1998076.1998136 · Cited by 9
Ruthven, I. G. (2012) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/2362724.2362747 · Cited by 8
Supporting semantic navigation
Kopak, R., Freund, L., & O'Brien, H. L. (2010) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/1840784.1840839 · Cited by 6
Nurse, A. (2014) · International Journal of Information Management · Cited by 6
Browsing patterns in retrieved documents
Kekäläinen, J., Arvola, P., & Kumpulainen, S. (2014) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/2637002.2637047 · Cited by 2
Self-citationClinical Decision Support Systems
Mendon√ßa, E., Jenders, R. A., & Lussier, Y. A. (2002) · Best practices · doi:10.1201/9781420031409.ch17 · Cited by 2
(2006) · DOAJ (DOAJ: Directory of Open Access Journals)
Research works printed but not linked: 11
- P. Arvola and J. Kekäläinen . 2010. Simulating user interaction in result document browsing . In Workshop Proceedings of the 33rd Annual International ACM SIGIR Conference: Simulation of Interaction, Automated Evaluation of Interactive IR. ACM , New York, NY, 27--28. P. Arvola and J. Kekäläinen. 2010. Simulating user interaction in result document browsing. In Workshop Proceedings of the 33rd Annual International ACM SIGIR Conference: Simulation of Interaction, Automated Evaluation of Interactive IR. ACM, New York, NY, 27--28.
- N. J. Belkin . 1980 . Anomalous states of knowledge as a basis for information retrieval . Can. J. Inform. Lib. Sci. 5 (1980), 133 -- 143 . N. J. Belkin. 1980. Anomalous states of knowledge as a basis for information retrieval. Can. J. Inform. Lib. Sci. 5 (1980), 133--143.
- N. J. Belkin . 2010. On the evaluation of interactive information retrieval systems . In The Janus Faced Scholar. A Festschrift in Honor of Peter Ingwersen . Det Informationsvidenskabelige Akademi (Royal School of Library and Information Science , Copenhagen ); ISSI (International Society for Scientometrics and Informetrics), Copenhagen, Denmark, 13--22. N. J. Belkin. 2010. On the evaluation of interactive information retrieval systems. In The Janus Faced Scholar. A Festschrift in Honor of Peter Ingwersen. Det Informationsvidenskabelige Akademi (Royal School of Library and Information Science, Copenhagen); ISSI (International Society for Scientometrics and Informetrics), Copenhagen, Denmark, 13--22.
- 10.1002/asi.v56:10 · doi:10.1002/asi.v56:10
- 10.1037/0012-1649.25.5.765 · doi:10.1037/0012-1649.25.5.765
- C. Cool and N. J. Belkin . 2002. A classification of interactions with information. In Emerging Frameworks and Methods , Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS4) . Greenwood Village, CO, Libraries Unlimited, 1--15. C. Cool and N. J. Belkin. 2002. A classification of interactions with information. In Emerging Frameworks and Methods, Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS4). Greenwood Village, CO, Libraries Unlimited, 1--15.
- S. Goldman K. Lawless and F. Manning. 2013. Research and development of multiple source comprehension assessment. In Reading - From Words to Multiple Texts M. A. Britt S. R. Goldman and J.-F. Rouet (Eds.). Routledge New York NY 180--199. S. Goldman K. Lawless and F. Manning. 2013. Research and development of multiple source comprehension assessment. In Reading - From Words to Multiple Texts M. A. Britt S. R. Goldman and J.-F. Rouet (Eds.). Routledge New York NY 180--199.
- 10.1002/(sici)1097-4571(199601)47:1%3c50::aid-asi5%3e3.3.co;2-y · doi:10.1002/(sici)1097-4571(199601)47:1<50::aid-asi5>3.3.co;2-y
- K. Järvelin . 2011. Evaluation . In Interactive Information Seeking, Behaviour and Retrieval, I. Ruthven and D. Kelly (Eds.). Facet Publishing , London, UK , 113--138. K. Järvelin. 2011. Evaluation. In Interactive Information Seeking, Behaviour and Retrieval, I. Ruthven and D. Kelly (Eds.). Facet Publishing, London, UK, 113--138.
- 10.1002/aris.144.v42:1 · doi:10.1002/aris.144.v42:1
- T. Saracevic . 1996 b. Relevance reconsidered ’96. In Information science: Integration in Perspectives . Proceedings of the 2nd Conference on Conceptions of Library and Information Science. The Royal School of Librarianship, Copenhagen, DK, 201--218 . T. Saracevic. 1996b. Relevance reconsidered ’96. In Information science: Integration in Perspectives. Proceedings of the 2nd Conference on Conceptions of Library and Information Science. The Royal School of Librarianship, Copenhagen, DK, 201--218.
Cited by
Li, Y., & Liu, C. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2018.10.012 · Cited by 79
Effects of task complexity on online search behavior of adolescents
Walhout, J., Oomen, P., Jarodzka, H., & Brand‐Gruwel, S. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23782 · Cited by 62
Kruschwitz, U., & Hull, C. (2017) · now publishers, Inc. eBooks · doi:10.1561/9781680833058 · Cited by 58
Late, E., & Kumpulainen, S. (2021) · Journal of Documentation · doi:10.1108/jd-04-2021-0078 · Cited by 52
Self-citationvon Hoyer, J. F., Hoppe, A., Kammerer, Y., Otto, C. A., Pardi, G., Rokicki, M., Yu, R., Dietze, S., Ewerth, R., & Holtz, P. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.827748 · Cited by 45
Metadata
- Authors
- Järvelin, K., Vakkari, P., Arvola, P., Baskaya, F., Järvelin, A., Kekäläinen, J., Keskustalo, H., Kumpulainen, S., Saastamoinen, M., Savolainen, R., & Sormunen, E.
- Affiliations
- University of Tampere (FI)
- University of Tampere, Finland
- ORCID
- Venue
- ACM Transactions on Information Systems
- Published
- 2015-02-17
- Volume
- 33
- Issue
- 1
- Pages
- 1-30
- ISSN
- 1046-8188 (ISSN-L)
- 1558-2868
- OpenAlex
- W2013726998
- Cited by
- 80
- References
- 135
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 6 · 2023: 8 · 2022: 7 · 2021: 14 · 2020: 4 · 2019: 7 · 2018: 13 · 2017: 6 · 2016: 2 · 2015: 1
- FWCI
- 8.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Information Systems (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Academy of Finland (#133021)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Keywords
- information retrieval evaluation, program theory, learning tasks, information seeking, task performance, causality assessment, information system effectiveness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.