Interactive IR User Study Design, Evaluation, and Reporting

Liu, Jiqun; Shah, Chirag

doi:10.1007/978-3-031-02319-4

Cited by 26Information Retrieval and Search BehaviorUsability and User Interface DesignRecommender Systems and Techniques

The DOI matches its Crossref record; 112 of the 112 references it cites were found in a registry or an index.

Interactive IR User Study Design, Evaluation, and Reporting

Liu, Jiqun, Shah, Chirag·2019·Synthesis Lectures on Information Concepts, Retrieval, and Services

DOI 10.1007/978-3-031-02319-4

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1007/978-3-031-02319-4

Cites

112 of 112 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Üstün, Y. E. (2019) · DergiPark (Istanbul University) · Cited by 169,622
Untitled work
(2014) · Cited by 15,747
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941
RELEVANCE: A review of and a framework for the thinking on the notion in information science
Saračević, T. (1975) · Journal of the American Society for Information Science · doi:10.1002/asi.4630260604 · Cited by 863
COGNITIVE PERSPECTIVES OF INFORMATION RETRIEVAL INTERACTION: ELEMENTS OF A COGNITIVE IR THEORY
INGWERSEN, P. (1996) · Journal of Documentation · doi:10.1108/eb026960 · Cited by 717
Methods for Evaluating Interactive Information Retrieval Systems with Users
Kelly, D. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000012 · Cited by 693
Stuff I've Seen
Dumais, S., Cutrell, E. B., Cadiz, J. J., Jancke, G., Sarin, R., & Robbins, D. C. (2016) · ACM SIGIR Forum · doi:10.1145/2888422.2888425 · Cited by 691
Book Review: Integrating Research: A Guide for Literature Reviews
Kolbe, R. H., & Cooper, H. M. (1991) · Journal of Marketing Research · doi:10.2307/3172876 · Cited by 598
Evaluation on crowdsourcing research: Current status and future direction
Zhao, Y. C., & Zhu, Q. (2012) · Information Systems Frontiers · doi:10.1007/s10796-012-9350-4 · Cited by 566
Information retrieval system evaluation
Sanderson, M., & Zobel, J. (2005) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1076034.1076064 · Cited by 382
A large scale study of wireless search behavior
Kamvar, M., & Baluja, S. (2006) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124877 · Cited by 278
The cognitive viewpoint in information science
Belkin, N. J. (1990) · Journal of Information Science · doi:10.1177/016555159001600104 · Cited by 253
Conceptual framework for tasks in information studies
Byström, K., & Hansen, P. (2005) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20197 · Cited by 220
Helping people find what they don't know
Belkin, N. J. (2000) · Communications of the ACM · doi:10.1145/345124.345143 · Cited by 214
How does search behavior change as search becomes more difficult?
Aula, A., Khan, R. M., & Guan, Z. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753333 · Cited by 209
Some(what) grand challenges for information retrieval
Belkin, N. J. (2008) · ACM SIGIR Forum · doi:10.1145/1394251.1394261 · Cited by 149
Informing the Design of Spoken Conversational Search
Trippas, J. R., Spina, D., Cavedon, L., Joho, H., & Sanderson, M. (2018) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3176349.3176387 · Cited by 147
Potential for personalization
Teevan, J. B., Dumais, S., & Horvitz, E. (2010) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1721831.1721835 · Cited by 145
Distribution of cognitive load in Web search
Jacek Gwizdka, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21385 · Cited by 129
Towards better measurement of attention and satisfaction in mobile search
Lagun, D., Hsieh, C.-H., Webster, D. R., & Navalpakkam, V. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609631 · Cited by 126
A study of medical and health queries to web search engines
Spink, A. H., Yang, Y. P., Jansen, B. J. J., Nykanen, P., Lorence, D. P., Özmutlu, S., & Özmutlu, H. C. (2004) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2004.00481.x · Cited by 123
Searching, browsing, and clicking in a search session
Jiang, J., He, D., & Allan, J. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609633 · Cited by 123
Search behaviors in different task types
Liu, J., Cole, M., Liu, C., Bierig, R., Jacek Gwizdka, A., Belkin, N. J., Zhang, J., & Zhang, X. (2010) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/1816123.1816134 · Cited by 116
Self-citation
Exploring and exploiting user search behavior on mobile and tablet devices to improve search relevance
Song, Y., Ma, H., Wang, H., & Wang, K. (2013) · International World Wide Web Conference (WWW) · doi:10.1145/2488388.2488493 · Cited by 115
On the history of evaluation in IR
Robertson, S. E. (2008) · Journal of Information Science · doi:10.1177/0165551507086989 · Cited by 115
A systematic review of interactive information retrieval evaluation studies, 1967–2006
Kelly, D., & Sugimoto, C. R. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22799 · Cited by 109
Incorporating User Expectations and Behavior into the Measurement of Search Effectiveness
Moffat, A., Bailey, P., Scholer, F., & Thomas, P. (2017) · ACM Transactions on Information Systems · doi:10.1145/3052768 · Cited by 108
When does Relevance Mean Usefulness and User Satisfaction in Web Search?
Mao, J., Liu, Y., Zhou, K., Nie, J., Song, J., Zhang, M., Ma, S., Sun, J., & Luo, H. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2911507 · Cited by 103
Modelling interaction with economic models of search
Azzopardi, L. A. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609574 · Cited by 102
User Activity Patterns During Information Search
Cole, M., Hendahewa, C., Belkin, N. J., & Shah, C. S. (2015) · ACM Transactions on Information Systems · doi:10.1145/2699656 · Cited by 101
Self-citation
Characterizing relevance with eye-tracking measures
Jacek Gwizdka, A. (2014) · Information Interaction in Context Symposium (IIiX) · doi:10.1145/2637002.2637011 · Cited by 94
Curiosity Killed the Cat, but Makes Crowdwork Better
Law, E., Yin, M., Goh, J., Chen, K., Terry, M. A., & Gajos, K. Z. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858144 · Cited by 92
Predicting term-relevance from brain signals
Eugster, M. J. A., Ruotsalo, T., Spapé, M., Kosunen, I., Barral, O., Ravaja, N., Jacucci, G., & Samuel Kaski, A. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609594 · Cited by 89
Win-win search
Luo, J., Zhang, S., & Yang, G. H. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609629 · Cited by 88
How query cost affects search behavior
Azzopardi, L. A., Kelly, D., & Brennan, K. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484049 · Cited by 88
Interactive information retrieval
Ruthven, I. G. (2008) · Annual Review of Information Science and Technology · doi:10.1002/aris.2008.1440420109 · Cited by 88
Temporal dynamics of eye‐tracking and EEG during reading and relevance decisions
Jacek Gwizdka, A., Hosseini, R., Cole, M., & Wang, S. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23904 · Cited by 84
Exploring the Role of Conversational Cues in Guided Task Support with Virtual Assistants
Vtyurina, A., & Fourney, A. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173782 · Cited by 83
A two-dimensional click model for query auto-completion
Li, Y., Dong, A., Wang, H., Deng, H., Chang, Y., & Zhai, C. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609571 · Cited by 81
How many results per page?
Kelly, D., & Azzopardi, L. A. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767732 · Cited by 80
An exploration of the relationships between work task and interactive information search behavior
Li, Y., & Belkin, N. J. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21359 · Cited by 79
Increasing Reproducibility in IR
Ferro, N., Fuhr, N., Järvelin, K., Kando, N., Lippold, M., & Zobel, J. (2016) · ACM SIGIR Forum · doi:10.1145/2964797.2964808 · Cited by 73
Evaluating the impact of MeSH (Medical Subject Headings) terms on different types of searchers
Liu, Y., & Wacholder, N. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2017.03.004 · Cited by 71
Personalized web exploration with task models
Ahn, J., Brusilovsky, P. L., He, D., Grady, J. P., & Li, Q. L. (2008) · International World Wide Web Conference (WWW) · doi:10.1145/1367497.1367499 · Cited by 70
Statistical Significance, Power, and Sample Sizes
Sakai, T. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2911492 · Cited by 70
Interactive Intent Modeling for Exploratory Search
Ruotsalo, T., Peltonen, J., Eugster, M. J. A., Głowacka, D., Floréen, P., Myllymäki, P. J., Jacucci, G., & Samuel Kaski, A. (2018) · ACM Transactions on Information Systems · doi:10.1145/3231593 · Cited by 70
The search dashboard: how reflection and comparison impact search behavior
Bateman, S., Teevan, J., & White, R. W. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208311 · Cited by 69
SIGIR Initiative to Implement ACM Artifact Review and Badging
Ferro, N., & Kelly, D. (2018) · ACM SIGIR Forum · doi:10.1145/3274784.3274786 · Cited by 69
Awareness in collaborative information seeking
Shah, C. S., & Marchionini, G. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21379 · Cited by 66
Self-citation
Elicitation of term relevance feedback
Kelly, D., & Fu, X. (2006) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1148170.1148249 · Cited by 65
Exploring the relationships between work task and search task in information search
Li, Y. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20977 · Cited by 62
Designing for Exploratory Search on Touch Devices
Klouche, K., Ruotsalo, T., Cabral, D., Andolina, S., Bellucci, A., & Jacucci, G. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702489 · Cited by 60
Task, Information Seeking Intentions, and User Behavior
Liu, J., Mitsui, M. R., Belkin, N. J., & Shah, C. S. (2019) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298922 · Cited by 59
Self-citation
Meta-evaluation of Online and Offline Web Search Evaluation Metrics
Chen, Y., Zhou, K., Liu, Y., Zhang, M., & Ma, S. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080804 · Cited by 57
Skill discovery in virtual assistants
White, R. W. (2018) · Communications of the ACM · doi:10.1145/3185336 · Cited by 56
Characterizing the value of personalizing search
Teevan, J. B., Dumais, S., & Horvitz, E. (2007) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1277741.1277894 · Cited by 55
Learning to Diversify Search Results via Subtopic Attention
Jiang, Z., Wen, J.-R., Dou, Z., Zhao, W. X., Nie, J., & Yue, M. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080805 · Cited by 54
An effective implicit relevance feedback technique using affective, physiological and behavioural features
Moshfeghi, Y., & Jose, J. M. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484074 · Cited by 51
Retrieval Algorithms Optimized for Human Learning
Syed, R., & Collins‐Thompson, K. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080835 · Cited by 50
How Does Domain Expertise Affect Users’ Search Interaction and Outcome in Exploratory Search?
Mao, J., Liu, Y., Kando, N., Zhang, M., & Ma, S. (2018) · ACM Transactions on Information Systems · doi:10.1145/3223045 · Cited by 50
Multidimensional relevance modeling via psychometrics and crowdsourcing
Zhang, Y., Zhang, J., Lease, M., & Jacek Gwizdka, A. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609577 · Cited by 49
Personalization of search results using interaction behaviors in search sessions
Liu, C., Belkin, N. J., & Cole, M. (2012) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2348283.2348314 · Cited by 49
Proceedings of the Conference
Kretschy, F., & Vienna, M. G. (1931) · Bee World · doi:10.1080/0005772x.1931.11092979 · Cited by 49
A study of the use of simulated work task situations in interactive information retrieval evaluations
Borlund, P. (2016) · Journal of Documentation · doi:10.1108/jd-06-2015-0068 · Cited by 49
Image search by concept map
Xu, H., Wang, J., Hua, X., & Li, S. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835497 · Cited by 48
Studying trailfinding algorithms for enhanced web search
Singla, A. K., White, R. W., & Huang, J. J. S. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835524 · Cited by 48
Using Information Scent to Understand Mobile and Desktop Web Search Behavior
Ong, K., Järvelin, K., Sanderson, M., & Scholer, F. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080817 · Cited by 47
Understanding the Needs of Searchers with Dyslexia
Morris, M. R., Fourney, A., Ali, A. X., & Vonessen, L. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173609 · Cited by 46
Task-aware query recommendation
Feild, H. A., & Allan, J. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484069 · Cited by 45
Predicting users' domain knowledge in information retrieval using multiple regression analysis of search behaviors
Zhang, X., Liu, J., Cole, M., & Belkin, N. J. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23218 · Cited by 44
Self-citation
Searching on the Go
Harvey, M., & Pointon, M. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080770 · Cited by 44
RepliCHI
Wilson, M. L., Ed, H., Reeves, S., & Coyle, D. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2559233 · Cited by 43
Differences in the Use of Search Assistance for Tasks of Varying Complexity
Capra, R., Arguello, J., Crescenzi, A., & Vardell, E. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767741 · Cited by 42
An Eye-Tracking Study of Query Reformulation
Eickhoff, C., Dungs, S., & Tran, V. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767703 · Cited by 42
Predicting Information Seeking Intentions from Search Behaviors
Mitsui, M. R., Liu, J., Belkin, N. J., & Shah, C. S. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080737 · Cited by 42
Self-citation
Query Priming for Promoting Critical Thinking in Web Search
Yamamoto, Y., & Yamamoto, T. (2018) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3176349.3176377 · Cited by 42
Human performance and retrieval precision revisited
Smucker, M. D., & Jethani, C. P. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835549 · Cited by 39
Aggregated search interface preferences in multi-session search tasks
Bron, M., Gorp, J. V., Nack, F., Baltussen, L. B., & de Rijke, M. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484050 · Cited by 37
Learning to Adaptively Rank Document Retrieval System Configurations
Deveaud, R., Mothe, J., Ullah, M. Z., & Nie, J. (2018) · ACM Transactions on Information Systems · doi:10.1145/3231937 · Cited by 37
Overview of the CLEF eHealth Evaluation Lab 2018
Suominen, H., Kelly, L., Goeuriot, L., Aurélie Névéol, A., Ramadier, L., Robert, A., Kanoulas, E., Spijker, R., Azzopardi, L. A., Li, D., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98932-7_26 · Cited by 37
Differences in Eye-Tracking Measures Between Visits and Revisits to Relevant and Irrelevant Web Pages
Jacek Gwizdka, A., & Zhang, Y. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767795 · Cited by 36
"Satisfaction with Failure" or "Unsatisfied Success"
Liu, M., Liu, Y., Mao, J., Luo, C., Zhang, M., & Ma, S. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3186065 · Cited by 36
Work and work roles: a context of tasks
Huvila, I. (2008) · Journal of Documentation · doi:10.1108/00220410810912406 · Cited by 32
Collaborative Dynamic Queries
Hong, S.-S., Suh, M., Riche, N. H., Lee, J., Kim, J., & Zachry, M. R. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173640 · Cited by 32
The Effects of Manipulating Task Determinability on Search Behaviors and Outcomes
Capra, R., Arguello, J., O'Brien, H., Li, Y., & Choi, B. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210047 · Cited by 32
Engaged or Frustrated?
Edwards, A., & Kelly, D. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080818 · Cited by 31
Here and Now
Büschel, W., Mitschick, A., & Dachselt, R. (2018) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3176349.3176384 · Cited by 29
Supporting multiple information-seeking strategies in a single system framework
Yuan, X., & Belkin, N. J. (2007) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1277741.1277786 · Cited by 28
Comparing In Situ and Multidimensional Relevance Judgments
Jiang, J., He, D., & Allan, J. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080840 · Cited by 28
ScentBar
Umemoto, K., Yamamoto, T., & Tanaka, K. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2911546 · Cited by 26
Inferring Searcher Attention by Jointly Modeling User Interactions and Content Salience
Lagun, D., & Agichtein, E. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767745 · Cited by 25
The use of query auto-completion over the course of search sessions with multifaceted information needs
Smith, C. L., Jacek Gwizdka, A., & Feild, H. A. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2017.05.001 · Cited by 25
Evaluating Mobile Search with Height-Biased Gain
Luo, C., Liu, Y., Sakai, T., Zhang, F., Zhang, M., & Ma, S. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080795 · Cited by 25
Evaluating interactive information retrieval systems
Belkin, N. J., Dumais, S., Scholtz, J. C., & Wilkinson, R. G. (2004) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/985921.986162 · Cited by 24
Toward a unified model of human information behavior: an equilibrium perspective
Liu, J. (2017) · Journal of Documentation · doi:10.1108/jd-06-2016-0080 · Cited by 23
Self-citation
Discrimination between tasks with user activity patterns during information search
Cole, M., Hendahewa, C., Belkin, N. J., & Shah, C. S. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609591 · Cited by 21
Self-citation
TeddIR
Jansen, M., Bos, W. M., van der Vet, P., Huibers, T. W. C., & Hiemstra, D. (2010) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/1810543.1810592 · Cited by 21
User Behaviour and Task Characteristics
He, J., & Yılmaz, E. (2017) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3020165.3020188 · Cited by 21
Investigating users' query formulations for cognitive search intents
Kato, M. P., Yamamoto, T., Ohshima, H., & Tanaka, K. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609566 · Cited by 20
Salton Award Lecture
Belkin, N. J. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767854 · Cited by 17
EUSUM
Wan, X., Li, H., & Xiao, J. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835532 · Cited by 16
To Blend or Not to Blend?
Turpin, L., Kelly, D., & Arguello, J. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2914739 · Cited by 15
Information Fostering - Being Proactive with Information Seeking and Retrieval
Shah, C. S. (2018) · doi:10.1145/3176349.3176389 · Cited by 15
Self-citation
User involvement and system support in applying search tactics
Xie, I., Joo, S., & Bennett‐Kapusniak, R. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23765 · Cited by 14
Asking "Good" Questions
O'Brien, H. L., & McCay‐Peet, L. (2017) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3020165.3020167 · Cited by 13
Internet Search Roles of Adults in their Homes
Kim, J., McNally, B., Norooz, L., & Druin, A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025572 · Cited by 11
Calibration
Borisov, A., Kiseleva, J., Markov, I., & de Rijke, M. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3269260 · Cited by 11
NeuroIIR
Jacek Gwizdka, A., & Mostafa, J. (2017) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3020165.3022165 · Cited by 10
Juggling with Information Sources, Task Type, and Information Quality
Wang, Y., Sarkar, S., & Shah, C. S. (2018) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3176349.3176390 · Cited by 9
Self-citation
Panopticon as an eLearning support search tool
Nicholson, J., Huber, M. W., Jackson, D., & Olivier, P. L. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557082 · Cited by 6
Information attribute motivators of personal health information management activities
Sun, S., & Belkin, N. J. (2015) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2015.145052010046 · Cited by 5
A Reference-Dependent Model of Search Evaluation
Liu, J. (2019) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298970 · Cited by 2
Self-citation

Flagged first, then by citations

Sorted by citations

Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors
Liu, J. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102522 · Cited by 63
Self-citation
Identifying and Predicting the States of Complex Search Tasks
Liu, J., Sarkar, S., & Shah, C. S. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377976 · Cited by 50
Self-citation
Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation
Liu, J. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103007 · Cited by 23
Self-citation
Can search result summaries enhance the web search efficiency and experiences of the visually impaired users?
Aqle, A., Al-Thani, D., & Jaoua, A. (2022) · Universal Access in the Information Society · doi:10.1007/s10209-020-00777-w · Cited by 16
Understanding Voice Search Behavior: Review and Synthesis of Research
Xing, Z., Yuan, X., Wu, D., Huang, Y., & Mostafa, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60117-1_23 · Cited by 8
The landscape of data reuse in interactive information retrieval: Motivations, sources, and evaluation of reusability
Jiang, T., Li, W., & Liu, J. (2025) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.25020 · Cited by 6
Self-citation
Search under Uncertainty: Cognitive Biases and Heuristics: A Tutorial on Testing, Mitigating and Accounting for Cognitive Biases in Search Experiments
Liu, J., & Azzopardi, L. A. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3661382 · Cited by 5
Self-citation
How real is real enough? Participant feedback on a behavioral simulation used for information-seeking behavior research
Buhler, A., Brannon, B., Cataldo, T. T., Faniel, I. M., Connaway, L. S., Valenza, J., Elrod, R. E., & Cyr, C. (2022) · Journal of Librarianship and Information Science · doi:10.1177/09610006211067799 · Cited by 4
Characterizing and Early Predicting User Performance for Adaptive Search Path Recommendation
Wang, B., & Liu, J. (2023) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.799 · Cited by 4
Self-citation
Extracting the Implicit Search States from Explicit Behavioral Signals in Complex Search Tasks
Wang, B., & Liu, J. (2021) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.587 · Cited by 2
Self-citation
Patterns of Scholarly Communication in Global Information Retrieval Research: A Bibliometric Analysis (1954–2021)
Nazim, M., Rehman, S. U., Iqbal, A., & Ahmad, S. (2024) · Science & Technology Libraries · doi:10.1080/0194262x.2024.2429064 · Cited by 1
Towards Reproducibility of Interactive Retrieval Experiments: Framework and Case Study
Friese, J. I., & Fuhr, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88717-8_11 · Cited by 1
LISP – A Rich Interaction Dataset and Loggable Interactive Search Platform
Friese, J. I., Kruff, A. K., Schaer, P., Fuhr, N., & Ferro, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_54 · Cited by 1
Reflection on future directions: a systematic review of reported limitations and solutions in interactive information retrieval user studies
Jiang, T., & Liu, J. (2022) · Aslib Journal of Information Management · doi:10.1108/ajim-05-2022-0253 · Cited by 1
Self-citation
Evaluating Cognitive Biases in Conversational and Generative IIR: A Tutorial
Azzopardi, L. A., & Liu, J. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698437 · Cited by 1
Self-citation
Your plan may succeed, but what about failure? Investigating how people use ChatGPT for long-term life task planningPreprint
Wang, B., & Liu, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.11096 · Cited by 1
Self-citation
Introduction
Liu, J. (2023) · The information retrieval series · doi:10.1007/978-3-031-23229-9_1
Self-citation
Formally Modeling Users in Information Retrieval
Liu, J. (2023) · The information retrieval series · doi:10.1007/978-3-031-23229-9_2
Self-citation
Back to the Fundamentals: Extend the Rational Assumptions
Liu, J. (2023) · The information retrieval series · doi:10.1007/978-3-031-23229-9_5
Self-citation
Conclusion
Liu, J. (2023) · The information retrieval series · doi:10.1007/978-3-031-23229-9_8
Self-citation
From Rational Agent to Human with Bounded Rationality
Liu, J. (2023) · The information retrieval series · doi:10.1007/978-3-031-23229-9_3
Self-citation
5 − 4 ≠ 4 − 3: On the Uneven Gaps between Different Levels of Graded User Satisfaction in Interactive Information Retrieval Evaluation
Liu, J., & Han, F. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.509
Self-citation
Reproduction and Simulation of Interactive Retrieval Experiments
Friese, J. I. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56069-9_40
User Studies in Generative Interactive IR ( GenIIR ): An ISIC ‐Informed Systematic Review
Wang, A., He, D., Luo, Z., & Ma, F. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1515
ESCOMIC: User Adaptive Ex plainable Search for Comic Books
Shashidhar, S. B., Polley, S., Roy, S., & Nürnberger, A. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808367
Beyond States: Investigating the Effects of Context on User Modeling with Feature-Conditioned Markov Models
Friese, J. I., Kruff, A. K., Breuer, T., Schaer, P., & Fuhr, N. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840602

Authors: Liu, J., & Shah, C.
Affiliations: Rutgers, The State University of New Jersey (US)
ORCID: Liu, Jiqun
Shah, Chirag
Venue: Synthesis Lectures on Information Concepts, Retrieval, and Services
Publisher: Springer International Publishing
Published: 2019
Volume: 11
Issue: 2
Pages: i-75
ISSN: 1947-945X (ISSN-L)
1947-9468
DOI: 10.1007/978-3-031-02319-4
OpenAlex: W2948113848
Cited by: 26
References: 112
Citations by year: 2026: 3 · 2025: 4 · 2024: 4 · 2023: 7 · 2022: 3 · 2021: 2 · 2020: 3
FWCI: 1.74
Type: book
Language: en
Open access: closed
Licence: www.springer.com/tdm (tdm)
www.springer.com/tdm (vor)
Locations: Synthesis lectures on information concepts, retrieval, and services (publishedVersion)
CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
Funders: National Science Foundation (1423239, 1717488)
Topics: Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords: interactive information retrieval, user study evaluation, information retrieval evaluation, meta-evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Interactive IR User Study Design, Evaluation, and Reporting

Overview

Integrity signals

Verification

Claims the paper makes

Citations

Cites

Cited by

Related

Metadata

Record

About these checks