Information‐seeking behaviors of computer scientists: Challenges for electronic literature search tools
Abstract
Since the recent emergence of electronic literature resources, researchers have begun to adopt new information‐seeking practices. The purpose of this research is to investigate the information needs and searching behaviors of researchers, and their implications for electronic literature search tools. We conducted mixed‐method case studies involving interviews, diary logs, and observations of computer scientists followed by a web‐based survey to validate our findings. The results show that computer science researchers have the following main purposes for seeking information: keeping up to date, exploring new topics, reviewing literature, collaborating, preparing lectures, and recommending material for students. We found that keeping up to date with research is the most frequent purpose and exploring unfamiliar research areas is the most difficult. Furthermore, we found that literature searching is a collaborative process and, depending on the search purpose, different information sources and navigation strategies are used. On the basis of these findings we discuss six design challenges for literature search tools, which are: providing support for keeping up to date with research, exploring unfamiliar topics, browsing user history, collaborating and sharing, performing a federated search that goes beyond scholarly research, and sorting and navigating the results.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Google Scholar emerged as the number one go‐to academic search engine for most academic users due to its great ease of use and performance in informational and exploratory searches.
“In fact, it was due to its great ease of use and performance in informational and exploratory searches78 that Google Scholar emerged as the number one go‐to academic search engine for most academic users.79, 80, 81, 82”
Cites
Greenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045 · cited 2× · see citing works
“Challenge 6: Navigation and Ranking Support Citation chaining (snowballing) and browsing through reputable forums are known to be the most common SLR techniques used by researchers (Greenhalgh & Peacock, 2005).”
MethodExploratory search: from finding to understanding
Marchionini, G. (2006) · Communications of the ACM · doi:10.1145/1121949.1121979 · Cited by 1,506 · see citing works
Integrating case study and survey research methods: an example in information systems
Gable, G. G. (1994) · European Journal of Information Systems · doi:10.1057/ejis.1994.12 · Cited by 899 · see citing works
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659 · see citing works
The Google generation: the information behaviour of the researcher of the future
Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B., Withey, R., Jamali, H. R., Dobrowolski, T., & Tenopir, C. (2008) · Aslib Proceedings · doi:10.1108/00012530810887953 · Cited by 617 · see citing works
Determining the informational, navigational, and transactional intent of Web queries
Jansen, B. J. J., Booth, D. L., & Spink, A. H. (2007) · Information Processing & Management · doi:10.1016/j.ipm.2007.07.015 · Cited by 470 · see citing works
Use and Users of Electronic Library Resources: An Overview and Analysis of Recent Research Studies
Tenopir, C. (2003) · Zenodo · doi:10.5281/zenodo.7779277 · Cited by 361 · see citing works
Web search personalization with ontological user profiles
Sieg, A., Mobasher, B., & Burke, R. (2007) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1321440.1321515 · Cited by 322 · see citing works
White, R. W., Kules, B., Drucker, S. M., & schraefel, M. C. (2006) · Communications of the ACM · doi:10.1145/1121949.1121978 · Cited by 194 · see citing works
White, R. W., Kules, B., Drucker, S. M., & schraefel, M. C. (2006) · Communications of the ACM · Cited by 192 · see citing works
Information seeking behavior of academic scientists
Hemminger, B. M., Lu, D., Vaughan, K. T. L., & Adams, S. J. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20686 · Cited by 169 · see citing works
Determining the informational, navigational and transactional intent of web queries
Booth, D. L., Jansen, B. J. J., & Spink, A. H. (2008) · Office of Education Research; Faculty of Education · Cited by 166 · see citing works
National study of information seeking behavior of academic researchers in the United States
Niu, X., Hemminger, B. M., Lown, C., Adams, S. J., Brown, C. M., Level, A. V., McLure, M., Powers, A., Tennant, M. R., & Cataldo, T. T. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21307 · Cited by 139 · see citing works
Cognitive strategies in web searching.
Navarro-Prieto, N., Scaife, M., & Rogers, Y. (1999) · UCL Discovery (University College London) · Cited by 135 · see citing works
Nowell, L., France, R. K., Hix, D., Heath, L. S., & Fox, E. A. (1996) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/243199.243214 · Cited by 135 · see citing works
A study of factors that affect the information‐seeking behavior of academic scientists
Niu, X., & Hemminger, B. M. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21669 · Cited by 129 · see citing works
Patterns of Journal Use by Faculty at Three Diverse Universities
King, D. W., Tenopir, C., Montgomery, C. H., & Aerni, S. E. (2003) · D-Lib Magazine · doi:10.1045/october2003-king · Cited by 86 · see citing works
A study of graduate student end‐users’ use and perception of electronic journals
Liew, C. L., Foo, S. S.-B., & Chennupati, K. R. (2000) · Online Information Review · doi:10.1108/14684520010350650 · Cited by 84 · see citing works
Survey and evaluation of query intent detection methods
Brenes, D. J., Gayo-Avello, D., & Pérez-González, K. (2009) · doi:10.1145/1507509.1507510 · Cited by 56 · see citing works
Hightower, C., & Caldwell, C. (2010) · Issues in Science and Technology Librarianship · doi:10.29173/istl2545 · Cited by 35 · see citing works
McDonough, J. P. (1999) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1999)50:10 · Cited by 35 · see citing works
Diversity in the Information Seeking Behaviour of the Virtual Scholar: Institutional Comparisons
Nicholas, D., Huntington, P. A., & Jamali, H. R. (2007) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2007.09.001 · Cited by 32 · see citing works
Impact of research cultures on the use of digital library resources
Talja, S., Vakkari, P., Fry, J., & Wouters, P. F. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20650 · Cited by 31 · see citing works
Vakkari, P., & Talja, S. (2006) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 24 · see citing works
Trends in the use of digital libraries by scientists in 2000‐2005: A case study of finelib
Vakkari, P. (2006) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.1450430132 · Cited by 21 · see citing works
Information seeking behavior of scientists in the electronic information age
BrownCecelia, M. (1999) · Journal of the American Society for Information Science and Technology · doi:10.5555/318976.318979 · Cited by 13 · see citing works
Dervin, B. L., & Reinhard, C. D. (2007) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504403114 · Cited by 6 · see citing works
Flagged first, then by citations
4 of the 31 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Gusenbauer, M., & Haddaway, N. R. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1378 · Cited by 2,379 · see citing works
Athukorala, K., Głowacka, D., Jacucci, G., Oulasvirta, A., & Vreeken, J. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23617 · Cited by 150 · see citing works
Self-citationOsman, M. A., Schick-Makaroff, K., Thompson, S., Bialy, L., Featherstone, R., Kurzawa, J., Zaidi, D., Okpechi, I., Habib, S., Shojai, S., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001629 · Cited by 108 · see citing works
“Finally, through backward citation chaining 29 we reviewed the reference lists of included studies for relevant studies not identified from our initial search until saturation was achieved.”
Gusenbauer, M., & Haddaway, N. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1457 · Cited by 88 · see citing works
Raamkumar, A. S., Foo, S. S.-B., & Pang, N. L.-S. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2016.12.006 · Cited by 58 · see citing works
Zhong, Z., Hu, D., Zheng, F., Ding, S., & Luo, A. (2018) · Nurse Education Today · doi:10.1016/j.nedt.2018.01.004 · Cited by 58 · see citing works
Athukorala, K., Medlar, A. J., Oulasvirta, A., Jacucci, G., & Głowacka, D. (2016) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2856767.2856786 · Cited by 42 · see citing works
Self-citationData-Seeking Behaviour in the Social Sciences
Krämer, T., Papenmeier, A., Carevic, Z., Kern, D., & Mathiak, B. (2021) · International Journal on Digital Libraries · doi:10.1007/s00799-021-00303-0 · Cited by 38 · see citing works
Supporting exploratory search tasks with interactive user modeling
Ruotsalo, T., Athukorala, K., Głowacka, D., Konyushkova, K., Oulasvirta, A., Kaipiainen, S., Samuel Kaski, A., & Jacucci, G. (2013) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14505001040 · Cited by 35 · see citing works
Self-citationSurvival analysis of author keywords: An application to the library and information sciences area
Peset, F., Garzón-Farinós, F., González, L.-M., García‐Massó, X., Ferrer-Sapena, A., Toca-Herrera, J. L., & Sánchez-Pérez, E. A. (2019) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24248 · Cited by 35 · see citing works
Information Seeking Behaviors, Attitudes, and Choices of Academic Chemists
Gordon, I. D., Meindl, P. E., WHITE, M. J., & Szigeti, K. (2018) · Science & Technology Libraries · doi:10.1080/0194262x.2018.1445063 · Cited by 29 · see citing works
PaperPoles: Facilitating adaptive visual exploration of scientific publications by citation links
He, J., Ping, Q., Lou, W., & Chen, C. (2019) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24171 · Cited by 26 · see citing works
Athukorala, K., Oulasvirta, A., Głowacka, D., Vreeken, J., & Jacucci, G. (2014) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2661829.2661904 · Cited by 21 · see citing works
Self-citationSchick‐Makaroff, K., Thummapol, O., Thompson, S., Flynn, R., Karimi-Dehkordi, M., Klarenbach, S. W., Sawatzky, R. G., & Greenhalgh, J. (2019) · Systematic Reviews · doi:10.1186/s13643-018-0911-6 · Cited by 20 · see citing works
Understanding Creative Thinking Processes in Searching for New Ideas
Chavula, C., Choi, Y., & Rieh, S. Y. (2022) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505783 · Cited by 20 · see citing works
Rousi, A. M. (2022) · Journal of Librarianship and Information Science · doi:10.1177/09610006221093049 · Cited by 16 · see citing works
Editable User Profiles for Controllable Text Recommendations
Mysore, S., Jasim, M., McCallum, A., & Zamani, H. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591677 · Cited by 16 · see citing works
Investigating exploratory search activities based on the stratagem level in digital libraries
Carevic, Z., Lusky, M., van Hoek, W., & Mayr, P. (2017) · International Journal on Digital Libraries · doi:10.1007/s00799-017-0226-6 · Cited by 16 · see citing works
How Data Scientists Review the Scholarly Literature
Mysore, S., Jasim, M., Song, H., Akbar, S., Randall, A. K. C., & Mahyar, N. (2023) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3576840.3578309 · Cited by 16 · see citing works
Anatomy of scholarly information behavior patterns in the wake of academic social media platforms
Alhoori, H., Samaka, M., Furuta, R. K., & Fox, E. A. (2018) · International Journal on Digital Libraries · doi:10.1007/s00799-018-0255-9 · Cited by 15 · see citing works
Exploratory Search for Scientific Articles
Nedumov, Y., & Kuznetsov, S. D. (2019) · Programming and Computer Software · doi:10.1134/s0361768819070089 · Cited by 13 · see citing works
Information seeking within academic digital libraries
Hoeber, O., & Storie, D. (2022) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3529372.3533286 · Cited by 10 · see citing works
McCay‐Peet, L., Quan‐Haase, A., & Kern, D. (2015) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2015.145052010070 · Cited by 10 · see citing works
Klopfenstein, D. V., & Dampier, W. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1456 · Cited by 10 · see citing works
SciNoon: Exploratory Search System for Scientific Groups.
Nedumov, Y., Babichev, A., Mashonsky, I., & Semina, N. (2019) · IUI Workshops · Cited by 9 · see citing works
Zhang, X., Zhu, S., Zhao, Y. C., Jia, M., & Zhu, Q. (2024) · Information Research · doi:10.47989/ir292856 · Cited by 8 · see citing works
Visual Exploration of Literature with Argo Scholar
Li, K., Yang, H., Montoya, E., Upadhayay, A., Zhou, Z., Saad-Falcon, J., & Chau, D. H. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557177 · Cited by 7 · see citing works
Embedding-based Scientific Literature Discovery in a Text Editor Application
Gökçe, O., Prada, J., Nikolov, N. I., Gu, N., & Hahnloser, R. H. R. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.36 · Cited by 7 · see citing works
Graduate student search strategies within academic digital libraries
Hoeber, O., & Storie, D. (2023) · International Journal on Digital Libraries · doi:10.1007/s00799-023-00378-x · Cited by 7 · see citing works
Knight, I. A., Wilson, M. L., Brailsford, D. F., & Milić-Frayling, N. (2019) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298937 · Cited by 7 · see citing works
Visualizing activity traces to support collaborative literature searching
Athukorala, K., Micallef, L., An, C., Reijonen, A., Peltonen, J., Ruotsalo, T., & Jacucci, G. (2017) · International Symposium on Visual Information Communication and Interaction (VINCI) · doi:10.1145/3105971.3105981 · Cited by 6 · see citing works
Self-citationHuang, C., Casey, A., Głowacka, D., & Medlar, A. J. (2019) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298953 · Cited by 7 · see citing works
Uncharted Territory: Understanding Exploratory Search Behaviours in Literature Reviews
Soufan, A., Ruthven, I. G., & Azzopardi, L. A. (2024) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3627508.3638334 · Cited by 5 · see citing works
How Semantic Processing of Words Evokes Changes in Pupil
Pluchino, P., Gamberini, L., Barral, O., & Minelle, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13500-7_8 · Cited by 5 · see citing works
Hagiwara, Y., Ishita, E., Watanabe, Y., & Tomiura, Y. (2022) · College & Research Libraries · doi:10.5860/crl.83.4.610 · Cited by 5 · see citing works
Navigating Complex Information Spaces: A Portfolio Theory Approach
Bandyopadhyay, P., Ruotsalo, T., Ukkonen, A., & Jacucci, G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13500-7_13 · Cited by 3 · see citing works
Self-citationExploratory search for scientific articles
Nedumov, Y., & Kuznetsov, S. D. (2018) · Proceedings of the Institute for System Programming of RAS · doi:10.15514/ispras-2018-30(6)-10 · Cited by 3 · see citing works
Supporting exploratory search through user modeling
Athukorala, K., Oulasvirta, A., Głowacka, D., Vreeken, J., & Jacucci, G. (2014) · Cited by 2 · see citing works
Self-citationSupporting Exploratory Search Through Interaction Modeling
Athukorala, K. (2014) · doi:10.1145/2663714.2668049 · Cited by 2 · see citing works
Self-citationTransitioning from benchmarks to a real-world case of information-seeking in Scientific Publications
Tahri, C., Bochnakian, A., Haouat, P., & Tannier, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.68 · Cited by 2 · see citing works
Instruction In Information Literacy And Its Function In Fostering Critical Thinking Abilities
Adhikari, H., & Joshi, Y. (2024) · doi:10.53555/kuey.v30i5.3305 · Cited by 2 · see citing works
Integrating measures of replicability into scholarly search: Challenges and opportunities
Wu, C., Chakravorti, T., Carroll, J. M., & Rajtmajer, S. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3643043 · Cited by 3 · see citing works
A task-based scientific paper recommender system for literature review and manuscript preparation
Raamkumar, A. S. (2018) · doi:10.32657/10220/46243 · Cited by 2 · see citing works
Resourceful interaction in information discovery
Jacucci, G. (2016) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew.2016.7495639 · Cited by 1 · see citing works
Self-citationBuilding User Groups Based on a Structural Representation of User Search Sessions
van Hoek, W., & Carevic, Z. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67008-9_36 · Cited by 1 · see citing works
Enhancing Exploratory Information-Seeking through Interaction Modeling
Athukorala, K. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08786-3_43 · Cited by 1 · see citing works
Self-citationINFORMATION SEEKING AND VERIFICATION OF WEB INFORMATION: A COMPARATIVE STUDY
(2022) · Issues in Information Systems · doi:10.48009/3_iis_2022_108 · Cited by 2 · see citing works
Anatomy of Scholarly Information Behavior Patterns in the Wake of Social Media.Preprint
Alhoori, H., Furuta, R. K., Samaka, M., & Fox, E. A. (2016) · arXiv (Cornell University)
Osman, M. A. (2018) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3h708g6d
Casarín, H. D. C. S., & Almeida, C. C. (2021) · Dialnet (Universidad de la Rioja) · doi:10.19132/1808-5245281.258-280
Hermina, U. N., & Liliyana, L. (2022) · International journal of business economics & management · doi:10.21744/ijbem.v5n4.2020
Contextualizing the Role of Web Search In Creative Workflows: Insights from a Longitudinal Study
Palani, S., & Dow, S. P. (2025) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3698204.3716457
Hrčková, A., Renoux, J., Calasanz, R. T., Chudá, D., Tamajka, M., & Šimko, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11108-1_34
AI Agents as Knowledge Explorers: A Case Study of 398 U.S. Case Law Quotations
Milbauer, J. (2025) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl67857.2025.00053
Workshop on Generative AI and Academic Search (GAI&AS)
Liu, Y., Arguello, J., Hoeber, O., Liu, C., Rieh, S. Y., & Sinnamon, L. S. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787953 · Cited by 1 · see citing works
- Authors
- Athukorala, K., Hoggan, E., Lehtiö, A., Ruotsalo, T., & Jacucci, G.
- Affiliations
- University of Helsinki (FI)
- Helsinki Institute for Information Technology (FI)
- Corresponding author
- Athukorala, Kumaripaba
- Hoggan, Eve
- Lehtiö, Anu
- Ruotsalo, Tuukka
- Jacucci, Giulio
- Venue
- Proceedings of the American Society for Information Science and Technology
- Publisher
- Wiley
- Published
- 2013-01
- Volume
- 50
- Issue
- 1
- Pages
- 1-11
- ISSN
- 0044-7870 (ISSN-L)
- 1550-8390
- OpenAlex
- W1948935217
- Cited by
- 60
- References
- 31
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 4 · 2023: 4 · 2022: 8 · 2021: 3 · 2020: 3 · 2019: 9 · 2018: 8 · 2017: 4 · 2016: 3 · 2015: 3 · 2014: 6 · 2013: 1
- FWCI
- 12.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the American Society for Information Science and Technology (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Physical Sciences › Computer Science › Information Systems › Library Science and Information
- Keywords
- information seeking, computer scientists, literature searching, keeping up to date, exploratory research, collaborative information seeking, federated search, information needs, design challenges
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
