Reactive information foraging: an empirical investigation of theory-based recommender systems for programmers
Abstract
Information Foraging Theory (IFT) has established itself as an important theory to explain how people seek information, but most work has focused more on the theory itself than on how best to apply it. In this paper, we investigate how to apply a reactive variant of IFT (Reactive IFT) to design IFT-based tools, with a special focus on such tools for ill-structured problems. Toward this end, we designed and implemented a variety of recommender algorithms to empirically investigate how to help people with the ill-structured problem of finding where to look for information while debugging source code. We varied the algorithms based on scent type supported (words alone vs. words + code structure), and based on use of foraging momentum to estimate rapidity of foragers' goal changes. Our empirical results showed that (1) using both words and code structure significantly improved the ability of the algorithms to recommend where software developers should look for information; (2) participants used recommendations to discover new places in the code and also as shortcuts to navigate to known places; and (3) low-momentum recommendations were significantly more useful than high-momentum recommendations, suggesting rapid and numerous goal changes in this type of setting. Overall, our contributions include two new recommendation algorithms, empirical evidence about when and why participants found IFT-based recommendations useful, and implications for the design of tools based on Reactive IFT.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Baeza-Yates, R. A., & Ribeiro-Neto, B. A. (1999) · Cited by 11,498 · see citing works
Resnick, P., & Varian, H. R. (1997) · Communications of the ACM · doi:10.1145/245108.245121 · Cited by 3,772 · see citing works
Pirolli, P. L. T., & Card, S. K. (1999) · Psychological Review · doi:10.1037/0033-295x.106.4.643 · Cited by 1,354 · see citing works
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685 · see citing works
Hill, W. C., Hollan, J. D., Wroblewski, D., & McCandless, T. (1992) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/142750.142751 · Cited by 518 · see citing works
Pirolli, P. L. T. (2007) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195173321.003.0001 · Cited by 484 · see citing works
Using information scent to model user information needs and actions and the Web
Ed, H., Pirolli, P. L. T., Chen, K., & Pitkow, J. E. (2001) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365325 · Cited by 418 · see citing works
Pirolli, P. L. T. (2007) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195173321.001.0001 · Cited by 384 · see citing works
Hipikat: recommending pertinent software development artifacts
Čubranić, D., & Murphy, G. C. (2003) · doi:10.5555/776816.776866 · Cited by 317 · see citing works
Kersten, M. A., & Murphy, G. C. (2005) · International Conference on Aspect-Oriented Software Development (AOSD) · doi:10.1145/1052898.1052912 · Cited by 299 · see citing works
SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web
Fu, W., & Pirolli, P. L. T. (2007) · Human-Computer Interaction · doi:10.1080/07370020701638806 · Cited by 283 · see citing works
Questions programmers ask during software evolution tasks
Sillito, J., Murphy, G. C., & De Volder, K. (2006) · Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1181775.1181779 · Cited by 270 · see citing works
Olston, C., & Ed, H. (2003) · ACM Transactions on Computer-Human Interaction · doi:10.1145/937549.937550 · Cited by 249 · see citing works
Code bubbles: a working set-based interface for code understanding and maintenance
Bragdon, A., Zeleznik, R., Reiss, S. P., Karumuri, S., Cheung, W., Kaplan, J., Coleman, C., Adeputra, F., & LaViola, J. J. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753706 · Cited by 174 · see citing works
Computational models of information scent-following in a very large browsable text collection
Pirolli, P. L. T. (1997) · doi:10.1145/258549.258558 · Cited by 166 · see citing works
How Programmers Debug, Revisited: An Information Foraging Theory Perspective
Lawrance, J., Bogart, C., Burnett, M., Bellamy, R., Rector, K., & Fleming, S. D. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.111 · Cited by 142 · see citing works
Self-citationEd, H., Rosien, A., Supattanasiri, G., Williams, A., Royer, C., Chow, C., Robles, E., Dalal, B., Chen, J. Y., & Cousins, S. (2003) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642699 · Cited by 128 · see citing works
Towards understanding programs through wear-based filtering
DeLine, R., Khella, A., Czerwinski, M., & Robertson, G. (2005) · Proceedings of the 2005 ACM symposium on Software visualization · doi:10.1145/1056018.1056044 · Cited by 91 · see citing works
Building Usage Contexts During Program Comprehension
Parnin, C., & Görg, C. (2006) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2006.14 · Cited by 77 · see citing works
Using information scent to model the dynamic foraging behavior of programmers in maintenance tasks
Lawrance, J., Bellamy, R., Burnett, M., & Rector, K. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357261 · Cited by 76 · see citing works
Self-citationEnd-user debugging strategies: A sensemaking perspective
Grigoreanu, V., Burnett, M., Wiedenbeck, S., Cao, J., Rector, K., & Kwan, I. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2147783.2147788 · Cited by 60 · see citing works
Self-citationStorey, M., Best, C., Michaud, J., Rayside, D., Litoiu, M., & Musen, M. A. (2002) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/506443.506459 · Cited by 55 · see citing works
Reactive information foraging for evolving goals
Lawrance, J., Burnett, M., Bellamy, R., Bogart, C., & Swart, C. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753332 · Cited by 50 · see citing works
Self-citationModeling programmer navigation: A head-to-head empirical evaluation of predictive models
Piorkowski, D., Fleming, S. D., Scaffidi, C., John, L. K., Bogart, C., John, B. E., Burnett, M., & Bellamy, R. (2011) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2011.6070387 · Cited by 39 · see citing works
Self-citationManaging software change tasks: an exploratory study
Sillito, J., Volder, K. D., Fisher, B., & Murphy, G. C. (2005) · International Symposium on Empirical Software Engineering · doi:10.1109/isese.2005.1541811 · Cited by 39 · see citing works
Joshi, A. N. (2008) · doi:10.1109/iciinfs.2008.4798328 · Cited by 39 · see citing works
Relo: Helping Users Manage Context during Interactive Exploratory Visualization of Large Codebases
Sinha, V., Karger, D. R., & Miller, R. C. (2006) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2006.40 · Cited by 33 · see citing works
Information foraging : Why google makes people leave your site faster
Nielsen, J. (2003) · Medical Entomology and Zoology · Cited by 33 · see citing works
Evaluating a fisheye view of source code
Jakobsen, M. R., & Hornbæk, K. (2006) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1124772.1124830 · Cited by 28 · see citing works
Spool, J. M., Snyder, C., & Schroeder, W. (1995) · doi:10.1145/223355.223483 · Cited by 23 · see citing works
BEACONS IN PROGRAM COMPREHENSION
Wiedenbeck, S., & Evans, N. J. (1986) · ACM SIGCHI Bulletin · doi:10.1145/15683.1044090 · Cited by 8 · see citing works
Non-programmers identifying functionality in unfamiliar code: Strategies and barriers
Gross, P., & Kelleher, C. L. (2009) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2009.5295294 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
What recommendation systems for software engineering recommend: A systematic literature review
Gašparič, M., & Janes, A. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.11.036 · Cited by 78 · see citing works
The whats and hows of programmers' foraging diets
Piorkowski, D. J., Fleming, S. D., Kwan, I., Burnett, M. M., Scaffidi, C., Bellamy, R. K. E., & Jordahl, J. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466418 · Cited by 61 · see citing works
Self-citationForaging Among an Overabundance of Similar Variants
Ragavan, S. S., Kuttal, S. K., Hill, C., Sarma, A., Piorkowski, D., & Burnett, M. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858469 · Cited by 49 · see citing works
Self-citationDevelopers' code context models for change tasks
Fritz, T., Shepherd, D. C., Kevic, K., Snipes, W., & Bräunlich, C. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635905 · Cited by 45 · see citing works
Henley, A. Z., & Fleming, S. D. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557073 · Cited by 41 · see citing works
Self-citationTo fix or to learn? How production bias affects developers' information foraging during debugging
Piorkowski, D., Fleming, S. D., Scaffidi, C. P., Burnett, M., Kwan, I., Henley, A. Z., Macbeth, J., Hill, C., & Horvath, A. (2015) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2015.7332447 · Cited by 37 · see citing works
Self-citationForaging and navigations, fundamentally: developers' predictions of value and cost
Piorkowski, D., Henley, A. Z., Nabi, T., Fleming, S. D., Scaffidi, C., & Burnett, M. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950302 · Cited by 38 · see citing works
Self-citationEye gaze and interaction contexts for change tasks – Observations and potential
Kevic, K., Walters, B. M., Shaffer, T., Sharif, B., Shepherd, D. C., & Fritz, T. (2016) · Journal of Systems and Software · doi:10.1016/j.jss.2016.03.030 · Cited by 33 · see citing works
A diagnosis-based approach to software comprehension
Perez, A., & Abreu, R. (2014) · International Conference on Program Comprehension · doi:10.1145/2597008.2597151 · Cited by 26 · see citing works
Attention in Software Maintenance: An Eye Tracking Study
Ahrens, M., Schneider, K. A., & Busch, M. (2019) · doi:10.1109/emip.2019.00009 · Cited by 26 · see citing works
Supporting Code Comprehension via Annotations: Right Information at the Right Time and Place
Adeli, M., NELSON, N. M., Chattopadhyay, S., Coffey, H., Henley, A. Z., & Sarma, A. (2020) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc50065.2020.9127264 · Cited by 20 · see citing works
Supporting the Design of Machine Learning Workflows with a Recommendation System
Jannach, D., Jugovac, M., & Lerche, L. (2016) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2852082 · Cited by 20 · see citing works
Digital Nudges for Encouraging Developer Actions
Brown, C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00082 · Cited by 19 · see citing works
Shakil, A., Lutteroth, C., & Weber, G. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300306 · Cited by 18 · see citing works
The Impact of View Histories on Edit Recommendations
Lee, S., Kang, S., Kim, S., & Staats, M. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2362138 · Cited by 17 · see citing works
Ragavan, S. S., Pandya, B., Piorkowski, D., Hill, C., Kuttal, S. K., Sarma, A., & Burnett, M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025818 · Cited by 16 · see citing works
Self-citationTowards a Model of API Learning
Kelleher, C. L., & Ichinco, M. (2019) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2019.8818850 · Cited by 15 · see citing works
How end-user programmers forage in online repositories? An information foraging perspective
Kuttal, S. K., Kim, S. Y., Martos, C., & Bejarano, A. (2020) · Journal of Computer Languages · doi:10.1016/j.cola.2020.101010 · Cited by 13 · see citing works
Toward Principles for the Design of Navigation Affordances in Code Editors
Henley, A. Z., Fleming, S. D., & Luong, M. V. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025645 · Cited by 11 · see citing works
Self-citationHelping programmers navigate code faster with Patchworks: A simulation study
Henley, A. Z., Singh, A., Fleming, S. D., & Luong, M. V. (2014) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2014.6883026 · Cited by 11 · see citing works
Self-citationFraming program comprehension as fault localization
Perez, A., & Abreu, R. (2016) · Journal of Software Evolution and Process · doi:10.1002/smr.1799 · Cited by 10 · see citing works
Towards aiding within-patch information foraging by end-user programmers
Athreya, B., & Scaffidi, C. (2014) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2014.6883015 · Cited by 9 · see citing works
Self-citationPenney, S., Dodge, J. E., Anderson, A. W., Hilderbrand, C., Simpson, L., & Burnett, M. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3396047 · Cited by 8 · see citing works
Self-citationInformation Seeking Behavior for Bugs on GitHub: An Information Foraging Perspective
Sedhain, A., & Kuttal, S. K. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833144 · Cited by 8 · see citing works
How software users recommend tools to each other
Brown, C., Middleton, J., Sharma, E., & Murphy-Hill, E. R. (2017) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2017.8103460 · Cited by 8 · see citing works
Version Control Systems: An Information Foraging Perspective
Ragavan, S. S., Codoban, M., Piorkowski, D., Dig, D., & Burnett, M. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2931296 · Cited by 7 · see citing works
Self-citationAdaptive Recommendation-based Modeling Support for Data Analysis Workflows
Jannach, D., Jugovac, M., & Lerche, L. (2015) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2678025.2701380 · Cited by 7 · see citing works
Foraging goes mobile: Foraging while debugging on mobile devices
Piorkowski, D., Penney, S., Henley, A. Z., Pistoia, M., Burnett, M., Tripp, O., & Ferrara, P. (2017) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2017.8103444 · Cited by 5 · see citing works
Self-citationAn Empirical Evaluation of Models of Programmer Navigation
Singh, A., Henley, A. Z., Fleming, S. D., & Luong, M. V. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.84 · Cited by 5 · see citing works
Self-citationFollow-Up Attention: An Empirical Study of Developer and Neural Model Code Exploration
Paltenghi, M., Pandita, R., Henley, A. Z., & Ziegler, A. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3445338 · Cited by 5 · see citing works
CoMoGen: An Approach to Locate Relevant Task Context by Combining Search and Navigation
Kevic, K., Fritz, T., & Shepherd, D. C. (2014) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2014.28 · Cited by 4 · see citing works
Context-Based Search, Recommendation and Browsing in Software Development
Antunes, B., Furtado, B., & Gomes, P. (2014) · Springer eBooks · doi:10.1007/978-1-4939-1887-4_4 · Cited by 4 · see citing works
Investigating User Search Tactic Patterns and System Support in Using Digital Libraries.
Joo, S. (2013) · UWM Digital Commons (University of Wisconsin–Milwaukee) · Cited by 3 · see citing works
Iqbal, R., Grzywaczewski, A., & Chang, V. I. C. (2016) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2016.05.018 · Cited by 3 · see citing works
CodeDeviant: Helping Programmers Detect Edits That Accidentally Alter Program Behavior
Henley, A. Z., & Fleming, S. D. (2018) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2018.8506567 · Cited by 3 · see citing works
Self-citationTowards an Empirically-Based IDE: An Analysis of Code Size and Screen Space
Short, A. C., & Henley, A. Z. (2019) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2019.8818857 · Cited by 3 · see citing works
Predicting code context models for software development tasks
Wan, Z., Murphy, G. C., & Xia, X. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416544 · Cited by 3 · see citing works
Estimating Foraging Values and Costs in Stack Overflow
Sedhain, A., Ragavan, S. S., McKinney, B. A., & Kuttal, S. K. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833135 · Cited by 3 · see citing works
A sensemaking analysis of API learning using React
Kelleher, C. L., & Brachman, M. (2022) · Journal of Computer Languages · doi:10.1016/j.cola.2022.101189 · Cited by 2 · see citing works
Predicting Information Foraging on Q&A Websites
Sedhain, A., Ragavan, S. S., McKinney, B. A., Leon, S., & Kuttal, S. K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93536-7_23 · Cited by 2 · see citing works
Using Eye Gaze Data to Recognize Task-Relevant Source Code Better and More Fine-Grained
Kevic, K. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.152 · Cited by 2 · see citing works
Modeling Foraging Behavior in GitHub
Sedhain, A., Wang, Y., McKinney, B. A., & Kuttal, S. K. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-35998-9_21 · Cited by 1 · see citing works
A collaborative approach to web information foraging based on multi-agent systems
Drias, Y., & Pasi, G. (2017) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3106426.3106533 · Cited by 1 · see citing works
A Collaborative Multi-Agent Approach to Web Information Foraging.
Drias, Y., & Pasi, G. (2017) · IIR eBooks · Cited by 1 · see citing works
An Approach to Context Modeling in Software Development
Antunes, B., Cordeiro, J., & Gomes, P. (2013) · Communications in computer and information science · doi:10.1007/978-3-642-45404-2_13 · Cited by 1 · see citing works
Upreti, D., Maheshwari, A., Tabb, T., Polykretis, I., Gallo, E. M., Stewart, K. M., LaToza, T. D., & Danielescu, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713249
Where Will They Click Next? A Social Foraging Model for Collaborating Teams
Leon, S., & Kuttal, S. K. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791506
A Needle in a Haystack: Developers’ Information Foraging on the Wild Web
Kuttal, S. K., Sthapit, S., Phillips, R., & Rahal, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29175-2_17
Effectiveness of Spatial Overviews for Navigating Software Projects
Morrice, W., Gutwin, C. A., Stavness, I., & Cockburn, A. (2026) · Graphics Interface · doi:10.1145/3831423.3831459
Hau, K., Cheng, J., Guo, J. Q., Lyons, K. E., & Zhou, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2744517
- Authors
- Piorkowski, D., Fleming, S., Scaffidi, C., Bogart, C., Burnett, M., John, B., Bellamy, R., & Swart, C.
- Affiliations
- Oregon State University (US)
- IBM (United States) (US)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 1471-1480
- OpenAlex
- W2116109573
- Cited by
- 50
- References
- 32
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 1 · 2023: 1 · 2022: 3 · 2020: 4 · 2019: 6 · 2018: 1 · 2017: 7 · 2016: 7 · 2015: 3 · 2014: 8 · 2013: 3
- FWCI
- 3.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Air Force Office of Scientific Research
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- information foraging theory, recommender systems, debugging, source code navigation, ill-structured problems, scent, code structure, goal change, software developers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
