Enabling Data-Driven API Design with Community Usage Data: A Need-Finding Study
The DOI matches its Crossref record; 57 of the 59 references it cites were found in a registry or an index.
Abstract
APIs are becoming the fundamental building block of modern software and their usability is crucial to programming efficiency and software quality. Yet API designers find it hard to gather and interpret user feedback on their APIs. To close the gap, we interviewed 23 API designers from 6 companies and 11 open-source projects to understand their practices and needs. The primary way of gathering user feedback is through bug reports and peer reviews, as formal usability testing is prohibitively expensive to conduct in practice. Participants expressed a strong desire to gather real-world use cases and understand users' mental models, but there was a lack of tool support for such needs. In particular, participants were curious about where users got stuck, their workarounds, common mistakes, and unanticipated corner cases. We highlight several opportunities to address those unmet needs, including developing new mechanisms that systematically elicit users' mental models, building mining frameworks that identify recurring patterns beyond shallow statistics about API usage, and exploring alternative design choices made in similar libraries.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The SAGE handbook of qualitative research
(2005) · Choice Reviews Online · doi:10.5860/choice.43-1330 · Cited by 10,183
The Sage Handbook of Qualitative Research
Ho, R. T. H., Ng, S. M., & Ho, D. Y. F. (2007) · Asian Journal of Social Psychology · doi:10.1111/j.1467-839x.2007.00237.x · Cited by 2,167
Usability Analysis of Visual Programming Environments: A ‘Cognitive Dimensions’ Framework
Green, T. R. G., & Petre, M. (1996) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1996.0009 · Cited by 1,251
Social coding in GitHub: transparency and collaboration in an open software repository
Dabbish, L., Stuart, C., Tsay, J., & Herbsleb, J. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145396 · Cited by 999 · cited 1×
“, which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundSix Learning Barriers in End-User Programming Systems
Ko, A. J., Myers, B. A., & Aung, H. H. (2004) · 2004 IEEE Symposium on Visual Languages - Human Centric Computing · doi:10.1109/vlhcc.2004.47 · Cited by 549 · cited 2×
“Previous studies have shown that programmers at all levels, from novices to experts, often find it difficult to learn new APIs [27, 40, 47, 15, 49].”
BackgroundWhy eve and mallory love android: an analysis of android SSL (in)security
Fahl, S., Harbach, M., Muders, T., Baumgärtner, L., Freisleben, B., & Smith, M. (2012) · Proceedings of the 2012 ACM conference on Computer and communications security · doi:10.1145/2382196.2382205 · Cited by 490 · cited 1×
“Making mistakes in API usage could lead to severe program failures such as runtime exceptions [60] and security vulnerabilities [21, 18, 16].”
BackgroundGu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950334 · Cited by 485 · cited 1×
“Large-scale community data has driven big success stories in domains such as bug detection [24, 60] and code completion [25, 45, 3], but remains under-utilized in the context of human-centered API design and usability evaluation.”
BackgroundAn empirical study of cryptographic misuse in android applications
Egele, M., Brumley, D., Fratantonio, Y., & Kruegel, C. (2013) · Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security - CCS '13 · doi:10.1145/2508859.2516693 · Cited by 435 · cited 1×
“Making mistakes in API usage could lead to severe program failures such as runtime exceptions [60] and security vulnerabilities [21, 18, 16].”
BackgroundThe most dangerous code in the world: validating SSL certificates in non-browser software
Georgiev, M., Iyengar, S., Jana, S., Anubhai, R., Boneh, D., & Shmatikov, V. (2012) · Proceedings of the 2012 ACM conference on Computer and communications security · doi:10.1145/2382196.2382204 · Cited by 413 · cited 1×
“Making mistakes in API usage could lead to severe program failures such as runtime exceptions [60] and security vulnerabilities [21, 18, 16].”
BackgroundMAPO: Mining and Recommending API Usage Patterns
Zhong, H., Xie, T., Zhang, L., Pei, J., & Mei, H. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-03013-0_15 · Cited by 410 · cited 4×
“There is a lot of literature on mining common patterns from code repositories [62, 35, 59, 41, 60].”
BackgroundMethodInfluentialWhat Makes APIs Hard to Learn? Answers from Developers
Robillard, M. P. (2009) · IEEE Software · doi:10.1109/ms.2009.193 · Cited by 382 · cited 4×
“Previous studies have shown that programmers at all levels, from novices to experts, often find it difficult to learn new APIs [27, 40, 47, 15, 49].”
BackgroundResultInfluentialA field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352 · cited 1×
“Given the increasing number and complexity of APIs, learning and using APIs is becoming a common activity and a key challenge in modern programming [27, 47, 48, 40].”
BackgroundGraph-based mining of multiple object usage patterns
Nguyen, T. T., Nguyen, H. A., Pham, N. H., Al-Kofahi, J. M., & Nguyen, T. N. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595767 · Cited by 292 · cited 3×
“For example, while existing API usage mining techniques [62, 59, 41, 22] mainly focused on identifying common patterns of correct usage, designers wished to gather more unanticipated corner cases and user mistakes from a broader scope of community data such as online discussions, emails, and issue…”
BackgroundMethodMyers, B. A., & Stylos, J. (2016) · Communications of the ACM · doi:10.1145/2896587 · Cited by 269 · cited 4×
“Previous studies have shown that programmers at all levels, from novices to experts, often find it difficult to learn new APIs [27, 40, 47, 15, 49].”
BackgroundMethodInfluentialAugmenting API documentation with insights from stack overflow
Treude, C., & Robillard, M. P. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884800 · Cited by 239 · cited 1×
“There has been a lot of interest in mining programming insights from online coding communities, e.g., API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundHow developers search for code: a case study
Sadowski, C., Stolee, K. T., & Elbaum, S. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786855 · Cited by 238 · cited 1×
“Previous studies have shown that programmers at all levels, from novices to experts, often find it difficult to learn new APIs [27, 40, 47, 15, 49].”
BackgroundTell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225 · cited 1×
“Existing approaches either elicit users’ mental models directly via interviews [57, 26] or indirectly through questionnaires [28, 50], which is deemed to be expensive and unscalable given the large number of APIs and use cases.”
BackgroundImpression formation in online peer production: activity traces and personal profiles in github
Marlow, J., Dabbish, L., & Herbsleb, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441792 · Cited by 223 · cited 1×
“, which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundHow to break an API: cost negotiation and community values in three software ecosystems
Bogart, C., Kästner, C., Herbsleb, J., & Thung, F. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950325 · Cited by 207 · cited 2×
“[4] studied how community-specific values, policies, and tools may influence how developers respond to API changes that may break client code.”
BackgroundCode completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · ACM SIGPLAN Notices · doi:10.1145/2666356.2594321 · Cited by 189
Are code examples on an online Q&A forum reliable?: a study of API misuse on stack overflow
Zhang, T., Upadhyaya, G., Reinhardt, A., Rajan, H., & Kim, M. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180260 · Cited by 167 · cited 5×
“Making mistakes in API usage could lead to severe program failures such as runtime exceptions [60] and security vulnerabilities [21, 18, 16].”
BackgroundMethodInfluentialSelf-citationThe Factory Pattern in API Design: A Usability Evaluation
Ellis, B., Stylos, J., & Myers, B. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.85 · Cited by 148 · cited 1×
“, the use of factory pattern [17], the use of default class constructors [51], and method placement [54].”
MethodValiev, M., Vasilescu, B., & Herbsleb, J. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236062 · Cited by 144 · cited 2×
“[58] analyzed how project-level factors (e.”
BackgroundHow it works: a field study of non-technical users interacting with an intelligent system
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240630 · Cited by 137 · cited 1×
“Existing approaches either elicit users’ mental models directly via interviews [57, 26] or indirectly through questionnaires [28, 50], which is deemed to be expensive and unscalable given the large number of APIs and use cases.”
BackgroundCreating a shared understanding of testing culture on a social coding site
Pham, R., Singer, L., Liskin, O., Filho, F. F., & Schneider, K. A. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606557 · Cited by 135 · cited 1×
“, which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundToward harnessing user feedback for machine learning
Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T., Sullivan, E., Drummond, R., & Herlocker, J. (2007) · Proceedings of the 12th international conference on Intelligent user interfaces · doi:10.1145/1216295.1216316 · Cited by 132 · cited 1×
“Existing approaches either elicit users’ mental models directly via interviews [57, 26] or indirectly through questionnaires [28, 50], which is deemed to be expensive and unscalable given the large number of APIs and use cases.”
BackgroundDeveloper onboarding in GitHub: the role of prior social links and language experience
Casalnuovo, C., Vasilescu, B., Devanbu, P., & Filkov, V. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786854 · Cited by 126 · cited 1×
“, which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundDagenais, B., & Robillard, M. P. (2010) · Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1882291.1882312 · Cited by 124 · cited 1×
“Writing API documentation and keeping it up-to-date is challenging [12, 13].”
BackgroundRecovering traceability links between an API and its learning resources
Dagenais, B., & Robillard, M. P. (2012) · doi:10.5555/2337223.2337230 · Cited by 120
Going Farther Together: The Impact of Social Capital on Sustained Participation in Open Source
Qiu, H. S., Nolte, A., Brown, A., Serebrenik, A., & Vasilescu, B. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00078 · Cited by 116 · cited 1×
“…studies about how information signals from software communities such as GitHub can influence the decision making process in software development, e.g., which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundUsability Implications of Requiring Parameters in Objects' Constructors
Stylos, J., & Clarke, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.92 · Cited by 110 · cited 1×
“, the use of factory pattern [17], the use of default class constructors [51], and method placement [54].”
MethodHow Well Do Search Engines Support Code Retrieval on the Web?
Sim, S. E., Umarji, M., Ratanotayanon, S., & Lopes, C. V. (2011) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2063239.2063243 · Cited by 103
The implications of method placement on API learnability
Stylos, J., & Myers, B. A. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453117 · Cited by 102 · cited 1×
“, the use of factory pattern [17], the use of default class constructors [51], and method placement [54].”
Method(2018) · doi:10.1145/3180155.3180209 · Cited by 98 · cited 1×
“, which project to depend on [56], which developer to follow [11, 33], how to test a project [42], which project to contribute to [6, 44, 43].”
BackgroundImproving API Caveats Accessibility by Mining API Caveats Knowledge Graph
Li, H., Li, S., Sun, J., Xing, Z., Peng, X., Liu, M., & Zhao, X. (2018) · 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2018.00028 · Cited by 97 · cited 1×
“, API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundGruska, N., Wasylkowski, A., & Zeller, A. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831723 · Cited by 95 · cited 1×
“Large-scale community data has driven big success stories in domains such as bug detection [24, 60] and code completion [25, 45, 3], but remains under-utilized in the context of human-centered API design and usability evaluation.”
BackgroundFixing Recurring Crash Bugs via Analyzing Q&A Sites (T)
Gao, Q., Zhang, H., Wang, J., Xiong, Y., Zhang, L., & Mei, H. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.81 · Cited by 86 · cited 1×
“, API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundChen, C., Xing, Z., Liu, Y., & Xiong, K. O. L. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2896123 · Cited by 82
The Signals that Potential Contributors Look for When Choosing Open-source Projects
Qiu, H. S., Li, Y. L., Padala, S., Sarma, A., & Vasilescu, B. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359224 · Cited by 75
Impact of interruption style on end-user debugging
Robertson, T. J., Prabhakararao, S., Burnett, M., Cook, C., Ruthruff, J. R., Beckwith, L., & Phalgune, A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985729 · Cited by 71 · cited 1×
“Such instrumentation methods can also be further augmented with prompting questions to directly solicit explanations on API usage, though that would require further investigation of interruption styles [46] to efficiently solicit explanations without causing harm to users’ productivity.”
BackgroundExPort: Detecting and visualizing API usages in large source code repositories
Moritz, E., Linares‐Vásquez, M., Poshyvanyk, D., Grechanik, M., McMillan, C., & Gethers, M. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693127 · Cited by 68 · cited 2×
“There is a lot of literature on mining common patterns from code repositories [62, 35, 59, 41, 60].”
BackgroundMethodSearch-Based Test Input Generation for String Data Types Using the Results of Web Queries
McMinn, P., Shahbaz, M., & Stevenson, M. (2012) · 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2012.94 · Cited by 59 · cited 1×
“, API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundAnalyzing and Supporting Adaptation of Online Code Examples
Zhang, T., Yang, D., Lopes, C. V., & Kim, M. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00046 · Cited by 51 · cited 1×
“, API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundSelf-citationControlling Transparency in an Online Learning Environment
Robison, T., & Tanimoto, S. (2007) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2007) · doi:10.1109/vlhcc.2007.36 · Cited by 49 · cited 1×
“Stylos and Myers proposed a list of API design decisions in object-oriented programming languages such as Java and C#, based on API design principles and recommendations from textbooks, guidelines, lab studies, and online discussions [53].”
BackgroundChen, F., & Kim, S. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786819 · Cited by 48 · cited 1×
“There has been a lot of interest in mining programming insights from online coding communities, e.g., API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundVisualizing API Usage Examples at Scale
Glassman, E. L., Zhang, T., Hartmann, B., & Kim, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174154 · Cited by 48 · cited 3×
“EXAMPLORE [22] consolidates unique API usage features extracted from a large collection of code examples into a single API skeleton and allows developers to quickly filter the code corpus by selecting desired features in the skeleton.”
BackgroundMethodSelf-citationA case study of API redesign for improved usability
Stylos, J., Graf, B., Busse, D. K., Ziegler, C., Ehret, R., & Karstens, J. (2008) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2008.4639083 · Cited by 47 · cited 3×
“User-centered design can produce usable APIs with great clarity, learnability, and programming efficiency [40, 39, 52].”
BackgroundMethodAutomatically Generating Precise Oracles from Structured Natural Language Specifications
Motwani, M., & Brun, Y. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00035 · Cited by 44 · cited 1×
“, API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundMapping the Space of API Design Decisions
Stylos, J., & Myers, B. (2007) · Proceedings/Proceedings -- IEEE Symposium on Visual Languages and Human-Centric Computing · doi:10.1109/vlhcc.2007.44 · Cited by 39
API Designers in the Field: Design Practices and Challenges for Creating Usable APIs
Murphy, L., Kery, M. B., Alliyu, O., Macvean, A., & Myers, B. A. (2018) · 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2018.8506523 · Cited by 38 · cited 1×
“Unfortunately, a recent study found that API designers still have difficulties in gathering and interpreting user feedback from their communities [37].”
BackgroundFarooq, U., Welicki, L., & Zirkler, D. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753677 · Cited by 34 · cited 3×
“Prior work on API design and usability evaluation either focuses on small-scale methods that only involve a small group of stakeholders to review API design [52, 19, 32], or only leverages pre-defined heuristics that do not account for real usage scenarios or user feedback [10, 38].”
BackgroundMethodWhat’s Spain’s Paris? Mining analogical libraries from Q&A discussions
Chen, C., Xing, Z., & Liu, Y. (2018) · Empirical Software Engineering · doi:10.1007/s10664-018-9657-y · Cited by 26
Macvean, A., Daughtry, J. M., Church, L., & Citro, C. (2016) · PPIG · Cited by 18
Macvean, A., Malý, M., & Daughtry, J. M. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2851602 · Cited by 15 · cited 2×
“Prior work on API design and usability evaluation either focuses on small-scale methods that only involve a small group of stakeholders to review API design [52, 19, 32], or only leverages pre-defined heuristics that do not account for real usage scenarios or user feedback [10, 38].”
BackgroundMethodDiscovering API usability problems at scale
Murphy-Hill, E., Sadowski, C., Head, A., Daughtry, J., Macvean, A., Jaspan, C., & Winter, C. (2018) · Proceedings of the 2nd International Workshop on API Usage and Evolution · doi:10.1145/3194793.3194795 · Cited by 15 · cited 2×
“Prior work on API design and usability evaluation either focuses on small-scale methods that only involve a small group of stakeholders to review API design [52, 19, 32], or only leverages pre-defined heuristics that do not account for real usage scenarios or user feedback [10, 38].”
BackgroundMethodMethods for Investigating Mental Models for Learners of APIs
Horvath, A., Nagy, M., Voichick, F., Kery, M. B., & Myers, B. A. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312897 · Cited by 13 · cited 1×
“Existing approaches either elicit users’ mental models directly via interviews [57, 26] or indirectly through questionnaires [28, 50], which is deemed to be expensive and unscalable given the large number of APIs and use cases.”
Background
Research works printed but not linked: 7
- Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01989 (2016). · arXiv
- Steven Clarke. 2005. Describing and measuring API usability with the cognitive dimensions. In Cognitive Dimensions of Notations 10th Anniversary Workshop. Citeseer, 131.
- 10.5555/2337223.2337255 · doi:10.5555/2337223.2337255
- Robyn Longhurst. 2003. Semi-structured interviews and focus groups. Key methods in geography 3 (2003), 143--156.
- 10.1109/WAPI.2017.2 · DOI
- 10.5555/2486788.2486804 · doi:10.5555/2486788.2486804
- 10.5555/2487085.2487146 · doi:10.5555/2487085.2487146
Cited by
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
Wish You Were Here: Mental and Physiological Effects of Remote Music Collaboration in Mixed Reality
Schlagowski, R., Nazarenko, D., Can, Y. S., Gupta, K., Mertes, S., Billinghurst, M., & André, E. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581162 · Cited by 60
“[23, 48, 104]) and allows researchers to code and analyze freeform text such as interview transcripts or, in our case, open feedback that users provided at the end of the experiment.”
BackgroundPräzi: from package-based to call-based dependency networks
Hejderup, J., Beller, M., Τriantafyllou, Κ., & Gousios, G. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10071-9 · Cited by 31
“Following Zhang et al. (2020a)’s need-finding study on data-driven API design, we are investigating possibilities to mine program contexts and error-inducing patterns using Pr¨azi to extract API usage patterns beyond syntactic features and frequencies.”
BackgroundMethodHow statically-typed functional programmers write code
Lubin, J., & Chasins, S. E. (2021) · Proceedings of the ACM on Programming Languages · doi:10.1145/3485532 · Cited by 18
“The grounded theory approach to data analysis has become increasingly popular in the software engineering community (Adolph et al., 2012, Coleman and O’Connor, 2007, Hoda et al., 2013, Jantunen and Gause, 2014, Pang et al., 2020, Zhang et al., 2020).”
MethodConcept-Annotated Examples for Library Comparison
Yan, L., Kim, M., Hartmann, B., Zhang, T., & Glassman, E. L. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545647 · Cited by 20
Self-citation
Metadata
- Authors
- Zhang, T., Hartmann, B., Kim, M., & Glassman, E. L.
- Affiliations
- Harvard University (US)
- University of California, Berkeley (US)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3028676906
- Cited by
- 25
- References
- 59
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 6 · 2023: 2 · 2022: 5 · 2021: 9 · 2020: 1
- FWCI
- 3.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (CCF-1764077, CCF-1527923, CCF-1723773)
- Office of Naval Research (N00014-18-1-2037)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- API design, API usability, user feedback, mental models, bug reports, peer review, usage mining, open source software, corner cases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.