CoChat
Sign Up
Cited by 29Software Engineering Techniques and PracticesOpen Source Software InnovationsDigital Economy and Work Transformation

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Freelancers in the Software Development Process: A Systematic Mapping Study

Open access
Gupta, Varun, Fernandez-Crehuet, Jose Maria, Hanne, Thomas·2020·Processes
DOI 10.3390/pr8101215
Open this paper in CoChat — free accountOpen PDF

Abstract

[Context] Freelancers could catalyze the software development process by providing their niche skills to generate high quality outputs. They could help companies (including startups) to foster innovations by suggesting creative ideas and providing their expertise in implementing them (for instance, designing solutions, coding solutions etc.). Freelancers could effectively and efficiently work as a virtual member of the software development team. The company must make informed decisions about which task to allot to the freelancer, which freelancer to select, pricing the task, and evaluating the submitted work. On the other hand, the freelancer should make an informed decision about evaluating the monetary value of the task to be charged, trusting the requester, analyzing the skills requirement of the task (finding matches between skill requirement and skills processed), selecting the best task, and maintaining the highest level of reputation. However, the literature does not provide freelancers and the companies the guidelines that support their decision making. However, if freelancers are selected carefully for the most suitable task, the companies will benefit a lot in terms of improved software development metrics. [Objectives] The objective of this paper is to provide the research community the research trends in freelancer-supported software development. This helps to understand that which software development areas have higher concentrations of research efforts, which area has the support of empirical evidence to support management decision makings, and which area requires the research attention. [Method] The systematic study is conducted by planning the mapping protocol, executing the protocol, and reporting the findings using various visualization tools like bar charts and pie charts. The search process was planned to be executed using set of inclusion and exclusion conditions on four bibliographic databases (IEEExplore, Springerlink, Sciencedirect, and ACM digital library). The relevant papers are selected by applying inclusion and exclusion conditions. The google citations of the relevant papers are subject to the inclusion and exclusion conditions again to include the more relevant papers. Finally, the systematic schema was created and populated after analyzing the studies abstracts. [Results] The results indicate the following (a) The research focus is on generic software development (78%) rather on individual life cycle activities. (b) The number of empirical studies is limited (25%). (c) A number of studies proposing solutions and evaluating on live cases in industrial settings are missing from the literature. This is in comparison to the validation approaches (72%) i.e., solutions tested in laboratory settings. (d) At present, the literature has limited ability to provide the software companies (including startups) with the guidelines (in the form of opinions and experience reports) for involving freelancers in the software development process. (e) The reported challenges include Collaboration and Coordination (33%), Developer Recommendation (or selection) (19%), Team Formulation (14%), Task Recommendation (allocation) (14%), Task Decomposition (11%), Privacy and Security (Confidentiality) (11%), Budget Estimation (8%), Recognition (8%), Trust Issues (8%), Market Dynamism (6%), Intellectual Property Issues (6%), Participation of Crowd Worker (6%), and Capacity Utilization (3%). These challenges are highly interactive, and each challenge impacts all other challenges. (e) Recent focus of the researchers (total 7 studies in 2019) is on generic software development handling the collaboration and coordination (3 studies out of 7), Developer recommendation (2 studies out of 7), and task recommendation (2 studies out of 7). [Conclusion] The freelancer-driven software engineering research area has got the attraction of the researchers, but it will take a long time to gain maturity. This puts an urgent call for more empirical studies and evaluation-based solution research that could help companies (including startups) to foster innovations. Further, the research focus should be well distributed among the various development phases to address the unique challenges associated with individual activities. The accurate management of the freelancer in the software development could help companies and startups to foster innovations and remain competitive in the marketplace.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/pr8101215

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Freelance tasks tend to be more self-contained than full-time software engineering tasks.

    “Freelance tasks tend to be more self-contained than full-time software engineering tasks (Gupta et al., 2020).”

    From SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? (2025) · cites this paper as Gupta et al., 2020

    The Introduction (page unavailable) describes freelancers’ work as short-term and task-based, unlike full-time employment. The Research Study section (page unavailable) says outsourced tasks need to be made self-contained, but emphasizes that doing so is challenging. It does not establish that freelance tasks tend to be more self-contained than full-time software engineering tasks. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Systematic Mapping Studies in Software Engineering

    Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · cited 3×

    “This is because systematic mapping studies are well suited to provide classification and counting of the broad research areas rather providing empirical evidence-based on the synthesis of the literature [4].”
    Background

Flagged first, then by citations

Research works printed but not linked: 46

  1. Gupta, V., Fernandez-Crehuet, J.M., Gupta, C., and Hanne, T. (2020). Freelancing Models for Fostering Innovation and Problem Solving in Software Startups: An Empirical Comparative Study. Sustainability, (Under Review). · DOI
  2. Gupta, V., Fernandez-Crehuet, J.M., Hanne, T., and Telesko, R. (2020). Requirements Engineering in Software Startups: A Systematic Mapping Study. Appl. Sci., 10. · DOI
  3. Gupta, 2020, Fostering Product Innovations in Software Startups through Freelancer Supported Requirement Engineering, Results Eng., Volume 8, 100175, 10.1016/j.rineng.2020.100175 · DOI
  4. Petersen, 2015, Guidelines for conducting systematic mapping studies in software engineering: An update, Inf. Softw. Technol., 64, 1, 10.1016/j.infsof.2015.03.007 · DOI · Paper page
  5. Petersen, K., and Gencel, C. (2013, January 23–26). Worldviews, research methods, and their relationship to validity in empirical software engineering research. Proceedings of the 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, Ankara, Turkey. · DOI
  6. Dwarakanath, A., Chintala, U., Shrikanth, N.C., Virdi, G., Kass, A., Chandran, A., and Paul, S. (2015, January 19). Crowd build: A methodology for enterprise software development using crowdsourcing. Proceedings of the 2015 IEEE/ACM 2nd International Workshop on CrowdSourcing in Software Engineering, Florence, Italy. · DOI
  7. Bernabé, R.B., Navia, I.Á., and García-Peñalvo, F.J. (2015, January 7–9). Faat: Freelance as a team. Proceedings of the 3rd International Conference on Technological Ecosystems for Enhancing Multiculturality, Porto, Portugal. · DOI
  8. Song, F., Chen, H., and Fu, Y. (2019, January 9–11). An approach to rapid worker discovery in software crowdsourcing. Proceedings of the International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP 2015), Melbourne, VIC, Australia.
  9. Murray-Rust, D., Scekic, O., and Lin, D. (2015). Worker-centric Design for Software Crowdsourcing: Towards Cloud Careers. Crowdsourcing, Springer. · DOI
  10. Shao, W., Wang, X., and Jiao, W. (2016). A developer recommendation framework in software crowdsourcing development. National Software Application Conference, Springer. · DOI
  11. Abhinav, K., and Dubey, A. (2017, January 5–7). Predicting budget for Crowdsourced and freelance software development projects. Proceedings of the 10th Innovations in Software Engineering Conference, Jaipur, India. · DOI
  12. Al-bloush, H., and Solemon, B. (2017). Intellectual Property Challenges in the Crowdsourced Software Engineering: An Analysis of Crowdsourcing Platforms. International Conference of Reliable Information and Communication Technology, Springer. · DOI
  13. Chen, Y., Lee, S.W., Xie, Y., Yang, Y., Lasecki, W.S., and Oney, S. (2017, January 6–11). Codeon: On-demand software development assistance. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA. · DOI
  14. Lee, S.W., Chen, Y., Klugman, N., Gouravajhala, S.R., Chen, A., and Lasecki, W.S. (2017, January 6–11). Exploring coordination models for ad hoc programming teams. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, CO, USA. · DOI
  15. Abhinav, K., Dubey, A., Jain, S., Virdi, G., Kass, A., and Mehta, M. (2017, January 20–28). Crowd Advisor: A framework for freelancer assessment in online marketplace. Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), Buenos Aires, Argentina. · DOI
  16. Dubey, A., Singi, K., and Kaulgud, V. (2017, January 22–23). Personas and redundancies in crowdsourced testing. Proceedings of the 2017 IEEE 12th International Conference on Global Software Engineering (ICGSE), Buenos Aires Argentina. · DOI
  17. Chatterjee, 2017, Work capacity of regulated freelance platforms: Fundamental limits and decentralized schemes, IEEE/ACM Trans. Netw., 25, 3641, 10.1109/TNET.2017.2766280 · DOI
  18. Dubey, A., Abhinav, K., and Virdi, G. (2017, January 20–28). A framework to preserve confidentiality in crowdsourced software development. Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), Buenos Aires Argentina. · DOI
  19. Arenas, C., García, V.H.M., and Otálora, J.E. (2018, January 6–10). Crowdworking as a knowledge management mechanism in the elicitation of missional software requirement. Proceedings of the International Conference on Knowledge Management in Organizations, Zilina, Slovakia. · DOI
  20. Tsikerdekis, 2018, Persistent code contribution: A ranking algorithm for code contribution in crowdsourced software, Empir. Softw. Eng., 23, 1871, 10.1007/s10664-017-9575-4 · DOI
  21. Zhang, 2020, Developer recommendation for Topcoder through a meta-learning based policy model, Empir. Softw. Eng., 25, 859, 10.1007/s10664-019-09755-0 · DOI
  22. Ivan, 2019, Using profiling to assemble an agile collaborative software development team made up of freelancers, Procedia Comput. Sci., 162, 562, 10.1016/j.procs.2019.12.024 · DOI
  23. Dubey, A., Abhinav, K., Taneja, S., Virdi, G., Dwarakanath, A., Kass, A., and Kuriakose, M.S. (2016, January 2–5). Dynamics of software development crowdsourcing. Proceedings of the 2016 IEEE 11th International Conference on Global Software Engineering (ICGSE), Orange County, CA, USA. · DOI
  24. Long, 2016, A user-oriented resource scheduling method for improving agile software pattern in cloud environment, J. Cent. S. Univ., 23, 2906, 10.1007/s11771-016-3354-z · DOI
  25. Machado, L., Kroll, J., Marczak, S., and Prikladnicki, R. (2016, January 2–5). Breaking collaboration barriers through communication practices in software crowdsourcing. Proceedings of the 2016 IEEE 11th International Conference on Global Software Engineering (ICGSE), Orange County, CA, USA. · DOI
  26. Sharma, S., Hasteer, N., and Van-Belle, J.P. (2017, January 5–6). An exploratory study on perception of Indian crowd towards crowdsourcing software development. Proceedings of the 2017 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India. · DOI
  27. Machado, L.D.S. (2018). Empirical Studies about Collaboration in Competitive Software Crowdsourcing. [Ph.D. Thesis, Pontifical Catholic University of Rio Grande do Sul]. Available online: http://tede2.pucrs.br/tede2/bitstream/tede/8256/2/LETICIA_SANTOS_MACHADO_TES.pdf.
  28. Sari, A., and Alptekin, G.I. (2018, January 20–22). A Prize Determination Approach for Crowdsourced Software Development. Proceedings of the 2018 2nd European Conference on Electrical Engineering and Computer Science (EECS), Bern, Switzerland. · DOI
  29. Hasteer, 2019, Barriers to crowdsource software development: Evaluating the intertwined effects, Int. J. Bus. Inf. Syst., 32, 127
  30. Illahi, 2019, An Empirical Study on Competitive Crowdsource Software Development: Motivating and Inhibiting Factors, IEEE Access, 7, 62042, 10.1109/ACCESS.2019.2915604 · DOI
  31. Karim, M.R., Yang, Y., Messinger, D., and Ruhe, G. (2018, January 3–6). Learn or Earn? Intelligent Task Recommendation for Competitive Crowdsourced Software Development. Proceedings of the 51st Hawaii International Conference on System Sciences, Waikoloa Village, HI, USA. · DOI
  32. Johnson, M., and Stevens, P. (2018, January 9–12). Confidentiality in the process of (model-driven) software development. Proceedings of the Conference Companion of the 2nd International Conference on Art, Science, and Engineering of Programming, New York, NY, USA. · DOI
  33. Kaur, H., Gordon, M., Yang, Y., Bigham, J.P., Teevan, J., Kamar, E., and Lasecki, W.S. (2017, January 23–26). Crowdmask: Using crowds to preserve privacy in crowd-powered systems via progressive filtering. Proceedings of the Fifth AAAI Conference on Human Computation and Crowdsourcing, Québec City, QC, Canada. · DOI
  34. Lee, S.W., Krosnick, R., Park, S.Y., Keelean, B., Vaidya, S., O’Keefe, S.D., and Lasecki, W.S. (2018). Exploring real-time collaboration in crowd-powered systems through a UI design tool. Proceedings of the ACM on Human-Computer Interaction, ACM. · DOI
  35. Aghayi, E., LaToza, T.D., Surendra, P., and Abolghasemi, S. (2019). Implementing Microservices through Microtasks. arXiv.
  36. Binzagr, F., and Medjahed, B. (2018, January 12–15). Crowdmashup: Recommending crowdsourcing teams for mashup development. Proceedings of the International Conference on Service-Oriented Computing, Hangzhou, China. · DOI
  37. Li, S., Chen, H., Shu, Y., and Hu, F. (2016, January 10–11). The verification technique of complex tasks’ nonfunctional specifications in software crowdsourcing. Proceedings of the 2016 5th International Conference on Computer Science and Network Technology (ICCSNT), Changchun, China. · DOI
  38. Shu, Y., Chen, H., Li, S., and Hu, F. (2016, January 10–11). The verification approach to complex tasks’ functional specification in software crowdsourcing. Proceedings of the 2016 5th International Conference on Computer Science and Network Technology (ICCSNT), Changchun, China. · DOI
  39. Yu, 2019, Crowdsourcing Software Task Assignment Method for Collaborative Development, IEEE Access, 7, 35743, 10.1109/ACCESS.2019.2905054 · DOI
  40. Yu, X., He, Y., Fu, Y., Xin, Y., Du, J., and Ni, W. (2019, January 16–18). Cross-Domain Developer Recommendation Algorithm Based on Feature Matching. Proceedings of the CCF Conference on Computer Supported Cooperative Work and Social Computing, Kunming, China. · DOI
  41. Wieringa, 2006, Requirements engineering paper classification and evaluation criteria: A proposal and a discussion, Requir. Eng., 11, 102, 10.1007/s00766-005-0021-6 · DOI · Paper page
  42. Paternoster, 2014, Software development in startup companies: A systematic mapping study, Inf. Softw. Technol., 56, 1200, 10.1016/j.infsof.2014.04.014 · DOI · Paper page
  43. Giardino, C., Wang, X., and Abrahamsson, P. (2014, January 16–18). Why Early-Stage Software Startups Fail: A Behavioral Framework. Proceedings of the International Conference of Software Business (ICSOB 2014), Paphos, Cyprus. · DOI
  44. Cantamessa, M., Gatteschi, V., Perboli, G., and Rosano, M. (2018). Startups’ Roads to Failure. Sustainability, 10. · DOI
  45. Tripathi, 2019, Startup ecosystem effect on minimum viable product development in software startups, Inf. Softw. Technol., 114, 77, 10.1016/j.infsof.2019.06.008 · DOI
  46. Crowne, 2002, Why software product startups fail and what to do about it. Evolution of software product development in startup companies, Proceedings of the IEEE International Engineering Management Conference, Volume 1, 338, 10.1109/IEMC.2002.1038454 · DOI

Cited by

Sorted by citations

Metadata

Authors
Gupta, V., Fernandez-Crehuet, J. M., & Hanne, T.
Affiliations
  • FHNW University of Applied Sciences and Arts Northwestern Switzerland (CH)
  • Departamento de Ingeniería de Organización, Administración de empresas y Estadística, Universidad Politécnica de Madrid, 28006 Madrid, Spain
Corresponding author
Gupta, Varun
ORCID
Venue
Processes
Publisher
MDPI AG
Published
2020-09-27
Volume
8
Issue
10
Pages
1215
ISSN
2227-9717 (ISSN-L)
DOI
10.3390/pr8101215
OpenAlex
W3088721372
Cited by
29
References
1
Citations by year
2026: 3 · 2025: 3 · 2024: 5 · 2023: 7 · 2022: 5 · 2021: 3 · 2020: 3
FWCI
4.23
Type
journal article
Language
en
Open access
Licence
Locations
  • Processes (publishedVersion, cc-by)
  • Hispana (acceptedVersion, cc-by-nc-nd)
  • Archivo Digital UPM (Universidad Politécnica de Madrid) (acceptedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Social Sciences › Sociology and Political Science › Digital Economy and Work Transformation
Keywords
freelancers, software development process, task allocation, task pricing, job evaluation, reputation management, collaboration and coordination, developer recommendation, research trends, startups, developer satisfaction, developer motivation, developer retention, developer turnover, developer onboarding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review