Cited by 77Software Engineering Techniques and PracticesUsability and User Interface DesignAdvanced Software Engineering Methodologies

The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index; 7 of the 151 sources its claims cite were checked for support.

An empirically based model of software prototyping: a mapping study and a multi-case study

Open access
Bjarnason, Elizabeth, Lang, Franz, Mjöberg, Alexander·2023·Empirical Software Engineering

Abstract

Context: Prototyping is an established practice within product and user interface design that is also used as a requirements engineering (RE) practice within agile development. Even so, there is a lack of theory on prototyping. Aims: Our main research objective is to support practitioners in improving on their prototyping practices. Method: We have designed a model that describes key aspects of the practice of prototyping. The model is based on a systematic mapping study consisting of thirty-three primary studies and on empirical data from twelve case companies. We validate and demonstrate the applicability of our model through a focus group at one company and through semi-structured interviews at eleven (other) startup companies. Results: Our prototyping aspects model (PAM) consists of five aspects of prototyping, namely purpose, prototype scope, prototype media, prototype use, and exploration strategy. This model has enabled practitioners to discuss their prototyping practices in terms of the concepts provided by our model. Conclusions: The model can be used to categorise prototyping instances and can thereby support practitioners in reflecting and improving on their prototyping practices.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents the Prototyping Aspects Model (PAM), a five-aspect model of software prototyping—purpose, prototype scope, prototype media, prototype use, and exploration strategy—built from a systematic mapping study and refined through empirical validation. PAM was used to categorize 43 prototyping instances from 11 startups and revised to add prototype media and refine purpose, use, and exploration strategy.

Key findings

  • PAM consists of five aspects: purpose of prototyping, prototype scope, prototype media, prototype use, and exploration strategy; purpose includes exploration, communication, incremental development, quality improvement, and validation & testing of problem-solution fit, product-market fit, technical feasibility, and usability.
  • The systematic mapping study included 33 primary studies published 1988–2018: 17 in software engineering, 11 in human factors & design, and 5 in requirements engineering; research types were 14 evaluation, 3 experience, 5 philosophical, 9 solution proposal, and 2 validation.
  • The multi-case study covered 11 startups (Companies A–K) through 12 interviews with 13 interviewees and identified 43 prototyping instances; the model was revised by adding prototype media as a fifth aspect, changing Communication & Alignment to Communication: Sales, Alignment, replacing market/business viability with Problem-Solution fit and Product-Market fit, and restructuring prototype use and exploration strategy.
  • The focus group at Telavox with five practitioners validated the relevance of the initial model, and practitioners agreed that the scenarios corresponded to typical agile prototyping practice.

Methods

  • The study iteratively designed PAM using a systematic mapping study, a focus group at one case company (Telavox), and a multi-case study of eleven additional startup companies, addressing RQ1–RQ4.
  • The mapping study searched LUBsearch and ACM in February 2020, yielding 4,671 articles, 146 after title scan, 54 after abstract scan, and 33 included primary studies; the multi-case study used semi-structured interviews with 11 startups, 12 interviews, and 13 interviewees.
  • Data were analyzed with open coding and thematic coding in NVivo; the model was validated through reanalysis of primary studies, the focus group, categorization of 43 prototyping instances, and interviewee validation of company-specific descriptions.

Limitations

  • Paper-stated: theoretical validity risks include possibly missing relevant previous research, the model not fully aligning with current industry practice, and the modified aspects (prototype media, prototype use, exploration strategy) not being presented to interviewees; generalizability is believed to be medium, and further research is needed with large established software product companies.
  • Paper-stated: empirical data lacked support for some aspects and facets, particularly exploration strategy and quality improvement; focus group and interview data are confidential and not publicly available, and the single first researcher’s interpretation of interview material was only partly mitigated by interviewee validation.
  • Evident from the text: validation relied on one initial case company (Telavox) and eleven startups, mostly in inception or stabilisation phases, recruited through convenience sampling from local incubators, so the findings may not extend to other organizational contexts.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s10664-023-10331-w

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

7 of 151 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Software startups typically perform light-weight, informal and ad-hoc requirements engineering, in particular during the early stages of the startup venture due to limited resources.

    “Software startups typically perform light-weight, informal and ad-hoc requirements engineering, in particular during the early stages of the startup venture due to limited resources (Giardino et al. 2014;Klotins et al. 2019;Nguyen-Duc et al. 2017;Terho et al. 2016).”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites Giardino et al. 2014 source

      The section “Software development is driven by customers, who act as designers” says startups define a minimal set of functional requirements while learning the needs of their first customers. “Practical takeaways” describes ad-hoc agile and lean methods, and “Lack of resources” says resources are extremely limited. The paper does not specifically say that requirements engineering is informal or ad hoc because of limited resources, particularly in the venture’s early stages. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Klotins et al. 2019 source

      Section 3.1 describes start-ups gathering requirements through interviews, brainstorming and customer feedback, while the Introduction notes their limited resources. The paper does not establish that requirements engineering is typically lightweight, informal and ad hoc, or that resource constraints cause this approach particularly in early stages. Page numbers were not available in our copy. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    The preference for certain types of prototype media is often due to the skills and previous experiences of prototyping technologies and tools of those involved in the startup, which play an important role when selecting the media used for prototyping.

    “This preference for certain types of prototype media is often due to the skills and previous experiences of prototyping technologies and tools of those involved in the startup, which play an important role when selecting the media used for prototyping (Gupta et al. 2021;Nguyen-Duc et al. 2017).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Gupta et al. 2021 source

      The Evaluation of Prototyping Tools section says startups selected tools based on familiarity from previous work and that competence affected their use of Adobe XD. It supports the role of skills and experience in tool choice, but does not establish that those factors determine a preference for particular types of prototype media. The supplied passages do not give a page number for this section. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data

    PAM consists of five aspects of prototyping: purpose, prototype scope, prototype media, prototype use, and exploration strategy.

    “Our prototyping aspects model (PAM) consists of five aspects of prototyping, namely purpose, prototype scope, prototype media, prototype use, and exploration strategy.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The five aspects of PAM are sufficient to characterise prototyping practices.
  • Findingnone given

    PAM has enabled practitioners to discuss their prototyping practices in terms of the concepts provided by the model.

    “This model has enabled practitioners to discuss their prototyping practices in terms of the concepts provided by our model.”
    Quote in the paperSubstantial claim
  • Findingown data

    The previous structure of the Prototype use aspect was less aligned with how the interviewees described their use of prototypes.

    “This is the aspect where the previous structure was less aligned with how the interviewees described their use of prototypes.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. 189
  • It may state two separate claims as one. 138
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Prototyping is used as a means to learn about the user and the market, and to increase the chances of business success.

    “Thus, prototyping is used as a means to learn about the user and the market (Giardino et al. 2015;Paternoster et al. 2014), and to increase the chances of business success.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Giardino et al. 2015 source
    • cites Paternoster et al. 2014 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    This cost-benefit aspects is especially important to software startups who operate under very uncertain and resource-constraint conditions with the aim of exploring new business opportunities and develop innovative products.

    “This cost-benefit aspects is especially important to software startups who operate under very uncertain and resource-constraint conditions with the aim of exploring new business opportunities and develop innovative products (Giardino et al. 2014;Paternoster et al. 2014).”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites Giardino et al. 2014 source
    • cites Paternoster et al. 2014 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    This complexity in managing quality requirements has also been observed in the context of another agile RE practice, namely the one of using test cases as requirements.

    “This complexity in managing quality requirements has also been observed in the context of another agile RE practice, namely the one of using test cases as requirements (Bjarnason et al. 2016).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Bjarnason et al. 2016 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Interviewees responded mainly by agreeing to the descriptions and highlighting minor misunderstandings and additional information, after which the manuscript was revised.

    “The interviewees responded, mainly by agreeing to the descriptions and by highlighting some minor misunderstandings and additional information, after which the manuscript was revised accordingly.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    After the first two interviews, it became clear that the terms prototyping and prototype are not uniformly understood, which caused some initial confusions at these interviews.

    “After the first two interviews, it became clear that the terms prototyping and prototype are not uniformly understood, which caused some initial confusions at these interviews.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    A similar fifth aspect of prototype media was added in the second version of our model based on the insights gained from our multi-case study.

    “However, a similar fifth aspect of prototype media was added in the second version of our model based on the insights gained from our multi-case study.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingreasoning

    The aspect of strategy was renamed exploration strategy and the aspect of review method was renamed prototype use to more clearly reflect what these aspects represent.

    “The aspect of strategy was renamed exploration strategy and the aspect of review method was renamed prototype use to more clearly reflect what these aspects represent.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingreasoning

    The facets related to validation & testing were grouped to provide a better overview of the purposes of prototyping.

    “The facets related to validation & testing were grouped to provide a better overview of the purposes of prototyping.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingreasoning

    Finally, the facet of usage environment was added to prototype use since the feedback that can be obtained from demonstrating or testing a prototype may vary for, e.g., a loud or a dark environment compared to in a meeting setting.

    “Finally, the facet of usage environment was added to prototype use since the feedback that can be obtained from demonstrating or testing a prototype may vary for, e.g., a loud or a dark environment compared to in a meeting setting.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Two additional facets were added to prototype scope, namely interactive & haptic behaviour, and realistic data, both of which were observed in the original analysis, but initially not included.

    “Two additional facets were added to prototype scope, namely interactive & haptic behaviour, and realistic data, both of which were observed in the original analysis, but initially not included.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    This internal validation mainly led to renaming aspects, adding, and restructuring some facets.

    “This internal validation mainly led to renaming aspects, adding, and restructuring some facets.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Two focus group participants said that early prototyping, e.g., through sketching, can be useful in understanding the problem.

    “In contrast, two participants said that early prototyping, e.g., through sketching, can be useful in understanding the problem.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The Exploration strategy aspect was the hardest to discuss with interviewees and the one with the least number of responses with the exception of parallel exploration.

    “This final aspect was the hardest to discuss with our interviewees, and the one with the least number of responses with the exception of parallel exploration.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    Reviewers are often internal within the startups and development team, as well as external such as customers, funders, or product users.

    “Thus, reviewers are often internal within the startups and development team, as well as, external such as customers, funders, or product users.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    A third interviewee said they focus one thing at a time and learn about, but in some cases parallel exploration is useful, e.g. for a pricing model.

    “A third interviewee said: “focus one thing at a time and learn about.But, in some cases [parallel exploration is useful] … e.g. for a pricing model” (Company I).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The focus group participants recognised the relevance of considering breadth and depth of functionality, and refinement of all identified facets except for data richness, which was not mentioned.

    “They recognised the relevance of considering breadth and depth of functionality, and refinement of all our identified facets (of the first version of our model) except for data richness, which was not mentioned during the focus group.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    One interviewee described videos as a way to present products far before they are ready.

    “One interviewee described videos as a way “to present products far before they are ready” (Company I).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Several focus group participants said that prototyping is time consuming and difficult, and rarely used during early phases and at the first meetings with customers.

    “Several participants said that prototyping is time consuming and difficult, and rarely used during early phases and at the first meetings with customers.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Prototyping instance I3 and interviews are a means of prototyping even without anything to show by talking to people about how they solve this problem today.

    “This interviewee also described prototyping instance I3 and interviews as a means of prototyping “even without anything to show” by “talking to people about how they solve this problem today” (Company I).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    The later more refined mockup A2 becomes a blueprint for the product to be built by developers.

    “The later more refined “mockup [A2] then becomes a blueprint for the product to be built by developers” (Company A).”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    Some startups find cheaper and easier prototype media such as videos and interviewing very useful.

    “which some of our startups find very useful.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    In early prototyping instance J1, Company J asked users and that became an important signal that their solution idea was viable.

    “In the early prototyping instance J1, they “asked people [users]… [and] that became an important signal” that their solution idea was viable”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence.

    Findingown data

    For more refined prototypes with respect to functionality, visual appearance, and interactivity, users may be encouraged to try the prototypes out for themselves.

    “For more refined prototypes w.r.t.functionality, visual appearance, and interactivity, the users may be encouraged to try the prototypes out for themselves.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    In prototyping instances H1, J6, I1 and I2, videos are used for sales & marketing (communication) and for validating product-market fit.

    “For example, in prototyping instances H1, J6, I1 and I2, videos are used for sales & marketing (communication) and for validating product-market fit.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    For Company J, prototyping instance J3 used FFF when they made something new and asked what they think.

    “For example, for Company J, the prototyping instance J3 used “FFF when we have made something new.What do they think?””
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it.

    Findingown data

    One interviewee believed that environment is an important aspect of mobile solutions, to test them in their correct context.

    “One interviewee believed that “[environment] is an important aspect of mobile solutions, to test them in their correct context” (Company I).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    The two facets related to prototyping iterations were indicated by a few interviewees.

    “The two facets related to prototyping iterations were indicated by a few of our interviewees.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    A startup that provides a technical solution uses scenarios to help non-techie customers see the value in the solution through presenting cases to them and avoid just clicking them through.

    “Similarly, a startup that provides a technical solution uses scenarios to help non-techie customers “see the value [in the solution] …[through] presenting these cases to them… and avoid just clicking us through” (Company G).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The dimension of usage environment worked well, even though many startups only tested their prototypes in meeting settings.

    “The dimension of usage environment worked well, even though many of our startups only tested their prototypes in meeting settings.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Some startups described using family and friends (FFF, Family, Friends, and Foes) for obtaining feedback on prototypes.

    “In addition, some startups described using family and friends (so called FFF, Family, Friends, and Foes) for obtaining feedback on prototypes.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    One interviewed technical lead advocated the need to iterate slowly to ensure what the stakeholder originally wished for.

    “For example, one interviewed technical lead advocated the “need to iterate slowly… [to ensure] what the stakeholder originally wished for” (Company D).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Mock-up C2 was used to explore what is required by the market, i.e. product-market fit.

    “For example, the mock-up C2 was used to “explore what is required [by the market]”, i.e. product-market fit”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Product-Market Fit is especially important for startups that must produce a prototype to confirm that they have buyers and thus revenue before incurring more costs without knowing that it is sellable.

    “Product-Market Fit is especially important for startups that “must produce a prototype to be able to confirm that you have buyers, and thus revenue… [before] incurring more costs without knowing that it is sellable” (Company A).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    Using multiple prototype versions in parallel was a clear and understandable strategy, though most startups stated that due to resource constraints they tend to focus on one version at a time.

    “Using multiple prototype versions in parallel was a clear and understandable strategy, though most of our startups stated that due to resource constraints they tend to focus on one version at a time.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    All interviewees described with whom prototypes were used, i.e. Reviewers, and often the same prototype is used with multiple kinds of reviewers.

    “All interviewees described with whom prototypes were used, i.e.Reviewers, and often the same prototype is used with multiple kinds of reviewers.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Findingown data

    One of the main purposes of prototyping is as a tool for selling products to customers.

    “One of the main purposes of prototyping is “as a tool for selling products to customers” (Company B)”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    The aggregated dimension of broad vs narrow functional scope was harder to convey, though examples of both broad and narrow prototypes covering many vs a few features were seen.

    “The aggregated dimension of broad vs narrow functional scope was harder to convey, though we saw examples of both broad and narrow prototypes covering many vs a few features.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    One interviewee described that they do not focus that much on interactivity at the beginning.

    “For example, one interviewee described that “we do not focus that much on interactivity at the beginning” (Company F).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    A startup that markets advanced AI algorithms described that they need accurate data to back up their algorithm and thus demonstrate their solution by using actual customer data.

    “Another startup that markets advanced AI algorithms described that they “need accurate data to back up [our algorithm]”, and thus demonstrate their solution by using “actual [customer] data” (Company H).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    There was some disagreement about prototyping in early project stages.

    “There was some disagreement about prototyping in early project stages.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingnone given

    A similar aspect related to paper prototyping was discussed during the original design of the model but discarded because prototype scope appeared sufficient.

    “A similar aspect, related to paper prototyping, was discussed by the authors during our original design of the model, but discarded at that point in time since the aspect of prototype scope appeared to be sufficient.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    When demonstrating paper mock-ups of prototyping instance F1, the startup would try to get customers to think about what they do then in that scenario.

    “For example, when demonstrating the paper mock-ups of prototyping instance F1, the startup would “try to get them [customers] to think about what they do then [in that scenario] (Company F).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Company J used videos in social media channels for sales & marketing purposes; funny videos became a bit viral.

    “Similarly, the now growing startup Company J used videos in social media channels for sales & marketing purposes; “funny videos that became a bit viral” (Company J).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    While scenario testing was included in the initial version of the model, interviewees primarily described the use of scenarios as an approach that could be applied both when demoing a prototype or when allowing users to interact directly with the prototype.

    “Finally, while scenario testing was included in our initial version of the model, our interviewees primarily described the use of scenarios as an approach that could be applied both when demoing a prototype, or when allowing users to interact directly with the prototype.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Company A invested quite a lot of work in building a high-fidelity interactive mockup (A2) that is fake but good, communicates that they know and builds trust, and based on that they get investors.

    “Company A had invested “quite a lot of work” in building a high-fidelity interactive mockup (prototyping instance A2) which “is fake, but quite good… communicates that we know and builds trust… Based on that we get investors.”(Company A).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    Users are encouraged to interact directly with the mock-up of prototyping instance F2 to see reactions, how they reason, how they select and react to things, which provides the startup with rich feedback.

    “Similarly, users are encouraged to interact directly with the mock-up of prototyping instance F2 “to see reactions….how they reason, how they select and react to things” (Company F), which provides the startup with rich feedback.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it.

    Findingown data

    When data is fairly realistic it can be shown to customers without any problems.

    “and that when “the data is fairly realistic… [it] can be shown to customer without any problems” (Company G).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    In total, we interviewed thirteen practitioners at the case companies, mostly founders and co-founders, but also some product owners and technical leads.

    “In total, we interviewed thirteen practitioners at the case companies, mostly founders and co-founders, but also some product owners and technical leads.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    For prototyping instances K4 and K5, the startup goes through a simple scenario, explains the challenges, and shows the solution.

    “A similar approach is seen for prototyping instances K4 and K5, where the startup “goes through a simple scenario, explains the challenges, and shows the solution” (Company K).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    The sketches of prototyping instance A1 were tried out on themselves and on people in their proximity, then they started developing.

    “For example, the sketches of prototyping instance A1 were “tried out on ourselves and on people in our proximity, then we started developing” (Company A).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    These participants preferred stakeholder interviews and competitor analysis to understand the problem domain.

    “Instead, these participants preferred stakeholder interviews and competitor analysis to understand the problem domain.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Company J uses interviewing for exploring the problem domain.

    “This startup also uses interviewing for exploring the problem domain.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingnone given

    For Validation & Testing, the boundaries between Market viability and Business viability were hard to determine for the prototyping instances, and the difference between these two is unclear.

    “For Validation & Testing, the boundaries between Market viability and Business viability were hard to determine for the prototyping instances, and the difference between these two is unclear.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingcitation

    Most of these companies were in the inception or stabilisation phase of their business ventures, while Companies E and J were in the growth stage.

    “Most of these companies were in the inception or stabilisation phase (Klotins et al. 2019) of their business ventures, while Companies E and J were in the growth stage (Klotins et al. 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Klotins et al. 2019

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    For Company F, testing in a live environment is difficult since their product targets clinics and it varies a lot how open they are with this.

    “For example, for Company F where testing in a live environment is “difficult since our product targets clinics… varies a lot how open they are with this.””
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    The fully functioning product prototype of B3 was available to the general public and customers get to use their stuff, allowing the startup to follow up how they are used in the field.

    “For example, the fully functioning product prototype of B3 was available to the general public and “customers get to use our stuff… [allowing the startup to] follow up how they are used in the field” (Company B).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    They had used simple prototypes in the form of sketches, which helped designers and users understand what they were discussing and assisted in framing the role of the system.

    “They had used simple prototypes in the form of sketches, which helped designers and users understand what they were discussing and assisted in framing the role of the system.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Interviewees described that they have chosen to produce a mock-up due to cost and time aspects.

    “Our interviewees described that they “have chosen to produce a mock-up due to cost and time aspects” (Company A).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Another interviewee mentioned that they always try to sketch different options, though without giving any concrete example of this.

    “Another interviewee mentioned that “we always try to sketch different [options]” (Company F) though without giving any concrete example of this (and therefor not seen in any of the reported prototyping instances.)”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    One interviewee said they go with one option first since they are a small team.

    “As one interviewee said: “we go with one [option] first since we are a small team” (Company H).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The authors may not put it forward as true. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

    Findingown data

    It is important to use realistic data otherwise customers probably get stuck on that.

    “Also, “it is really important to use that [realistic data] otherwise your [customers] probably get stuck on that” (Company F)”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    The Purpose aspect mostly worked well in discussing and categorising how the startups use prototyping, with only two minor revisions for Communication and Validation.

    “This aspect mostly worked well in discussing and categorising how the startups use prototyping, and only two minor revisions are made to this aspect, namely for the purposes Communication and Validation.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    For J4, Company J grabbed people in the corridors and asked them what they think, and then realised that they had missed several things.

    “and for J4 they “grabbed people in the corridors and ask them what they think… [and then] realised that we had missed several things”(Company J).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Findingown data

    In general, the focus group participants agreed to the scenarios, that they correspond to a typical way of working and to their prototyping practices, which they see as a good way of working with agile software development.

    “In general, the participants agreed to the scenarios, that they correspond to a typical way of working and to their prototyping practices, which they see as a good way of working with agile software development.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Company J later used a similar approach in prototyping instance J5 to validate their revenue model.

    “and later used a similar approach in prototyping instance J5 to validate their revenue model.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Five interviewees expressed the importance of realistic data, e.g. to capture behaviour around errors and bad data.

    “In addition, five interviewees expressed the importance of realistic data, e.g. to capture behaviour around “errors and bad data” (Company I).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    Startups then move towards more gradual development and more structured feature growth from the very basic experimental ideas up to the ready-for-market products.

    “And, then move towards “more gradual development … and more structured feature growth from the very basic experimental ideas up to the ready-formarket products” (Company E).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingreasoning

    Single option exploration is the most common approach among our startups since it is more cost-effective.

    “Single option exploration is the most common approach among our startups since it is more cost-effective.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    The multi-case study described 43 prototyping instances.

    “Overview of the 43 prototyping instances (Nn) described by case companies (N = A-K), categorized by our prototyping aspects model.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingnone given

    Our research indicates a number of potential relationships within the entities of the model, e.g. between prototype scope and prototyping purpose, that appear to affect the learnings that can be obtained.

    “Furthermore, our research indicates a number of potential relationships within the entities of the model, e.g. between prototype scope and prototyping purpose, that appear to affect the learnings that can be obtained.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    The feedback from the interviewees mainly concerned additions, e.g. mentioning more prototyping instances, which were then incorporated in the paper.

    “The feedback from the interviewees mainly concerned additions, e.g.mentioning more prototyping instances, which were then incorporated in the paper.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingnone given

    We have investigated the current body of knowledge regarding prototyping methodology through a systematic mapping study and a multi-case study of eleven startups.

    “We have investigated the current body of knowledge regarding prototyping methodology through a systematic mapping study and a multi-case study of eleven startups.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    While UX designers can quickly produce mock-ups using tools, e.g. Figma, software developers often prefer sketching and exploring their ideas directly in source code software.

    “For example, while UX designers can quickly produce mock-ups using tools, e.g.Figma, software developers often prefer sketching and exploring their ideas directly in source code software.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    We observe an increase in publication rate during the past decade and of empirically based research (RQ3).

    “We observe an increase in publication rate during the past decade and of empirically based research (RQ3).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    This is similar to our focus group participants who described only using single exploration in later development stages, while preferring a strategy of parallel exploration in early stages to keep an open mind to alternative solutions and avoid getting locked into one solution option at an early stage.

    “This is similar to our focus group participants who described only using single exploration in later development stages, while preferring a strategy of parallel exploration in early stages to keep an open mind to alternative solutions and avoid getting locked into one solution option at an early stage.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    The number of articles appear to have increased somewhat over the years from on average 1 to 2 articles a year, except around the year 2000.

    “The number of articles appear to have increased somewhat over the years from on average 1 to 2 articles a year, except around the year 2000.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    Our focus group participants described using prototypes with customers either through a demonstration without any direct user interaction, or as free testing of a beta version of the software.

    “Our focus group participants described using prototypes with customers either through a demonstration without any direct user interaction, or as free testing of a beta version of the software.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    Five interviewees in the startups could relate to the importance of using realistic data, mainly to convey realistic scenarios and trust to customers, but also for validating error cases connected to missing or badly formatted data.

    “However, five of our interviewees in the startups could relate to the importance of using realistic data, mainly to convey realistic scenarios and trust to customers, but also for validating error cases connected to missing or badly formatted data.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    Focus group participants actively related to three facets of refinement, namely functionality, visual appearance and interactive behaviour, but did not mention data richness.

    “While our focus group participants actively related to three of the facets of refinement, namely functionality, visual appearance and interactive behaviour, they did not mention data richness.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    The systematic map on which our model is based consists of thirty-three primary studies and represents a wide set of papers from the past thirty years from the areas of human factors and design, requirements engineering, and agile software development (RQ2).

    “The systematic map on which our model is based consists of thirty-three primary studies and represents a wide set of papers from the past thirty years from the areas of human factors and design, requirements engineering, and agile software development (RQ2).”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    While all these strategies are relevant to prototyping, we found that when discussing exploration strategy with practitioners a different set of facets is more suitable.

    “While all these strategies are relevant to prototyping, we found that when discussing exploration strategy with practitioners a different set of facets is more suitable.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    The majority of the primary studies (17) were within software engineering (in general), 11 within human factors (HF) & design, and 5 within requirements engineering (RE).

    “The majority of these (17) were within software engineering (in general), 11 within human factors (HF) & design, and 5 within requirements engineering (RE), see Table5.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingnone given

    Direct interaction with prototypes is common for internal use and when the prototype is slightly more refined.

    “Direct interaction with prototypes is common for internal use and when the prototype is slightly more refined.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Findingnone given

    There appears to be an increase of in-vivo research on prototyping methodology during the past decade.

    “although there appears to be an increase of in-vivo research during the past decade.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    An additional purpose of prototyping is to specify requirements and to act as a requirements specification, as was mentioned by several of our case companies.

    “An additional purpose of prototyping is to specify requirements and to act as a requirements specification, as was mentioned by several of our case companies.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Study design and conductown data

    PAM is based on a systematic mapping study of thirty-three primary studies and empirical data from twelve case companies.

    “The model is based on a systematic mapping study consisting of thirty-three primary studies and on empirical data from twelve case companies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The systematic mapping study's primary studies cover the relevant body of knowledge on prototyping methodology.
  • Probably not a substantial claim.

    Interpretationreasoning

    Software startups face the challenge of balancing the cost of prototype scope and media against the gains and value that can be obtained from more refined prototypes.

    “Thus, software startups face the challenge of balancing the cost of prototype scope and media against the gains and value that can be obtained from more refined prototypes.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationreasoning

    PAM can be used to categorise prototyping instances and can thereby support practitioners in reflecting and improving on their prototyping practices.

    “The model can be used to categorise prototyping instances and can thereby support practitioners in reflecting and improving on their prototyping practices.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The five aspects of PAM are sufficient to characterise prototyping practices.
  • It may state two separate claims as one.

    Premisecitation

    Internally, prototyping is used to explore and communicate detailed product requirements, while prototypes are used externally to communicate and validate product-market fit.

    “Internally, prototyping is used to explore and communicate detailed product requirements, while prototypes are used externally to communicate and validate product-market fit.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • It may state two separate claims as one.

    Premisecitation

    For startups, prototyping plays a vital role in market validation and in obtaining paying customers.

    “Thus, for startups, prototyping plays a vital role in market validation and in obtaining paying customers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    High-fidelity prototypes are more expensive to build but can be used to test and evaluate further details of the design, and even serve as a basis for development of the product under design.

    “while high-fidelity prototypes are more expensive to build they can be used to test and evaluate further details of the design, and even serve as a basis for development of the product under design (Rudd et al. 1996).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rudd et al. 1996 source
  • It may state two separate claims as one.

    Premisecitation

    The more refined prototype versions (realistic mock-ups and early versions of production software) are more costly to produce and require software engineering expertise.

    “The more refined prototype versions (realistic mock-ups and early versions of production software) are more costly to produce and require software engineering expertise.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Low-fidelity prototypes can convey a general look of an interface and are often used to communicate, educate and inform.

    “Low-fidelity prototypes can convey a general look of an interface and are often used to communicate, educate and inform, while high-fidelity prototypes are more expensive to build they can be used to test and evaluate further details of the design, and even serve as a basis for development of the product under design (Rudd et al. 1996).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rudd et al. 1996 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    In an earlier publication, prototyping is commonly used within early-stage startups as a natural part of developing and validating a product, but also that prototyping is implicitly required to ensure funding of the startup venture.

    “In that study (Bjarnason 2021), we found that prototyping is commonly used within early-stage startups as a natural part of developing and validating a product, but also that prototyping is implicitly required to ensure funding of the startup venture.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping can facilitate communication of design ideas by providing a concrete basis for discussions between developers, designers, and users.

    “Prototypes can facilitate communication of design ideas by providing “a concrete basis for discussions” between developers, designers, and users (Budde and Zullighoven 1990).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisereasoning

    The use of prototypes can stimulate user involvement, and support marketing and selling new product ideas to customers and to (internal) management.

    “Thus, the use of prototypes can stimulate user involvement, and support marketing and selling new product ideas to customers and to (internal) management.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Study design and conductown data

    PAM was validated and its applicability demonstrated through a focus group at one company and through semistructured interviews at eleven startup companies.

    “We validate and demonstrate the applicability of our model through a focus group at one company and through semistructured interviews at eleven (other) startup companies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The case companies' prototyping practices are representative enough to validate and improve PAM.
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping enables validating a solution proposal before developing the full product through cost effective testing with real users.

    “Thus, prototyping enables validating a solution proposal before developing the full product through cost effective testing with real users (Nielsen 1993).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nielsen 1993 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototypes serve as a vehicle that enables users and designers to develop a common language.

    “As such, prototypes “serve as a vehicle that enables users and designers to develop a common language” (Hakim and Spitzer 2000).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hakim and Spitzer 2000

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Premisecitation

    Software startups were selected as our object of study due to these companies frequent use of prototyping to explore and shape their business models and product offerings, as part of the Lean startup method where new products are designed through continuously building, measuring, and learning what the customers want and are willing to pay for.

    “Software startups were selected as our object of study due to these companies frequent use of prototyping to explore and shape their business models and product offerings, as part of the Lean startup method (Ries 2011) where new products are designed through continuously building, measuring, and learning what the customers want and are willing to pay for (Olsen 2015).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Ries 2011
    • cites Olsen 2015 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductown data

    We performed an initial validation of the draft version of the prototyping aspects model by re-analysing the articles in our systematic map.

    “We performed an initial validation of the draft version of the prototyping aspects model by re-analysing the articles in our systematic map (see below).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Prototyping is often used within agile development to detail and validate requirements, and to reduce uncertainty.

    “Thus, prototyping is often used within agile development to detail and validate requirements, and to reduce uncertainty (Bellomo et al. 2013).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bellomo et al. 2013 source
  • It may state two separate claims as one.

    Premisecitation

    User interface design can be validated through prototyping and uncover various usability issues.

    “User interface design can be validated through prototyping (Derboven et al. 2010;Hakim and Spitzer 2000;Zink et al. 2017) and uncover various usability issues (Lim et al. 2008).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Derboven et al. 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Hakim and Spitzer 2000

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Lim et al. 2008 source
  • Probably not a substantial claim.

    Study design and conductnone given

    The mapping study provided a broad base of scientific knowledge that supported our design of the model.

    “The mapping study provided a broad base of scientific knowledge that supported our design of the model.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The systematic mapping study's primary studies cover the relevant body of knowledge on prototyping methodology.
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Sefelin et al. reports that the number of suggestions and critiques do not significantly differ between prototypes with a high vs. low degree of refinement.

    “Sefelin et al. reports on a study on usability testing with prototypes where the number of suggestions and critiques do not significantly differ between prototypes with a high vs. low degree of refinement, i.e. between deep or shallow prototypes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sefelin et al. 2003 source
  • Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Yasar reports that simpler prototypes with narrow functionality scope and a low degree of refinement are cost effective and useful for validation purposes and to capture major issues.

    “Yasar even reports that simpler prototypes with narrow functionality scope and a low degree of refinement are cost effective and useful for validation purposes and to capture major issues (Yasar 207”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yasar 2007 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Prototyping has been identified as a practice commonly applied within agile software development to manage rapidly evolving requirements by the practices' ability to support customer communication, and for validating and refining requirements.

    “Prototyping has been identified as a practice commonly applied within agile software development to manage rapidly evolving requirements by the practices' ability to support customer communication, and for validating and refining requirements (Ramesh et al. 2010).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ramesh et al. 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    To ensure equal airtime, the participants were given turns to initiate the discussion for each talking point.

    “To ensure equal airtime, the participants were given turns to initiate the discussion for each talking point.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Premisecitation

    The source of requirements gradually shifts as potential customers are identified, and prototypes are produced that can be used to elicit new ideas and priorities also from customers and other external parties.

    “The source of requirements gradually shifts as potential customers are identified, and prototypes are produced that can be used to elicit new ideas and priorities also from customers and other external parties (Nguyen-Duc et al. 2017;Terho et al. 2016).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Terho et al. 2016 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductnone given

    Data extraction, data analysis and classification of general and specific items of the primary studies was performed by the last two authors, and then validated by the first author.

    “Data extraction, data analysis and classification of general and specific items of the primary studies was performed by the last two authors, and then validated by the first author.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductnone given

    The two last authors each performed this selection on the set from ACM and LUBsearch respectively. They aligned their selection by comparing and reaching consensus on inclusion and exclusion for a random set of 10 publications.

    “The two last authors each performed this selection on the set from ACM and LUBsearch respectively.They aligned their selection by comparing and reaching consensus on inclusion and exclusion for a random set of 10 publications.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductown data

    Searches were defined iteratively in-line with RQ1 through test searches and by consulting with two experts in prototyping.

    “Searches were defined iteratively in-line with RQ1 through test searches and by consulting with two experts in prototyping.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Very few startups work with parallel exploration and prototyping, and those who did work with parallel prototypes tended to have a background in usability and human-interaction design.

    “In our previous research on the use of prototyping in startups, we found that very few startups work with parallel exploration and prototyping, and that those who did work with parallel prototypes tended to have a background in usability and human-interaction design (Bjarnason 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    Through the case study, we explored the practical applicability of our model by using it to discuss and categorise prototyping instances, thereby validating the model further and identifying improvements to it (RQ4).

    “Through the case study, we explored the practical applicability of our model by using it to discuss and categorise prototyping instances, thereby validating the model further and identifying improvements to it (RQ4).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Problems and new solution directions can be discovered and explored through prototyping and can lead to new ideas and direct further exploration.

    “Problems and new solution directions can be discovered and explored through prototyping and can lead to new ideas and direct further exploration (Lim et al. 2008).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lim et al. 2008 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    Each participant was sent a draft of the resulting articles for the parts derived from their interview for validation purposes.

    “Furthermore, each participant was sent a draft of the resulting articles for the parts derived from their interview for validation purposes.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    In the validation stage, interviewees were asked to validate descriptions related to their companies and the co-authors discussed and agreed to the revised version of the prototyping aspects model.

    “Finally, in the validation stage of our case study, the interviewees were asked to validate the descriptions related to their companies and the co-authors discussed and agreed to the revised version of the prototyping aspects model.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Startups struggle to define solutions for which there is product-market fit and risk wasting time and resources on developing unsuccessful features.

    “While availability of open-source software and pay-as-you-go services provide software-based business opportunities, startups struggle to define solutions for which there is product-market fit and risk wasting time and resources on developing unsuccessful features (Giardino et al. 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Giardino et al. 2015 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Several researchers report that some usability issues can not be discovered unless the prototype provides a broad functional scope with a high degree of refinement, in particular for visual appearance and interactive behaviour.

    “Several researchers report that some usability issues can not be discovered unless the prototype provides a broad functional scope with a high degree of refinement, in particular for visual appearance and interactive behaviour (Liu and Khooshabeh 2003;McCurdy et al. 2006).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Liu and Khooshabeh 2003 source
    • cites McCurdy et al. 2006 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    An executable prototype provides a rich means of communicating requirements details, and reduces the risk of ambiguity, incompleteness and inconsistency in the requirements communication.

    “In particular, an executable prototype provides a rich means of communicating requirements details, and reduces the risk of ambiguity, incompleteness and inconsistency in the requirements communication (Acosta et al. 1994).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Acosta et al. 1994

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    The description of the case company and the case-specific results have been reviewed by the case company.

    “Furthermore, the description of the case company and the case-specific results have been reviewed by the case company.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Study design and conductown data

    Interviewees were provided with the prototyping instances identified for their startup, memos including quotes from transcripts, and company and interviewee characteristics.

    “The interviewees were provided with the prototyping instances identified for their startup, the memos related to these including the quotes extracted from the transcripts, and the company and interviewee characteristics reported in Section 4.2.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    Breadboard prototypes are used to investigate technical aspects and to support system specification and coding.

    “Similarly, breadboard prototypes are used to investigate technical aspects, e.g. in circuit design, and to support system specification and coding (Budde and Zullighoven 1990;Ciriello et al. 2017;Lichter et al. 1994;Lim et al. 2008).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Ciriello et al. 2017 source
    • cites Lichter et al. 1994 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The use of a live product version as a prototype poses conflicts between the need for quick feedback and a focus on product quality.

    “However, the use of a live product version as a prototype poses conflicts between the need for quick feedback and a focus on product quality (Ciriello et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ciriello et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping helped with challenges related to managing with very little requirements documentation, such as intangible and unaligned views and plans on what to develop.

    “They observed that prototyping helped with challenges related to managing with very little requirements documentation, such as intangible and unaligned views and plans on what to develop.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Käpyaho and Kauppinen 2015 source
  • It may state two separate claims as one.

    Premisecitation

    Prototyping is commonly used to explore the solution space and learn by experimenting with ideas, and is a foundational purpose for any prototyping.

    “Prototyping is commonly used to explore the solution space (Dow et al. 2011;Lim et al. 2008;Rahman et al. 2017;Wiberg and Stolterman 2014) and learn by experimenting with ideas (Budde and Zullighoven 1990;Chen et al. 1994;Tronvoll et al. 2017), and is a foundational purpose for any prototyping.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Dow et al. 2011 source
    • cites Lim et al. 2008 source
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationreasoning

    The facet of realistic data relates to look & feel, but primarily to the perspective of role since the degree to which provided data simulates actual use affects the user's ability to relate to their situation.

    “The facet of realistic data relates to look & feel, but primarily to the perspective of role since the degree to which provided data simulates actual use affects the user's ability to relate to their situation.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationown data

    We have identified eight purposes of prototyping: exploration, communication, incremental development, quality improvement, and validation & testing of problem-solution fit, product-market fit, technical feasibility, and usability.

    “We have identified eight purposes, namely exploration, communication, incremental development, quality improvement, and validation & testing of problem-solution fit, product-market fit, technical feasibility, and usability.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim.

    Interpretationcitation

    Our model characterizes the methodology of prototyping by five aspects: purpose of prototyping, prototype scope, prototype media, use of prototype, and exploration strategy.

    “Our model characterizes the methodology of prototyping by five aspects: purpose of prototyping, prototype scope, prototype media, use of prototype, and exploration strategy, and is here described based on related work.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    Prototyping facilitates presenting, discussing, and evaluating a product with external parties and internally within a project, thereby supporting decision making.

    “Prototyping thus facilitates presenting, discussing, and evaluating a product with external parties, such as customers and investors, and internally within a project, thereby supporting decision making (Budde and Zullighoven 1990;Raja 2009).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Raja 2009 source
  • Probably not a substantial claim.

    Study design and conductcitation

    The systematic mapping study was based on guidelines by Petersen et al. (Petersen et al. 2015) to explore and draw from current body of knowledge on prototyping to compile and provide an overarching view of prototyping methodology based on current knowledge.

    “We performed a systematic mapping study based on guidelines by Petersen et al. (Petersen et al. 2015) to explore and draw from the current body of knowledge on prototyping to compile and provide an overarching view of prototyping methodology based on current knowledge.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Petersen et al. 2015 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Yasar describes three perspectives to consider when prototyping: role, look & feel, and implementation.

    “Yasar describes three perspective to consider when prototyping: role (of product in a user's life), look & feel, and implementation (Yasar 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yasar 2007 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Prototyping addresses several challenges within agile but also identify a need to complement prototyping with additional practices.

    “Their findings indicate that prototyping addresses several challenges within agile but also identify a need to complement prototyping with additional practices.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Käpyaho and Kauppinen 2015 source
  • It may state two separate claims as one.

    Premisecitation

    Visions and ideas about a product can be communicated through prototyping, which provides a common language between developers and stakeholders and acts as an anchor for group communication.

    “Visions and ideas about a product can be communicated through prototyping, which provides a common language between developers and stakeholders (Budde and Zullighoven 1990;Ciriello et al. 2017;Rahman et al. 2017;Zink et al. 2017) and acts as an anchor for group communication (Dow et al. 2011).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Ciriello et al. 2017 source
    • cites Dow et al. 2011 source
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationcitation

    The scope of a prototype describes the extent to which a prototype resembles the final product and is represented by breadth of functionality, functional refinement, and data realism.

    “The scope of a prototype describes the extent to which a prototype resembles the final product and is, in our model, represented by the breadth of the prototype's functionality (Bruegger et al. 2009;Budde and Zullighoven 1990;Goldman and Narayanaswamy 1992;Hakim and Spitzer 2000;Lim et al. 2008;McCurdy et al. 2006;Tronvoll et al. 2017), and the facets of functional refinement (Bruegger et al. 2009”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Bruegger et al. 2009 source
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Lim et al. 2008 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    One case study of prototyping in twenty Norwegian startups identified factors that affect the speed of prototyping, including the choice of prototype tools and components, varying competences, and the communication and involvement of customers and external stakeholders in the prototyping.

    “One case study of prototyping in twenty Norwegian startups, by Nguyen-Duc et al., identified factors that affect the speed of prototyping, including the choice of prototype tools and components, varying competences, and the communication and involvement of customers and external stakeholders in the prototyping.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The purpose of prototyping and how the prototype is used regarding customer involvement are factors that need to be considered when selecting prototyping practices.

    “They observed that the purpose of prototyping and how the prototype is used regarding customer involvement are factors that need to be considered when selecting prototyping practices (Nguyen-Duc et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping multiple design in parallel and sharing these led to more creative and better final designs.

    “Research on the use of prototyping in generating new user interaction designs found that prototyping multiple design in parallel and sharing these led to more creative and better final designs (Dow et al. 2011).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dow et al. 2011 source
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductnone given

    The reanalysis was performed to improve reliability and internal validation of our prototyping aspects model, while the focus group was performed to validate the relevance and usefulness of our model with practitioners.

    “The reanalysis was performed to improve reliability and internal validation of our prototyping aspects model, while the focus group was performed to validate the relevance and usefulness of our model with practitioners.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given

    In our model, look & feel is captured by the facets of visual and interactive & haptic behaviour.

    “In our model, look & feel is captured by the facets of visual and interactive & haptic behaviour.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Study design and conductown data

    The initial draft version of our prototyping aspects model was validated through reanalysis of the primary studies of our systematic map and through a focus group at a case company.

    “The initial draft version of our prototyping aspects model was validated through reanalysis of the primary studies of our systematic map and through a focus group at a case company.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    Prototyping can be used to investigate customer needs, validate and clarify customer requirements and tasks, and thus support a company in designing a solution to address customer problems.

    “Prototyping can be used to investigate customer needs, validate and clarify customer requirements and tasks (Budde and Zullighoven 1990;Ciriello et al. 2017), and thus support a company in designing a solution to address customer problems.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Ciriello et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping enables cost effective testing with users.

    “and thus, enables cost effective testing with users (Shepherd and Gruber 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Shepherd and Gruber 2020 source
  • It may state two separate claims as one.

    Premisecitation

    Demonstrating a fully functioning (but early) version of production software may convey an overly positive view of development status to customers and other stakeholders, that in turn may lead to a push to release software prematurely, before sufficient quality has been achieved.

    “In this case, demonstrating a fully functioning (but early) version of production software may convey an overly positive view of development status to customers and other stakeholders, that in turn may lead to a push to release software prematurely, before sufficient quality has been achieved.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ramesh et al. 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Premisecitation

    Prototyping can be used to explore a product's business potential from the customer perspective, and to assess the value of the product and likeliness of purchase and the product's ability to fit within time and budget constraints.

    “Prototyping can be used to explores a product's business potential from the customer perspective, and to assess the value of the product and likeliness of purchase (McCurdy et al. 2006;Zink et al. 2017) and the product's ability to fit within time and budget constraints (Zink et al. 2017).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites McCurdy et al. 2006 source
    • cites Zink et al. 2017 source
  • It may state two separate claims as one.

    Premisecitation

    The scope affects the cost of producing a prototype and the feedback that can be obtained.

    “The scope affects the cost of producing a prototype and the feedback that can be obtained (Derboven et al. 2010;Lim et al. 2008;Liu and Khooshabeh 2003;McCurdy et al. 2006;Tronvoll et al. 2017).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Derboven et al. 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Lim et al. 2008 source
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    A prototype built to demonstrate feasibility can also support project acquisition.

    “Furthermore, Lichter et al. found that a prototype built to demonstrate feasibility can also support project acquisition (Lichter et al. 1994).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lichter et al. 1994 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The insights gained from such prototyping can be used to support business-related decision making.

    “The insights gained from such prototyping can be used to support business-related decision making (Ciriello et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ciriello et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Feasibility testing requires a high share of the time invested in a project.

    “Finally, Zink et al. found that feasibility testing requires a high share of the time invested in a project (Zink et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zink et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    The quality of stakeholder communication at the case company was increased through prototyping and mutual understanding between the development team and the customers was reached faster.

    “Furthermore, the quality of stakeholder communication at the case company was increased through prototyping and mutual understanding between the development team and the customers was reached faster.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Käpyaho and Kauppinen 2015 source
  • Probably not a substantial claim.

    Study design and conductown data

    The first author performed triangulation of the model through an independent re-analysis of the primary studies of our systematic map including (re)coding the full text of the articles in NVivo.

    “The first author performed triangulation of the model through an independent re-analysis of the primary studies of our systematic map including (re)coding the full text of the articles in NVivo.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Prototyping also increased the motivation for requirements work since updating a prototype was considered more motivating than writing requirements documents.

    “Prototyping also increased the motivation for requirements work since updating a prototype was considered more motivating than writing requirements documents (Käpyaho and Kauppinen 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Käpyaho and Kauppinen 2015 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Users preferred testing with a prototype with a high degree of interactive refinement since this provided freedom to explore the system by themselves.

    “However, the users preferred testing with a prototype with a high degree of interactive refinement since this provided freedom to explore the system by themselves (Sefelin et al. 2003).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sefelin et al. 2003 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    In agile development, prototyping is often an integral part of the development process with regular feedback from users and other stakeholders.

    “In agile development, prototyping is often an integral part of the development process with regular feedback from users and other stakeholders (Fairley and Willshire 2005), e.g. through validation of software at end-of-sprint demonstrations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fairley and Willshire 2005 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    One important success factor is to test the business idea early on to validate its viability in the market.

    “One important success factor is to test the business idea early on to validate its viability in the market (Block and MacMillan 1985).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Block and MacMillan 1985

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductown data

    All interviewees were contacted and asked to validate information relevant to their companies and to indicate whether they wished their company name included.

    “All interviewees were contacted and asked to validate the information relevant to their companies, and to indicate if they wished their company name to be included in the article or not.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    For the first version of the model, an additional design iteration was performed to increase internal validity of our model.

    “For the first version of the model, an additional design iteration was performed to increase internal validity of our model.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    Study selection was performed through a gradually extended scan of title, abstract, and full text using inclusion and exclusion criteria to guide the selection decisions, which resulted in identifying 33 primary studies.

    “Study selection was performed through a gradually extended scan of title, abstract, and full text using inclusion and exclusion criteria to guide the selection decisions (see below), which resulted in identifying 33 primary studies.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Usability testing was reported as the least common singular purpose for prototyping though it is often combined with other purposes.

    “Usability testing was reported by Zink et al. as the least common singular purpose for prototyping though it is often combined with other purposes (Zink et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zink et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    As the prototype representation of the (future) product becomes more concrete with each iteration, the product requirements thereby gradually become more specific.

    “As the prototype representation of the (future) product becomes more concrete with each iteration, the product requirements thereby gradually become more specific (Zink et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zink et al. 2017 source
  • It may state two separate claims as one.

    Premisecitation

    Proof-of-concept prototypes are built for technical feasibility and can be used to evaluate novel technical approaches.

    “Proof-of-concept prototypes are built for this purpose (Tronvoll et al. 2017) and can be used to evaluate novel technical approaches (Fern and Donaldson 1989).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Tronvoll et al. 2017 source
    • cites Fern and Donaldson 1989

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Study design and conductown data

    Our participants validated the four exploration strategies included in the first version model and described that the strategy is varied throughout development stages and for different prototyping purposes.

    “Our participants validated the four exploration strategies included in the first version model and described that the strategy is varied throughout development stages and for different prototyping purposes.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    Tronvoll's optimisation exploration uses parallel exploration with a focus on feature or product level, and with gradually decreasing iteration sizes.

    “Tronvoll's optimisation exploration uses parallel exploration with a focus on feature or product level, and with gradually decreasing iteration sizes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim.

    Interpretationcitation

    Our facet of single exploration corresponds to Tronvoll's point-based strategy where resources are focused on one single solution path and where alternatives are considered before being cemented.

    “Our facet of single exploration corresponds to Tronvoll's point-based strategy where resources are focused on one single solution path and where alternatives are considered before being cemented.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationreasoning

    The difference between point-based and flexible exploration could be deferred to the size of changes and connected to the stage and maturity of a product or an idea.

    “Similarly, the difference between point-based and flexible could also be deferred to the size of changes and connected to the stage and maturity of a product or an idea.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Premisecitation

    Sefelin et al. compared paper sketches to computer-based prototypes of identical breadth and level of refinement, and the results showed no major differences in obtained feedback depending on prototype media.

    “These prototypes were used for usability testing, and the results showed no major differences in the obtained feedback depending on prototype media.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sefelin et al. 2003 source
  • Probably not a substantial claim.

    Interpretationreasoning

    The lack of support for Quality Improvement in the case study is likely due to the nature of early startups, rather than lack of relevance in general of this purpose.

    “The lack of support for this prototyping option in our case study is likely due to the nature of early startups, rather than lack of relevance in general of this purpose.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given

    The exploration strategy determines which instances to pursue, how resources and decisions are organised, and how uncertainties are managed.

    “The exploration strategy determines which instances to pursue, how resources and decisions are organised, and how uncertainties are managed.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationown data

    A Review approach dimension is added with options scenario-based or free, where free covers both free testing, e.g. for beta releases, and demos without any clear scenarios.

    “Thus, we added Review approach as a dimension with the options scenario-based or free, where free, covers both free testing, e.g. for beta releases, and demos without any clear scenarios.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given

    Our terminology is more fine-grained and provides a more precise way of categorising prototypes by their breadth and degree of refinement with respect to different facets.

    “Our terminology is more fine-grained and provides a more precise way of categorising prototypes by their breadth and degree of refinement w.r.t.different facets.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given

    Other reasons for only testing in meeting settings are likely cost and awareness of when testing in a live environment is important.

    “Other reasons for this, are likely cost and awareness of when testing in a live environment is important.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Interpretationnone given

    This aspect concerns strategies used to traverse the solution space over time.

    “This aspect concerns strategies used to traverse the solution space over time.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationown data

    The prototype media aspect is reintroduced in the model because the multi-case study indicates that the choice of prototype media affects the costs and benefits of prototyping.

    “However, we reintroduce this aspect in our model since the multi-case study indicates that the choice of prototype media affects the costs and benefits of prototyping.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given

    Even a video or a physical model is included in our definition of a prototype, and can be used to validate product-market fit, or to communicate with stakeholders.

    “Even a video or a physical model is included in our definition of a prototype, and can be used to, e.g., validate product-market fit, or to communicate with stakeholders.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    A prototype can be constructed using a range of techniques and media, including sketching on paper or in PowerPoint, using a prototyping tool to produce a wireframe or a mock-up, or using an early version of product software as your prototype.

    “A prototype can be constructed using a range of techniques and media, including sketching on paper (Hendry et al. 2005) or in PowerPoint, using a prototyping tool to produce a wireframe or a mock-up (Liu and Khooshabeh 2003), or using an early version of product software as your prototype (McCurdy et al. 2006;Sefelin et al. 2003).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Hendry et al. 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Liu and Khooshabeh 2003 source
    • cites McCurdy et al. 2006 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Jaskiewicz et al. found that while longer iterations promote focusing on certain aspects, multiple short iterations can lead to a more diverse but superficial set of solution options.

    “Jaskiewicz et al. found that while longer iterations promote focusing on certain aspects, multiple short iterations can lead to a more diverse but superficial set of solution options (Jaskiewicz and Helm 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jaskiewicz and Helm 2018 source
  • Probably not a substantial claim.

    Interpretationreasoning

    Problem-Solution fit and Product-Market fit more clearly separate prototyping to ensure the product matches user and customer needs by addressing their problem from prototyping to validate that customers are willing to pay and thus a viable market and business opportunity exists.

    “This will more clearly separate between prototyping to ensure that the product matches the needs of users and customers by provides a solution that addressing their problem, and prototyping to validate that customers are willing to pay for your product and thus the existence of a viable market and business opportunities.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Sefelin et al. suggest selecting prototype media based on factors including the competence of those constructing the prototype, the abilities of the available prototyping tools, and the envisioned continuation of the prototyping.

    “While paper sketches are often cheaper and faster to construct, Sefelin et al. suggest selecting prototype media based on a number of factors, including the competence of those constructing the prototype, the abilities of the available prototyping tools, and the envisioned continuation of the prototyping.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sefelin et al. 2003 source
  • It may state two separate claims as one.

    Premisecitation

    The cost of producing prototypes, the gains and types of learning that can be obtained from these varies with the media.

    “While there is a correspondence between different kinds of prototype media and the scope of these, the cost of producing prototypes, the gains and types of learning that can be obtain from these varies with the media (McCurdy et al. 2006;Zainuddin and Liu 2012).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites McCurdy et al. 2006 source
    • cites Zainuddin and Liu 2012 source
  • Probably not a substantial claim.

    Interpretationown data

    The prototype scope aspect is revised to include Breadth as a stand-alone dimension, alongside functionality, visual appearance, interactive & haptic behaviour, and data realism.

    “For this reason, we revise this aspect to include Breadth as a stand-alone dimension, alongside the dimensions that can be refined, i.e. functionality, visual appearance, interactive & haptic behaviour, and data realism.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Hendry et al. concluded that using a prototype in the streets enabled them to reach otherwise inaccessible user segments.

    “Hendry et al. performed usability testing among home-less people and concluded that using a prototype in the streets enabled them to reach otherwise inaccessible user segments (Hendry et al. 2005).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hendry et al. 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Paper sketches provide more flexibility for designers in early stages, but require more effort to use with larger sets of users.

    “They found that while paper sketches provide more flexibility for designers in early stages, this kind of prototype media requires more effort to use with larger sets of users, e.g., to manually simulate interaction.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zainuddin and Liu 2012 source
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.

    Interpretationnone given

    Our model can support informed decisions of which prototype scope is required to meet the desired purposes of prototyping.

    “We believe that our model can support informed decisions of which prototype scope that is required to meet the desired purposes of prototyping.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationnone given

    This aspect concerns how a prototype is used to achieve a purpose and covers who the reviewers are, if direct prototype interact is used, what review approach is used, and in what environment the prototype is presented and reviewed.

    “This aspect concerns how a prototype is used to achieve a purpose and covers who the reviewers are (internal, external, or with family-friends-and-foes FFF), if direct prototype interact is used (or not), what review approach that is used (scenario based or free), and in what environment the prototype is presented and reviewed.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim.

    Interpretationown data

    The exploration strategy aspect is modified to include three facets: single or parallel exploration, iteration focus (business, product, feature or optimisation level), and iteration size.

    “Thus, we have modified the aspect of exploration strategy to include three facets: single or parallel exploration, iteration focus (business, product, feature or optimisation level), and iteration size.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationown data

    The Purpose aspect is adjusted to highlight that communication includes sales and marketing, changing Communication & Alignment to Communication: Sales, Alignment.

    “For these reasons, we adjust the detailing of the aspect of Purpose in our model to highlight that communication includes sales and marketing, thus Communication & Alignment is changed to Communication: Sales, Alignment.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationreasoning

    Prototyping in startups is primarily used for sales & marketing (Communication), internal exploration of the solution domain, feasibility testing of technical aspects of the solution design, and to validate the product-solution and product-market fit of their business model.

    “Thus, prototyping is primarily used for sales & marketing (Communication), internal exploration of the solution domain, feasibility testing of technical aspects of the solution design, and to validate the product-solution and product-market fit of their business model.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Interpretationreasoning

    Prototyping with an optimisation strategy, and thus with small changes between versions, could be conveyed through giving the Purpose of Quality improvement.

    “Prototyping with an optimisation strategy, and thus with small changes between versions, could be conveyed through giving the Purpose of Quality improvement.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    The model was presented to practitioners in the interviews and used in the analysis.

    “The model was presented to practitioners in the interviews and used in the analysis of”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    The usage environment for a prototype can affect the outcome both in the possible feedback and the representation of subjects in the review.

    “The usage environment for a prototype can affect the outcome (Grevet and Gilbert 2015;Lichter et al. 1994;Tronvoll et al. 2017) both in the possible feedback and the representation of subjects in the review.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Grevet and Gilbert 2015 source
    • cites Lichter et al. 1994 source
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Computer-based prototypes tended to yield more comments on graphical details, while paper sketches stimulated participants to a greater willingness to draw their suggestions.

    “However, the computerbased prototypes tended to yield more comments on graphical details, while the paper sketches stimulated participant to”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sefelin et al. 2003 source
  • Probably not a substantial claim.

    Interpretationreasoning

    Market viability and Business viability are replaced with Problem-Solution fit and Product-Market fit.

    “Instead, we replace these with the terms Problem-Solution fit and Product-Market fit (Osterwalder et al. 2014).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Osterwalder et al. 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may be too vague to check against evidence.

    Interpretationreasoning

    The revised scope model provides a clearer and more easily comprehensible model for describing the scope of a prototype.

    “We believe this provides a clearer and more easily comprehensible model for describing the scope of a prototype.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Premisecitation

    Within one iteration, several concurrent solution paths can be pursued in parallel, which can stimulate innovation and avoid imposing unnecessary limitations, but also cause contradictions between prototypes and complicate decision making.

    “Within one iteration, several concurrent solution paths can be pursued in parallel, which can stimulate innovation (Dow et al. 2011) and avoid imposing unnecessary limitations (Budde and Zullighoven 1990), but also cause contradictions between prototypes and complicate decision making (Jaskiewicz and Helm 2018).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Dow et al. 2011 source
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Jaskiewicz and Helm 2018 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Interactive computer-based mock-ups yielded more feedback when used for user testing.

    “Furthermore, Liu and Khooshabeh also found that interactive computer-based mock-ups yielded more feedback when used for user testing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zainuddin and Liu 2012 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The choice of media may affect the ease with which the prototype can be evaluated in some environments.

    “The choice of media may also affect the ease with which the prototype can be evaluated in some environments (Hendry et al. 2005).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hendry et al. 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim.

    Interpretationcitation

    We model the exploration strategy aspect using the three facets: single vs parallel exploration, iteration focus, and iteration size.

    “We model this aspect using the three facets: single vs parallel exploration, iteration focus, and iteration size These facets are partly based on the four strategies identified by Tronvoll et al., namely point-based, parallel (or set-based solution arrays), optimisation (or performance set investigations), and flexible exploration (Tronvoll et al. 2017).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Hendry et al. found that paper sketches were a good media for eliciting and validating requirements in actual environments and with some stakeholder groups.

    “Finally, Hendry et al. found that paper sketches were a good media for eliciting and validating requirements in actual environments and with some stakeholder groups, such as in public locations where people of all age categories can be accessed (Hendry et al. 2005).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hendry et al. 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Bellomo et al. describe how a company consciously selects the breadth and degree of refinement of prototype scope to match the prototyping purpose, and to validate the concept in focus.

    “Similarly, Bellomo et al. describe how a company consciously selects the breadth and degree of refinement of prototype scope to match the prototyping purpose, and to validate the concept in focus (Bellomo et al. 2013).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bellomo et al. 2013 source
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    The model was initially designed based on previous literature, then validated and adjusted through the empirical data of the focus group and the multi-case study.

    “The model was initially designed based on previous literature, then validated and adjusted through the empirical data of the focus group and the multi-case study.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationreasoning

    As a startup and their solution approach matures, they are more likely to want to explore more fine-grained variations, e.g. in what features or what user-interface design to implement, in which case point-based exploration, or even optimisation exploration is more relevant.

    “As a startup and their solution approach matures, they are more likely to want to explore more fine-grained variations, e.g. in what features or what user-interface design to implement, in which case point-based exploration, or even optimisation exploration is more relevant.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationnone given

    In startups, the main focus is on establishing a viable business model with a product solution that matches customer needs and problems, and developing initial product version to realise their solution ideas.

    “In startups, the main focus is on establishing a viable business model with a product solution that matches customer needs and problems, and developing initial product version to realise their solution ideas.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationreasoning

    In early stages of a startup, prototyping could be used for broad exploration of a suitable solution approach, in which case flexible exploration appears suitable.

    “For example, in the early stages of a startup prototyping could be used for broad exploration of a suitable solution approach, in which case flexible exploration appears suitable.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationreasoning

    We suggest that the digital environment also may play an important role here and is a facet to consider for use of a prototype.

    “We suggest that the digital environment also may play an important role here and is a facet to consider for use of a prototype.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Interpretationnone given

    A high-quality mock-up that covers all the features of a future system with toy examples has a broad prototype scope covering all major features but with a low degree of functional refinement, high visual appearance and interactive behaviour refinement, and low realistic data refinement.

    “For example, a high-quality mock-up that covers all the features of a future system with toy examples, has a broad prototype scope covering all major features but with a low degree of functional refinement of these, but with a high degree of refinement for their visual appearance and interactive behaviour, while it provides a low degree of refinement for realistic data (only toy examples).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductnone given

    The modified aspects (Prototype media, Prototype use, and Exploration strategy) were used in our analysis of the interview material but were not presented to the interviewees.

    “While these modified aspects were used in our analysis of the interview material, they were not presented to the interviewees.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationreasoning

    Choosing source code as the prototype media enables quickly getting started with actual development, but this also comes with risks related to product quality and to cementing solution ideas too early on.

    “While choosing source code as the prototype media enables quickly getting started with actual development, this also comes with risks related to product quality and to cementing solution ideas too early on.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationreasoning

    The lack of evidence for exploration strategy and quality improvement in our case studies may be due to contextual difference in how the practice of prototyping is used, but may also be an indication of areas for which increased insight may support practitioners in improving their prototyping practices.

    “The lack of evidence for these in our case studies may be due to contextual difference in how the practice of prototyping is used, but may also be an indication of areas for which increased insight may support practitioners in improving their prototyping practices.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Simpler and cheaper media such as paper or interviewing, or computer-based mock-ups can be used with good benefits, especially in the early stages of ideation and product design and development, such as in software startups.

    “Simpler and cheaper media such as paper or interviewing, or computer-based mock-ups can be used with good benefits, especially in the early stages of ideation and product design and development, such as in software startups (Nguyen-Duc et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nguyen-Duc et al. 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    For startups, prototypes play a vital role in sales & marketing and in obtaining funding for their venture.

    “In addition, for startups prototypes play a vital role in sales & marketing and in obtaining funding for their venture (Bjarnason 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bjarnason 2021 source
  • It may state two separate claims as one.

    Interpretationnone given

    The kind of prototype media used affects the cost of constructing it, but also the learnings and benefits that can be obtained from using it.

    “The kind of prototype media used affects the cost of constructing it, but also the learnings and benefits that can obtained from using it.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductnone given

    There is a risk that we have missed relevant previous research and that there are additional primary studies relevant to the topic of our systematic mapping study.

    “While we believe our model provides a good representation of the existing research on prototyping methodology, there is a risk that we have missed relevant previous research and that there are additional primary studies relevant to the topic of our systematic mapping study.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationnone given

    The prototype scope affects the cost of prototyping and the type of feedback that can be obtained.

    “The prototype scope also affects the cost of prototyping and the type of feedback that can be obtained.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Interpretationown data

    Since data richness is also the facet with the least number of supporting references in our mapping study, this is interpreted as an indication that further knowledge and research on the role of data richness in prototyping is needed.

    “Since it is also the facet with the least number of supporting references in our mapping study, we interpret this as an indication that further knowledge and research on the role of data richness in prototyping is needed.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductnone given

    The aspect of prototype media was not included in the model at this stage of the study and thus not covered in the focus group.

    “The aspect of prototype media was not included in the model at this stage of our study and thus not covered in the focus group.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Previous research identifies four main strategies, namely point-based, parallel, optimisation and flexible exploration.

    “Previous research identifies four main strategies, namely point-based, parallel, optimisation and flexible exploration (Tronvoll et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim.

    Interpretationnone given

    The breadth of a prototype and its degree of refinement for functionality, visual appearance, interactive behaviour, and data realism can be varied.

    “The breadth of a prototype and its degree of refinement for functionality, visual appearance, interactive behaviour, and data realism, can be varied.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    This corresponds to previous research that found that parallel exploration stimulates innovation.

    “This corresponds to previous research that found that parallel exploration stimulates innovation (Dow et al. 2011).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dow et al. 2011 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Interpretationreasoning

    Throw-away prototyping is often advocated since companies are then forced to separate between identifying the 'right' requirements (through prototyping) and implementing the requirements 'right'.

    “For these reasons, throw-away prototyping is often advocated since companies are then forced to separate between identifying the 'right' requirements (through prototyping) and implementing the requirements 'right'.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationnone given

    Compared to previous literature, our model also distinguishes between the type of facet that is refined, e.g. visuals or functionality, and provides a more fine-grained terminology for describing the scope of a prototype.

    “Compared to previous literature, our model also distinguishes between the type of facet that is refined, e.g.visuals or functionality, and provides a more fine-grained terminology for describing the scope of a prototype.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    Prototyping provides a powerful tool for communicating with customers and for creating a good communication climate within a development team by showing rather than just telling.

    “Prototyping also provides a powerful tool for communicating with customers and for creating a good communication climate within a development team (Dow et al. 2011) by showing rather than just telling.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dow et al. 2011 source
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

    Study design and conductreasoning

    Generalizability of our model beyond our case companies and the area of RE is believed to be medium.

    “Generalizability of our model beyond our case companies and the area of RE is believed to be medium.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    There is a risk that prototypes can convey an inaccurate perception of development status and create unrealistic expectations.

    “However, there is also a risk that prototypes can convey an inaccurate perception of development status and create unrealistic expectations (Lichter et al. 1994).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lichter et al. 1994 source
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationnone given

    At the heart of prototyping lies exploration of the solution space through experimenting with ideas, gathering feedback, and iteratively detailing, validating, and communicating product requirements.

    “At the heart of prototyping, lies exploration of the solution space through experimenting with ideas, gathering feedback, and iteratively detailing, validating, and communicating product requirements.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationnone given

    While parallel exploration requires more resources, it also allows for delaying decisions on alternative requirement options until more knowledge has been obtained for these.

    “While this approach requires more resources, it also allows for delaying decisions on alternative requirement options until more knowledge has been obtained for these.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Study design and conductreasoning

    We judge that descriptive and theoretical validity are high for our study.

    “We judge that both these aspects are high for our study.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductown data

    The results from the literature study and the focus group were presented at the case company, who also reviewed an earlier version of this paper describing the first version of our model without raising any concerns about misinterpretation.

    “In addition, the results from the literature study and the focus group were presented at the case company, who also reviewed an earlier version of this paper describing the first version of our model without raising any concerns about misinterpretation.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationnone given

    The strategy used to explore the solution space affects which prototyping instances are pursued and how resources are utilised.

    “The strategy used to explore the solution space affects which prototyping instances that are pursued and how resources are utilised.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Studies report contradicting results on the relationship between prototype scope and the feedback that can be obtained, and thus the purpose that can be fulfilled.

    “Furthermore, we note that studies report contradicting results on the relationship between prototype scope and the feedback that can be obtained, and thus the purpose that can be fulfilled.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Sefelin et al. 2003 source
    • cites Liu and Khooshabeh 2003 source
    • cites McCurdy et al. 2006 source
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductreasoning

    There is a risk that our model is not complete and that there are additional aspects of prototyping, and in particular additional facets, that should be included in our model.

    “Thus, there is a risk that our model is not complete and that there are additional aspects of prototyping, and in particular additional facets, that should be included in our model.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductown data

    The risk of the single first researcher mis-interpreting the interview material was partly mitigated by asking the interviewees to read through the summarising memos with selected quotes from the material and the set of identified prototyping instances.

    “the interview material was partly mitigated by asking the interviewees to read through the summarising memos with selected quotes from the material and the set of identified prototyping instances.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Interpretationnone given

    Prototyping can also support exploring the problem at hand.

    “Prototyping can also support exploring the problem at hand.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationcitation

    In both prototyping in small optimising iterations and managing quality requirements, the outcome focuses on evaluating against a range of values, as opposed to a simple pass/fail, and the increased complexity of needing to consider influencing factors.

    “In both cases, the outcome focuses on evaluating against a range of values, as opposed to a simple pass/fail, and the increased complexity of needing to considering influencing factors (Tronvoll et al. 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tronvoll et al. 2017 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationnone given

    The purpose of prototyping can vary and may consist of a combination of reasons.

    “The purpose of prototyping can vary and may consist of a combination of reasons.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Within agile, prototyping is a known RE practice where requirements are gradually defined, validated, and communicated through prototyping as part of the incremental development process.

    “Within agile, prototyping is a known RE practice where requirements are gradually defined, validated, and communicated through prototyping as part of the incremental development process (Ramesh et al. 2010).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ramesh et al. 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim.

    Interpretationreasoning

    This set of facets can also be used to describe all four strategies included in the first version of the model.

    “This set of facets can also be used to describe all four strategies included in the first version of the model.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The two dimensions of breadth and refinement have been described in previous literature using the terms horizontal versus vertical prototyping, where horizontal prototyping explores the breadth and vertical prototyping the refinement of the scope of the software being developed.

    “The two dimensions of breadth and refinement (in general) have been described in previous literature using the terms horizontal versus vertical prototyping (Budde and Zullighoven 1990), where horizontal prototyping explores the breadth and a vertical prototyping the refinement of the scope of the software being development.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Budde and Zullighoven 1990

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    In our previous research on digital work environments, we have identified systems interplay and work interplay as two important factors to consider in RE for systems and tools intended to be used in the work place, i.e. the interaction with other systems and with current work practices.

    “In our previous research on digital work environments, we have identified systems interplay and work interplay as two important factors to consider in RE for systems and tools intended to be used in the work place, i.e. the interaction with other systems and with current work practices (Håkansson and Bjarnason 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Håkansson and Bjarnason 2020 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Previous research identifies a connection between the length of an iteration and the number and quality of solution options, or requirement possibilities, that are obtained.

    “We note that previous research identifies a connection between the length of an iteration and the number and quality of solution options, or requirement possibilities, that are obtained (Jaskiewicz and Helm 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jaskiewicz and Helm 2018 source
  • Probably not a substantial claim.

    Interpretationreasoning

    The model is modified to cover single or parallel exploration where multiple solution paths are explored at once, the size and the focus of the iteration, e.g. business, product, feature, or optimisation level.

    “For that reason, we modify our model to cover single or parallel exploration where multiple solution paths are explored at once, the size and the focus of the iteration, e.g.business, product, feature, or optimisation level.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    There is a parallel between prototyping in small optimising iterations and how to manage quality (non-functional) requirements.

    “Furthermore, we note a parallel between prototyping in small optimising iterations with how to manage quality (non-functional) requirements (Berntsson Svensson and Regnell 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Berntsson Svensson and Regnell 2015

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.

    Interpretationreasoning

    These conflicting findings indicate that the matter is complex, and that further empirical research is needed to identify the relevant factors of prototyping practice and the environment, and the relations between these.

    “These conflicting findings indicate that the matter is complex, and that further empirical research is needed to identify the relevant factors of prototyping practice and the environment, and the relations between these.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductreasoning

    There is a risk that our model does not fully align with current prototyping practices in industry, even though we have taken initial steps to explore and validate this.

    “Furthermore, there is also a risk that our model does not fully align with current prototyping practices in industry, even though we have taken initial steps to explore and validate this.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationnone given

    With a prototype, new requirements can be elicited through exploration & learning and validated by testing business viability, market desirability, technical feasibility, or usability.

    “With a prototype, new requirements can be elicited through exploration & learning and validated by testing business viability, market desirability, technical feasibility, or usability.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductnone given

    The primary studies were analysed twice, first as part of the initial design by the 2nd and 3rd author who calibrated their views, and then independently by the 1st author, as part of the initial validation (after the focus group).

    “The primary studies were analysed twice, first as part of the initial design by the 2 nd and 3 rd author who calibrated their views, and then independently by the 1 st author, as part of the initial validation (after the focus group).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    The interactions around prototypes relate to the cognitive abilities and communication skills of those involved and is a complex and interesting area for further research.

    “The interactions around prototypes relate to the cognitive abilities (Cafer and Misra 2009) and communication skills (Ciriello et al. 2017) of those involved and is a complex and interesting area for further research.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Cafer and Misra 2009 source
    • cites Ciriello et al. 2017 source
  • Probably not a substantial claim.

    Study design and conductreasoning

    Several steps were taken to mitigate the risks of misunderstanding and misinterpreting the literature on which our model is based, the focus group participants, and the interviewees.

    “Several steps were taken to mitigate the risks of misunderstanding and misinterpreting the literature on which our model is based, the focus group participants, and the interviewees.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Previous research describes the impact of the physical environment.

    “Previous research describes the impact of the physical environment (Lichter et al. 1994).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lichter et al. 1994 source
248 of 248
  • The paper relies on this without support.

    The five aspects of PAM are sufficient to characterise prototyping practices.

    “The model can be used to categorise prototyping instances and can thereby support practitioners in reflecting and improving on their prototyping practices.”
  • The paper relies on this without support.

    The case companies' prototyping practices are representative enough to validate and improve PAM.

    “We validate and demonstrate the applicability of our model through a focus group at one company and through semistructured interviews at eleven (other) startup companies.”
  • The paper relies on this without support.

    The systematic mapping study's primary studies cover the relevant body of knowledge on prototyping methodology.

    “The mapping study provided a broad base of scientific knowledge that supported our design of the model.”
  • The paper relies on this without support.

    The practice of prototyping can achieve many different purposes that often vary throughout the life cycle of a project.

    “The practice of prototyping can achieve many different purposes that often vary throughout the life cycle of a project.”
  • The paper relies on this without support.

    When prototyping, several purposes may be satisfied simultaneously.

    “When prototyping, several purposes may be satisfied simultaneously, e.g., communicating the product idea to potential customers while also validating its market desirability.”
  • The paper relies on this without support.

    Product-market fit is a premise for business viability.

    “Product-market fit is a premise for business viability.”
  • The paper relies on this without support.

    The choice of reviewers affects the learnings that can be obtained from prototyping.

    “The choice of reviewers affects the learnings that can be obtained from prototyping.”
  • The paper relies on this without support.

    Purely internal use of a prototype can support brainstorming and allow designers and engineers to generate and organize ideas.

    “Purely internal use of a prototype can support brainstorming and allow designers and engineers to generate and organize ideas.”
  • The paper relies on this without support.

    Using a prototype with external parties often increases the cost due to increased expectations and demands on quality.

    “Using a prototype with external parties often increases the cost due to increased expectations and demands on quality.”
  • The paper relies on this without support.

    There is usually no direct reviewer interaction with the prototype during demonstrations.

    “There is usually no direct reviewer interaction with the prototype during demonstrations.”
  • The paper relies on this without support.

    The needs and goals may change for each iteration, which can enable focusing on specific product aspects or parts.

    “The needs and goals may change for each iteration, which can enable focusing on specific product aspects or parts.”
  • The paper relies on this without support.

    If the solution appears unsuitable it may be necessary to discard the prototype and redo the process.

    “If the solution appears unsuitable it may be necessary to discard the prototype and redo the process.”
  • The paper relies on this without support.

    Solution options are judged by performance, which affects the evaluation.

    “Solution options are judged by performance, which affects the evaluation.”
  • The paper relies on this without support.

    The interview data accurately reflect the prototyping practices of the startups.

    “the interview data to categorize the prototyping instances described by our interviewees.”
  • The paper relies on this without support.

    The systematic mapping study's set of primary studies is comprehensive enough to represent the current body of knowledge on prototyping methodology.

    “The systematic map on which our model is based consists of thirty-three primary studies and represents a wide set of papers from the past thirty years from the areas of human factors and design, requirements engineering, and agile software development (RQ2).”
15 of 15

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Interviewed practitioner count differs

    Minor

    The introduction reports interviews with twelve practitioners, but the case-company section reports thirteen practitioners. Twelve is elsewhere given as the number of interviews, with one interview involving two founders.

  • Startup company count differs

    Minor

    The model-refinement section says the multi-case study involved ten startup companies, while the methods section says it involved eleven.

2 of 2

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. BerntssonSvensson R, Regnell B (2015) A Case Study Evaluation of the Guideline-Supported QUPER Model for Elicitation of Quality Requirements. REFSQ 15:230–246
  2. Block, Z. and MacMillan, I.C. (1985) Milestones for Successful Venture Planning. Harvard Business Review, Sept 1985.
  3. Rex Hartson (2019). The UX Book. Morgan Kaufmann Publishers
  4. Nguyen-Duc A, Wang X, Abrahamsson P (2017) What Influences the Speed of Prototyping? An Empirical Investigation of Twenty Software Startups. Agile Proc in Softw Eng 2017:20–36
  5. Tullis, T.S. (1990). High-fidelity prototyping throughout the design process. Proceedings of the Human Factors Sciety 34th Annual Meeting 266

Cited by

Sorted by citations

Metadata

Authors
Bjarnason, E., Lang, F., & Mjöberg, A.
Affiliations
Lund University (SE)
Corresponding author
Bjarnason, Elizabeth
Venue
Empirical Software Engineering
Published
2023-08-30
Volume
28
Issue
5
ISSN
  • 1382-3256 (ISSN-L)
  • 1573-7616
OpenAlex
W4386292207
Cited by
75
References
57
Citations by year
2026: 45 · 2025: 25 · 2024: 5
FWCI
13.79
Type
journal article
Language
en
Open access
Locations
  • Empirical Software Engineering (publishedVersion, cc-by)
  • Lund University Publications (Lund University) (submittedVersion)
Funders
Lund University
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
Keywords
software prototyping, requirements engineering, agile software development, user interface design, exploration strategy, startups

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review