Context Matters: Volunteer Bias, Small Sample Size, and the Value of Comparison Groups in the Assessment of Research-Based Undergraduate Introductory Biology Lab Courses
Open accessAbstract
The shift from cookbook to authentic research-based lab courses in undergraduate biology necessitates the need for evaluation and assessment of these novel courses. Although the biology education community has made progress in this area, it is important that we interpret the effectiveness of these courses with caution and remain mindful of inherent limitations to our study designs that may impact internal and external validity. The specific context of a research study can have a dramatic impact on the conclusions. We present a case study of our own three-year investigation of the impact of a research-based introductory lab course, highlighting how volunteer students, a lack of a comparison group, and small sample sizes can be limitations of a study design that can affect the interpretation of the effectiveness of a course.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There may be a volunteer effect leading to some GPA gains because most programs compared students who volunteered to participate with students who did not volunteer.
“It is also important to note that students volunteered to participate in all of these programs, so there may be a volunteer effect that is leading to some of these gains (Brownell et al., 2013), because most programs compared students who volunteered to participate in the program with students who did not volunteer to participate.”
From Building Better Bridges into STEM: A Synthesis of 25 Years of Literature on STEM Summer Bridge Programs (2017) · cites this paper as Brownell et al., 2013
The Introduction’s case study shows that comparing volunteers with non-volunteers can change conclusions about gains in research interest and confidence in lab tasks. It mentions self-reported GPA only as a baseline characteristic, not as an outcome, and does not establish that most of the citing document’s programs used volunteer-versus-non-volunteer comparisons. (LLM, conf=0.87)
Experimental — accuracy not yet measuredAny comparison of bridge program participants with nonvolunteers should be viewed with caution, as there are likely motivation differences among these students.
“any comparison of the participants with nonvolunteers should be viewed with caution, as there are likely motivation differences among these students (Brownell et al., 2013)”
From Building Better Bridges into STEM: A Synthesis of 25 Years of Literature on STEM Summer Bridge Programs (2017) · cites this paper as Brownell et al., 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 22
- [3]Aronson BD Silveira LA 2009 From genes to proteins to behavior: a laboratory project that enhances student understanding in cell and molecular biology CBE Life Sci Educ 8 291 308 10.1187/cbe.09-07-0048 19952098 PMC2786280 · DOI
- [4]Baumler DJ 2012 Using comparative genomics for inquiry-based learning to dissect virulence of Escherichia coli O157: H7 and Yersinia pestis CBE Life Sci Educ 11 81 93 10.1187/cbe.10-04-0057 22383620 PMC3292067 · DOI
- [5]Brame CJ Pruitt WM Robinson LC 2008 A molecular genetics laboratory course applying bioinformatics and cell biology in the context of original research CBE Life Sci Educ 7 410 421 10.1187/cbe.08-07-0036 19047427 PMC2592051 · DOI
- [9]Casem ML 2006 Student perspectives on curricular change: lessons from an undergraduate lower-division biology core CBE Life Sci Educ 5 65 75 10.1187/cbe.05-06-0084 17012192 PMC1635130 · DOI
- [10]Cox-Paulson EA Grana TM Harris MA Batzli JM 2012 Studying human disease genes in Caenorhabditis elegans: a molecular genetics laboratory project CBE Life Sci Educ 11 165 179 10.1187/cbe-11-06-0045 22665589 PMC3366902 · DOI
- [11]Cunningham SC McNear B Pearlman RS Kern SE 2006 Beverage-agarose gel electrophoresis: an inquiry-based laboratory exercise with virtual adaptation CBE Life Sci Educ 5 281 286 10.1187/cbe.06-01-0139 17012220 PMC1618695 · DOI
- [12]Derting TL Williams KS Momsen JL Henkel TP 2011 Education research: set a high bar Science 2 1220 1221 10.1126/science.333.6047.1220-c 21885755 · DOI
- [13]Derting TL Ebert-May D 2010 Learner-centered inquiry in undergraduate biology: positive relationships with long-term student achievement CBE Life Sci Educ 9 462 472 10.1187/cbe.10-02-0011 21123693 PMC2995764 · DOI
- [14]DiBartolomeis SM 2011 A semester-long project for teaching basic techniques in molecular biology such as restriction fragment length polymorphism analysis to undergraduate and graduate students CBE Life Sci Educ 10 95 110 10.1187/cbe.10-07-0098 21364104 PMC3046893 · DOI
- [15]Feldman KA 1986 The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: a review and synthesis Res Higher Educ 24 139 213 10.1007/BF00991885 · DOI
- [16]Fukami T 2013 Integrating inquiry-based teaching with faculty research Science 339 1536 1537 10.1126/science.1229850 23539590 · DOI
- [17]Gess-Newsome J 2002 Pedagogical content knowledge: an introduction and orientation 3 17 Examining pedagogical content knowledge Springer Netherlands
- [18]Gormally C Brickman P Lutz M 2012 Developing a test of scientific literacy skills (TOSLS): measuring undergraduates’ evaluation of scientific information and arguments CBE Life Sci Educ 11 364 377 10.1187/cbe.12-03-0026 23222832 PMC3516792 · DOI · Paper page
- [21]Kloser MJ Brownell SE Chiariello NR Fukami T 2011 Integrating teaching and research in undergraduate biology laboratory education PLoS biology 9 11 e1001174 10.1371/journal.pbio.1001174 22110400 PMC3216991 · DOI
- [23]Marczyk G DeMatteo D Festinger D 2005 Essentials of research design and methodology 65 94 John Wiley & Sons Hoboken, NJ
- [25]Morine-Dershimer G Kent T 2002 The complex nature and sources of teachers’ pedagogical knowledge 21 50 Examining pedagogical content knowledge Springer Netherlands
- [27]Rosenthal R Rosnow RL 1975 The volunteer subject 121 133 John Wiley & Sons New York
- [28]Ruiz-Primo MA Briggs D Iverson H Talbot R Shepard LA 2011 Impact of undergraduate science course innovations on learning Science 331 1269 1270 10.1126/science.1198976 21393529 · DOI
- [29]Rushton S Morgan J Richard M 2007 Teacher’s Myers-Briggs personality profiles: identifying effective teacher personality traits Teaching and Teacher Education 23 432 441 10.1016/j.tate.2006.12.011 · DOI
- [31]Shavelson RJ Towne L 2002 Scientific research in education 80 126 The National Academies Press Washington, DC
- [34]Sundberg MD Armstrong JE Wischusen EW 2005 A reappraisal of the status of introductory biology laboratory education in US colleges & universities Am Biol Teach 67 525 529 10.1662/0002-7685(2005)067[0525:AROTSO]2.0.CO;2 · DOI
- [35]Tanner K Allen D 2004 Approaches to biology teaching and learning: learning styles and the problem of instructional selection—engaging all students in science courses Cell Biol Educ 3 197 201 10.1187/cbe.04-07-0050 15592590 PMC533116 · DOI
Cited by
Sorted by citations
Getting Under the Hood: How and for Whom Does Increasing Course Structure Work?
Eddy, S. L., & Hogan, K. A. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.14-03-0050 · Cited by 553 · see citing works
“…across contexts cannot be assumed, as there is some evidence that the success of classroom interventions depends on the student populations in the classroom (Brownell et al., 2013), instructor classroom management style (Borrego et al., 2013), and the topics being taught (Andrews et al., 2011).”
BackgroundBrownell, S. E., Hekmat-Scafe, D. S., Singla, V., Chandler Seawell, P., Conklin Imam, J. F., Eddy, S. L., Stearns, T., & Cyert, M. S. (2015) · CBE—Life Sciences Education · doi:10.1187/cbe.14-05-0092 · Cited by 287 · see citing works
Self-citationAshley, M., Cooper, K. M., Cala, J. M., & Brownell, S. E. (2017) · CBE—Life Sciences Education · doi:10.1187/cbe.17-05-0085 · Cited by 122 · see citing works
Self-citationTheobald, E. (2018) · CBE—Life Sciences Education · doi:10.1187/cbe.17-12-0280 · Cited by 114 · see citing works
- Authors
- Brownell, S. E., Kloser, M. J., Fukami, T., & Shavelson, R. J.
- Affiliations
- Arizona State University (US)
- University of Notre Dame (US)
- Stanford University (US)
- Corresponding author
- Brownell, Sara E.
- Venue
- Journal of Microbiology & Biology Education
- Publisher
- American Society for Microbiology
- Published
- 2013-01
- Volume
- 14
- Issue
- 2
- Pages
- 176-182
- ISSN
- 1935-7877 (ISSN-L)
- 1935-7885
- PMC
- PMC3867754
- OpenAlex
- W2099950290
- Cited by
- 78
- References
- 0
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 8 · 2023: 4 · 2022: 10 · 2021: 8 · 2020: 4 · 2019: 6 · 2018: 8 · 2017: 6 · 2016: 8 · 2015: 4 · 2014: 4
- FWCI
- 24.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Microbiology and Biology Education (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Arizona State University Library Digital Repository (Arizona State University) (submittedVersion, cc-by-nc-sa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (0941984)
- Topics
- Social Sciences › Education › Science Education and Pedagogy
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Social Sciences › Education › Educational Research and Methods
- Keywords
- course evaluation, volunteer bias, comparison groups, small sample size, study design limitations, internal validity, generalizability, course effectiveness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
