Older Adults and Crowdsourcing: Android TV App for Evaluating TEDx Subtitle Quality
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
In this paper we describe the insights from an exploratory qualitative pilot study testing the feasibility of a solution that would encourage older adults to participate in online crowdsourcing tasks in a non-computer scenario. Therefore, we developed an Android TV application using Amara API to retrieve subtitles for TEDx talks which allows the participants to detect and categorize errors to support the quality of the translation and transcription processes. It relies on the older adults' innate skills as long-time native language users and the motivating factors of this socially and personally beneficial task. The study allowed us to verify the underlying concept of using Smart TVs as interfaces for crowdsourcing, as well as possible barriers, including the interface, configuration issues, topics and the process itself. We have also assessed the older adults' interaction and engagement with this TV-enabled online crowdsourcing task and we are convinced that the design of our setup addresses some key barriers to crowdsourcing by older adults. It also validates avenues for further research in this area focused on such considerations as autonomy and freedom of choice, familiarity, physical and cognitive comfort as well as building confidence and the edutainment value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Positive studies, are still rare, and one shared feature is that they use older adults' strengths, such as their insights from life experience [5] or language proficiency [11] which increase with age.”
From Older Adults and Voice Interaction: A Pilot Study with Google Home (2019) · cites this paper as [11]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599 · cited 1×
“mification is a more psychological than technological issue [63]. Many methods and gamification tools are based on sound psychological foundations, like the well-established self-determination theory [50], which was verified in the context of older adults [54]. Although there are some promising reports on using gamifi…”
BackgroundSituated Learning: Legitimate Peripheral Participation.
Bloch, M., Lave, J., & Wenger, E. (1994) · Man · doi:10.2307/2804509 · Cited by 39,454
Situated Learning: Legitimate Peripheral Participation
Lave, J., & Wenger, E. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815355 · Cited by 35,324 · cited 1×
“mber 2018. Older Adults and Crowdsourcing: Android TV App for Evaluating TEDx Subtitle Quality 159:5 The TED Translators initiative is a true example of a community of practice which Leave and Wenger [36] describe as a group driven by common interest that collectively improves and gains knowledge. In 2014, Cámara, in…”
BackgroundFrom game design elements to gamefulness: defining "gamification"
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011) · Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments · doi:10.1145/2181037.2181040 · Cited by 8,402 · cited 1×
“Volunteering and user engagement are often considered in terms of using gamification techniques, i.e. the use of game design elements in non-game contexts [18].”
BackgroundGamification by Design: Implementing Game Mechanics in Web and Mobile Apps
Zichermann, G., & Cunningham, C. J. L. (2011) · Cited by 2,378
Czaja, S. J., Charness, N., Fisk, A. D., Hertzog, C., Nair, S. N., Rogers, W. A., & Sharit, J. (2006) · Psychology and Aging · doi:10.1037/0882-7974.21.2.333 · Cited by 2,113 · cited 1×
“Solutions Uncomfortable and costly setup [31, 51] TV sets at home [5, 34] Unfamiliar interfaces [7, 16, 17, 45] Text and remote control [23] Repetitive tasks [11] Educational wide-domain TEDx videos [4] Unclear social benefit [11] Improving subtitles for all [12] Unclear personal benefit [11, 56]…”
BackgroundDIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe.
(2013) · Joint Research Centre (European Commission) · doi:10.2788/52966 · Cited by 1,394 · cited 1×
“(e.g. creating content, programming), safety (e.g. using anti-virus or a firewall) and problem solving (e.g. installing new devices, using Internet banking), based on the Digital Competence Framework [22]. 3.3.3 Introductory interview. The semi-structured in depth introductory interview aimed to gauge two main issues.…”
MethodInfluentialKittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034 · cited 2×
“…and setup of online crowdsourcing translation process support based on commodity goods, in this case the Smart TV platform, and to mitigate the barriers that may exist for this type of activity, also related to vital challenges of crowd work systems raised at CSCW in recent years [15, 28, 61, 62].”
Effects of Volunteering on the Well-Being of Older Adults
Morrow-Howell, N., Hinterlong, J., Rozario, P. A., & Tang, F. (2003) · The Journals of Gerontology: Series B · doi:10.1093/geronb/58.3.s137 · Cited by 923 · cited 1×
“ng Encouraging older adults’ engagement in volunteering and crowdsourcing activities in general [39] benefits the society as a whole. Such activities have positive effects on older adults’ well-being [40], as well as their mental and physical health [37]. Thus, they can be regarded as a protective factor for their psy…”
BackgroundFormal Volunteering as a Protective Factor for Older Adults' Psychological Well-Being
Greenfield, E. A., & Marks, N. F. (2004) · The Journals of Gerontology Series B: Psychological Sciences and Social Sciences · doi:10.1093/geronb/59.5.s258 · Cited by 600 · cited 1×
“tivities have positive effects on older adults’ well-being [40], as well as their mental and physical health [37]. Thus, they can be regarded as a protective factor for their psychological well-being [24, 25]. In particular, they can also pose certain health advantages, as staying active and learning new things can de…”
Background
Research works printed but not linked: 5
- Sara J Czaja and Chin Chin Lee. 2009. Information technology and older adults. Human-computer interaction: Designing for diverse users and domains (2009), 18--30.
- Jennifer M Ortman, Victoria A Velkoff, Howard Hogan, et al. 2014. An aging nation: the older population in the United States. Washington, DC: US Census Bureau (2014), 25--1140.
- Pablo Romero-Fresco and Franz Pöchhacker. 2017. Quality assessment in interlingual live subtitling: The NTR Model. 16 (01 2017), 149--167.
- 10.5555/2305316.2305330 · doi:10.5555/2305316.2305330
- 10.5555/2073550 · doi:10.5555/2073550
Cited by
Older Adults and Voice Interaction: A Pilot Study with Google Home
Kowalski, J., Jaskulska, A., Skorupska, K., Abramczuk, K., Biele, C., Kopeć, W., & Marasek, K. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312973 · Cited by 125
Self-citationKou, Z., Shang, L., Zhang, Y., & Wang, D. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3492855 · Cited by 47
How Might Voice Assistants Raise Our Children?
Biele, C., Jaskulska, A., Kopeć, W., Kowalski, J., Skorupska, K., & Zdrodowska, A. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-11051-2_25 · Cited by 46
Self-citationVR with Older Adults: Participatory Design of a Virtual ATM Training Simulation
Kopeć, W., Wichrowski, M., Kalinowski, K., Jaskulska, A., Skorupska, K., Cnotkowski, D., Tyszka, J., Popieluch, A., Voitenkova, A., Masłyk, R., et al. (2019) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2019.12.110 · Cited by 26
Self-citation"Hi! I am the Crowd Tasker" Crowdsourcing through Digital Voice Assistants
Hettiachchi, D., Sarsenbayeva, Z., Allison, F., van Berkel, N., Dingler, T., Marini, G., Kostakos, V., & Goncalves, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376320 · Cited by 22
“In another example, Skorupska et al. [57] show that older adults can contribute to a transcription task while watching a movie.”
BackgroundVerma, N., Shiroma, K., Rich, K., Fleischmann, K. R., Xie, B., & Lee, M. K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-71305-8_32 · Cited by 17
The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines
Oppenlaender, J., Abbas, T., & Gadiraju, U. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641023 · Cited by 17
Older Adults’ Motivation and Engagement with Diverse Crowdsourcing Citizen Science Tasks
Skorupska, K., Jaskulska, A., Masłyk, R., Paluch, J., Nielek, R., & Kopeć, W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_7 · Cited by 11
Self-citationDesigning a Crowdsourcing System for the Elderly: A Gamified Approach to Speech Collection
Seong, E., & Kim, S. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382999 · Cited by 10
Antoine, A., Malacria, S., Marquardt, N., & Casiez, G. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445586 · Cited by 11
Jaskulska, A., Skorupska, K., Karpowicz, B., Biele, C., Kowalski, J., & Kopeć, W. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-74728-2_15 · Cited by 13
Self-citationWang, P., Hu, S., Wen, B., & Lu, Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642959 · Cited by 15
Deinstitutionalizing Independence: Discourses of Disability and Housing in Accessible Computing
Storer, K. M., & Branham, S. M. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471213 · Cited by 11
Skorupska, K., Núñez, M., Kopeć, W., & Nielek, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29387-1_25 · Cited by 4
Self-citationVideogame localisation, spelling errors and player reception
Deckert, M., & Hejduk, K. (2022) · Translation Cognition & Behavior · doi:10.1075/tcb.00064.dec · Cited by 3
Participation in crowdsourcing micro-tasks: what motivates brazilian older adults?
Amorim, A. M., & Vieira, V. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-023-01023-9 · Cited by 3
Kopeć, W., Jaskulska, A., Karpowicz, B., Pochwatko, G., Kornacka, M., & Skorupska, K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37649-8_18 · Cited by 1
Self-citationAsadi, Y. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.232434 · Cited by 2
Biele, C. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-90004-5_11
AFFORCE: Actionable Framework for Designing Crowdsourcing Experiences for Older Adults
Skorupska, K., Nielek, R., & Kopeć, W. (2021) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3486622.3494026
Self-citationSkorupska, K., Makowska-Tłomak, E., & Jaskulska, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-11432-8_16 · Cited by 1
Self-citationGeleneksel İç İçe Yerleşimler ve Kısıt Yerleşimi İçeren Melez Android Ön Yüz Geliştirme Yaklaşımı
Özçevik, Y. (2022) · International Journal of Innovative Engineering Applications · doi:10.46460/ijiea.1116222
Age-Inclusive Design of Video Streaming Services: Integrating Insights and Design Implications
Subramanian, S., & Moor, K. D. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-48306-6_5
Metadata
- Authors
- Skorupska, K., Nunez, M., Kopec, W., & Nielek, R.
- Affiliations
- Polish-Japanese Academy of Information Technology (PL)
- Polish-Japanese Academy of Information Technology, Warsaw, Poland
- ORCID
- Skorupska, Kinga
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2018-11
- Volume
- 2
- Issue
- CSCW
- Pages
- 1-23
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4289325569
- Cited by
- 23
- References
- 57
- Citations by year
- 2024: 2 · 2023: 3 · 2022: 5 · 2021: 7 · 2020: 2 · 2019: 4
- FWCI
- 6.73
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Topics
- Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Psychology › Developmental and Educational Psychology › Aging and Gerontology Research
- Keywords
- older adults, crowdsourcing, TEDx talks, user engagement, barriers to participation, edutainment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.