CoChat

Great Chain of Agents: The Role of Metaphorical Representation of Agents in Conversational Crowdsourcing

Open access
Jung, Ji-Youn, Qiu, Sihang, Bozzon, Alessandro, Gadiraju, Ujwal · 2022 · CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 78 of the 79 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Conversational agents are being widely adopted across several domains to serve a variety of purposes ranging from providing intelligent assistance to companionship. Recent literature has shown that users develop intuitive folk theories and a metaphorical understanding of conversational agents (CAs) due to the lack of a mental model of the agents. However, investigation of metaphorical agent representation in the HCI community has mainly focused on the human level, despite non-human metaphors for agents being prevalent in the real world. We adopted Lakoff and Turner’s ‘Great Chain of Being’ framework to systematically investigate the impact of using non-human metaphors to represent conversational agents on worker engagement in crowdsourcing marketplaces. We designed a text-based conversational agent that assists crowd workers in task execution. Through a between-subjects experimental study (N = 341), we explored how different human and non-human metaphors affect worker engagement, the perceived cognitive load of workers, intrinsic motivation, and their trust in the agents. Our findings bridge the gap of how users experience CAs with non-human metaphors in the context of conversational crowdsourcing.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3491102.3517653

Cites

78 of 79 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 79 references

Research works printed but not linked: 26

  1. Tahir Abbas , Ujwal Gadiraju , Vassilis-Javed Khan , and Panos Markopoulos . 2021 . Making Time Fly: Using Fillers to Improve Perceived Latency in Crowd-Powered Conversational Systems . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. 2–14 . Tahir Abbas, Ujwal Gadiraju, Vassilis-Javed Khan, and Panos Markopoulos. 2021. Making Time Fly: Using Fillers to Improve Perceived Latency in Crowd-Powered Conversational Systems. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. 2–14.
  2. Tahir Abbas , Vassilis-Javed Khan , Ujwal Gadiraju , and Panos Markopoulos . 2020 . Trainbot: A Conversational Interface to Train Crowd Workers for Delivering On-Demand Therapy . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8. 3–12 . Tahir Abbas, Vassilis-Javed Khan, Ujwal Gadiraju, and Panos Markopoulos. 2020. Trainbot: A Conversational Interface to Train Crowd Workers for Delivering On-Demand Therapy. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8. 3–12.
  3. Nancy Baym , Limor Shifman , Christopher Persaud , and Kelly Wagman . 2019 . INTELLIGENT FAILURES: CLIPPY MEMES AND THE LIMITS OF DIGITAL ASSISTANTS. AoIR Selected Papers of Internet Research 2019 (2019). Nancy Baym, Limor Shifman, Christopher Persaud, and Kelly Wagman. 2019. INTELLIGENT FAILURES: CLIPPY MEMES AND THE LIMITS OF DIGITAL ASSISTANTS. AoIR Selected Papers of Internet Research 2019 (2019).
  4. Johanna Böttcher , Tabea Lüttmer , Ulrich Wagner , Frank Asbrock , and Maarten H . W. van Zalk . 2020 . Examining the structural validity of stereotype content measures - A preregistered re-analysis of published data and discussion of psosisble future directions. Vol . 1. 0–2 pages. Johanna Böttcher, Tabea Lüttmer, Ulrich Wagner, Frank Asbrock, and Maarten H. W. van Zalk. 2020. Examining the structural validity of stereotype content measures - A preregistered re-analysis of published data and discussion of psosisble future directions. Vol. 1. 0–2 pages.
  5. Cynthia Breazeal . 2003. Toward sociable robots. Robotics and autonomous systems 42, 3-4 ( 2003 ), 167–175. Cynthia Breazeal. 2003. Toward sociable robots. Robotics and autonomous systems 42, 3-4 (2003), 167–175.
  6. Ewart J De Visser , Samuel S Monfort , Ryan McKendrick , Melissa AB Smith , Patrick E McKnight , Frank Krueger , and Raja Parasuraman . 2016 . Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology : Applied 22 , 3 (2016), 331 . Ewart J De Visser, Samuel S Monfort, Ryan McKendrick, Melissa AB Smith, Patrick E McKnight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology: Applied 22, 3 (2016), 331.
  7. Djellel Eddine Difallah , Michele Catasta , Gianluca Demartini , and Philippe Cudré-Mauroux . 2014 . Scaling-up the crowd: Micro-task pricing schemes for worker retention and latency improvement . In Second AAAI Conference on Human Computation and Crowdsourcing. Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, and Philippe Cudré-Mauroux. 2014. Scaling-up the crowd: Micro-task pricing schemes for worker retention and latency improvement. In Second AAAI Conference on Human Computation and Crowdsourcing.
  8. Graham Dove . 2020. Monsters, Metaphors, and Machine Learning . ( 2020 ), 1–17 . Graham Dove. 2020. Monsters, Metaphors, and Machine Learning. (2020), 1–17.
  9. Franz Faul , Edgar Erdfelder , Albert-Georg Lang , and Axel Buchner . 2007. G* Power 3 : A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods 39, 2 ( 2007 ), 175–191. Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods 39, 2 (2007), 175–191.
  10. Susan A Gelman and Cristine H Legare . 2011. Concepts and folk theories. Annual review of anthropology 40 ( 2011 ), 379–398. Susan A Gelman and Cristine H Legare. 2011. Concepts and folk theories. Annual review of anthropology 40 (2011), 379–398.
  11. Agneta Gulz . 2006 . Visual Design of Virtual Pedagogical Agents : Naturalism versus Stylization in Static Appearance . 6th International Conference on Intelligent Virtual Agents, IVA 2006 2006 (2006), 1–9. Agneta Gulz. 2006. Visual Design of Virtual Pedagogical Agents : Naturalism versus Stylization in Static Appearance. 6th International Conference on Intelligent Virtual Agents, IVA 2006 2006 (2006), 1–9.
  12. Sandra G Hart and Lowell E Staveland . 1988. Development of NASA-TLX. Human Mental Workload. Advances in Psychology52 ( 1988 ), 139–183. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX. Human Mental Workload. Advances in Psychology52 (1988), 139–183.
  13. Ting- Hao Kenneth Huang , Walter S Lasecki , and Jeffrey P Bigham . 2015 . Guardian: A crowd-powered spoken dialog system for web apis . In Third AAAI conference on human computation and crowdsourcing. Ting-Hao Kenneth Huang, Walter S Lasecki, and Jeffrey P Bigham. 2015. Guardian: A crowd-powered spoken dialog system for web apis. In Third AAAI conference on human computation and crowdsourcing.
  14. Mohammad Hossein Jarrahi , Will Sutherland , Sarah Beth Nelson , and Steve Sawyer . 2020. Platformic management, boundary resources for gig work, and worker autonomy. Computer supported cooperative work (CSCW) 29, 1 ( 2020 ), 153–189. Mohammad Hossein Jarrahi, Will Sutherland, Sarah Beth Nelson, and Steve Sawyer. 2020. Platformic management, boundary resources for gig work, and worker autonomy. Computer supported cooperative work (CSCW) 29, 1 (2020), 153–189.
  15. George Lakoff and Mark Johnson . 2008. Metaphors we live by . University of Chicago press. George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago press.
  16. Edward McAuley , Terry Duncan , and Vance V Tammen . 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research quarterly for exercise and sport 60, 1 ( 1989 ), 48–58. Edward McAuley, Terry Duncan, and Vance V Tammen. 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research quarterly for exercise and sport 60, 1 (1989), 48–58.
  17. Michael F McTear . 2016 . The rise of the conversational interface: A new kid on the block? . In International workshop on future and emerging trends in language technology. Springer, 38–49 . Michael F McTear. 2016. The rise of the conversational interface: A new kid on the block?. In International workshop on future and emerging trends in language technology. Springer, 38–49.
  18. Jeffrey M Rzeszotarski , Ed Chi , Praveen Paritosh , and Peng Dai . 2013 . Inserting micro-breaks into crowdsourcing workflows . In First AAAI Conference on Human Computation and Crowdsourcing. Jeffrey M Rzeszotarski, Ed Chi, Praveen Paritosh, and Peng Dai. 2013. Inserting micro-breaks into crowdsourcing workflows. In First AAAI Conference on Human Computation and Crowdsourcing.
  19. Edward Schneider , Yifan Wang , and Shanshan Yang . 2007 . Exploring the Uncanny Valley with Japanese Video Game Characters .. In DiGRA Conference. Edward Schneider, Yifan Wang, and Shanshan Yang. 2007. Exploring the Uncanny Valley with Japanese Video Game Characters.. In DiGRA Conference.
  20. Robin Sease . 2008. Metaphor’s role in the information behavior of humans interacting with computers. Information technology and libraries 27, 4 ( 2008 ), 9–16. Robin Sease. 2008. Metaphor’s role in the information behavior of humans interacting with computers. Information technology and libraries 27, 4 (2008), 9–16.
  21. Muzafer Sherif , Daniel Taub , and Carl I Hovland . 1958. Assimilation and contrast effects of anchoring stimuli on judgments.Journal of experimental psychology 55, 2 ( 1958 ), 150. Muzafer Sherif, Daniel Taub, and Carl I Hovland. 1958. Assimilation and contrast effects of anchoring stimuli on judgments.Journal of experimental psychology 55, 2 (1958), 150.
  22. Astrid M Von der Pütten , Nicole C Krämer , Jonathan Gratch , and Sin-Hwa Kang . 2010. “ It doesn’t matter what you are! ” explaining social effects of agents and avatars.Computers in Human Behavior( 2010 ). Astrid M Von der Pütten, Nicole C Krämer, Jonathan Gratch, and Sin-Hwa Kang. 2010. “It doesn’t matter what you are!” explaining social effects of agents and avatars.Computers in Human Behavior(2010).
  23. Alan Wexelblat . 1998 . Don’t make that face: A report on anthropomorphizing an interface . Intelligent Environments 173 , 179 (1998), 98 – 02 . Alan Wexelblat. 1998. Don’t make that face: A report on anthropomorphizing an interface. Intelligent Environments 173, 179 (1998), 98–02.
  24. 10.5555/1870559.1870562 · doi:10.5555/1870559.1870562
  25. Aiyana K Willard and Rita A McNamara . 2019. The minds of god (s) and humans: Differences in mind perception in Fiji and North America. Cognitive science 43, 1 ( 2019 ), e12703. Aiyana K Willard and Rita A McNamara. 2019. The minds of god (s) and humans: Differences in mind perception in Fiji and North America. Cognitive science 43, 1 (2019), e12703.
  26. Jie Yang , Judith Redi , Gianluca Demartini , and Alessandro Bozzon . 2016 . Modeling task complexity in crowdsourcing . In Fourth AAAI Conference on human computation and crowdsourcing. Jie Yang, Judith Redi, Gianluca Demartini, and Alessandro Bozzon. 2016. Modeling task complexity in crowdsourcing. In Fourth AAAI Conference on human computation and crowdsourcing.

Cited by

See all 66

Metadata

Authors
Jung, J.-Y., Qiu, S., Bozzon, A., & Gadiraju, U.
Affiliations
Delft University of Technology (NL)
Venue
CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2022-04-27
Pages
1-22
OpenAlex
W4224991000
Cited by
43
References
77
Citations by year
2026: 8 · 2025: 14 · 2024: 10 · 2023: 7 · 2022: 4
FWCI
16.85
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • Research Repository (Delft University of Technology) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
Keywords
chatbots, metaphorical representation, Great Chain of Being, employee engagement, cognitive load, intrinsic motivation, trust in agents

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.