The Entoptic Field Camera as Metaphor-Driven Research-through-Design with AI Technologies
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) technologies are widely deployed in smartphone photography; and prompt-based image synthesis models have rapidly become commonplace. In this paper, we describe a Research-through-Design (RtD) project which explores this shift in the means and modes of image production via the creation and use of the Entoptic Field Camera. Entoptic phenomena usually refer to perceptions of floaters or bright blue dots stemming from the physiological interplay of the eye and brain. We use the term entoptic as a metaphor to investigate how the material interplay of data and models in AI technologies shapes human experiences of reality. Through our case study using first-person design and a field study, we offer implications for critical, reflective, more-than-human and ludic design to engage AI technologies; the conceptualisation of an RtD research space which contributes to AI literacy discourses; and outline a research trajectory concerning materiality and design affordances of AI technologies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“[68] for an overview); not only referring to technical knowledge, exact terminologies or end-user competencies, but rather as also referring to the intuitive (mis)understandings and experiences of how AI technologies shape”
BackgroundHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“, learning, sentience) has been debated since the early days of AI research [72], the stubborn opacity of contemporary AI technologies at runtime [21, 30] also means that it requires in-depth practical and conceptual engagement to gain a foothold on them.”
BackgroundTechnology and the lifeworld: from garden to earth
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1535 · Cited by 1,281
Sengers, P., Boehner, K., David, S., & Kaye, J. '. (2005) · Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility · doi:10.1145/1094562.1094569 · Cited by 1,030
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863
Turkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 1×
“The question of causality, dependencies and mixed agency is a vital one with regards to AI technologies, which prove stubborn in various domains such as the socio-economic conditions of crowdworkers [53], extractive end-user engagements [33] or intellectual ownership (see e.”
BackgroundRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 1×
“[49, 57, 76, 81]) with the goal of explanatory guidance.”
Algorithms and their others: Algorithmic culture in context
Dourish, P. (2016) · Big Data & Society · doi:10.1177/2053951716665128 · Cited by 481 · cited 1×
“, learning, sentience) has been debated since the early days of AI research [72], the stubborn opacity of contemporary AI technologies at runtime [21, 30] also means that it requires in-depth practical and conceptual engagement to gain a foothold on them.”
BackgroundThe drift table: designing for ludic engagement
Gaver, W. W., Bowers, J., Boucher, A., Gellerson, H., Pennington, S., Schmidt, A., Steed, A., Villars, N., & Walker, B. (2004) · CHI '04 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/985921.985947 · Cited by 439
What Things Do: Philosophical Reflections on Technology, Agency, and Design
Verbeek, P.-P. (2005) · doi:10.1515/9780271033228 · Cited by 371
Artificial intelligence meets natural stupidity
McDermott, D. (1976) · ACM SIGART Bulletin · doi:10.1145/1045339.1045340 · Cited by 323 · cited 1×
“, learning, sentience) has been debated since the early days of AI research [72], the stubborn opacity of contemporary AI technologies at runtime [21, 30] also means that it requires in-depth practical and conceptual engagement to gain a foothold on them.”
BackgroundFrom Research Prototype to Research Product
Odom, W., Wakkary, R., Lim, Y.-K., Desjardins, A., Hengeveld, B., & Banks, R. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858447 · Cited by 259
Large image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257
Multimodal datasets: misogyny, pornography, and malignant stereotypes
Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021) · arXiv · doi:10.48550/arxiv.2110.01963 · Cited by 160
PromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157
Xie, Y., Chen, M., Kao, D., Gao, G., & Chen, X. '. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376807 · Cited by 141
Expanding and Refining Design and Criticality in HCI
Pierce, J., Sengers, P., Hirsch, T., Jenkins, T., Gaver, W., & DiSalvo, C. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702438 · Cited by 133 · cited 1×
“The entoptic metaphor, and the findings from its concretization in the Entoptic Field Camera, can be used to engage this complex through critical [6, 86] and reflective [98] design which seeks to unfold or question how agency is distributed with regards to AI technologies and the systems these are…”
Self-citationMachine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry
Benjamin, J. J., Berger, A., Merrill, N., & Pierce, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445481 · Cited by 130
Self-citationHeteromation and its (dis)contents: The invisible division of labor between humans and machines
Ekbia, H., & Nardi, B. (2014) · First Monday · doi:10.5210/fm.v19i6.5331 · Cited by 113
A Sample of One: First-Person Research Methods in HCI
Lucero, A., Desjardins, A., Neustaedter, C., Höök, K., Hassenzahl, M., & Cecchinato, M. E. (2019) · Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion · doi:10.1145/3301019.3319996 · Cited by 96 · cited 1×
“[69]) conducted by Benjamin over a period of approximately six months through a process informed by autobiographical design.”
MethodRemoval as a Method: A Fourth Wave HCI Approach to Understanding the Experience of Self-Tracking
Homewood, S., Karlsson, A., & Vallgårda, A. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395425 · Cited by 87
More-Than-Human Design and AI: In Conversation with Agents
Nicenboim, I., Giaccardi, E., Søndergaard, M. L. J., Reddy, A. V., Strengers, Y., Pierce, J., & Redström, J. (2020) · Companion Publication of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3393914.3395912 · Cited by 63
Self-citation
Research works printed but not linked: 78
- 2010. entoptic phenomena. In Concise Medical Dictionary. Oxford University Press. https://www.oxfordreference.com/view/10.1093/acref/9780199557141.001.0001/acref-9780199557141-e-3227 · doi:10.1093/acref/9780199557141.001.0001/acref-9780199557141-e-3227
- Philip Agre. 1997. Computation and human experience. Cambridge University Press, Cambridge ; New York.
- Philip E. Agre. 1997. Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI. In Social Science, Technical Systems, and Cooperative Work: Bridging the Great Divide, Geoff Bowker, Susan Leigh Star, Willian Turner, and Les Gasser (Eds.). L. Erlbaum Associates, New Jersey, USA, 131–156.
- M. M. Bakhtin. 1986. Speech genres and other late essays(1st ed ed.). Number no. 8 in University of Texas Press Slavic series. University of Texas Press, Austin.
- 10.1145/2470654.2466451 · DOI · Paper page
- 10.1111/ips.12017 · DOI
- Jesse Josua Benjamin. 2021. The Spark of a Future Anterior: An Archaeology of Entoptic Media. ŠUM Journal for Contemporary Art Criticism and Theory16 (2021), 2172–2184. https://www.sum.si/journal-articles/the-spark-of-a-future-anterior-an-archaeology-of-entoptic-media Publisher: Društvo Galerija Books.
- 10.1145/3492858 · DOI · Paper page
- 10.1145/3313831.3376429 · DOI
- 10.1145/1188816.1188820 · DOI · Paper page
- 10.1145/2212776.2212703 · DOI
- 10.1145/3325480.3326569 · doi:10.1145/3325480.3326569
- 10.1109/CVPR.2016.105 · DOI
- 10.1145/3461778.3462000 · DOI
- Benjamin Bratton and Blaise Agüera Y Arcas. 2022. The Model Is The Message. NOEMA (July 2022). https://www.noemamag.com/the-model-is-the-message
- 10.1111/j.1467-954X.1990.tb03351.x · DOI · Paper page
- Ernst Cassirer. 1953. Language and Myth(dover ed ed.). Dover Publications Inc., New York.
- 10.1109/CVPR.2018.00660 · DOI · Paper page
- Kate Crawford and Vladan Joler. 2018. Anatomy of an AI System. http://www.anatomyof.ai · DOI
- 10.1145/3322276.3322311 · DOI
- Carl DiSalvo. 2012. Adversarial design. MIT Press, Cambridge, Mass. ; London.
- Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. 2016. Adversarial feature learning. arXiv preprint arXiv:1605.09782 Published as a conference paper at ICLR 2017.(2016). · arXiv
- 10.1145/3313831.3376275 · DOI · Paper page
- Tony Dunne and Delena Pacenti Bill Gaver. 1999. Cultural Probes. ACM Interactions February (1999), 21–29.
- 10.1145/3491102.3501973 · DOI
- Vilém Flusser. 2011. Into the Universe of Technical Images. Number 32 in Electronic Mediations. University of Minnesota Press, Minneapolis, London.
- Christopher Frayling. 1994. Research in Art and Design. 1, 1 (1994). http://researchonline.rca.ac.uk/384/
- 10.1145/3532106.3533524 · DOI · Paper page
- 10.1145/2470654.2466474 · DOI
- 10.1145/3411764.3445232 · DOI
- Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:1406.2661 [cs, stat] (June 2014). http://arxiv.org/abs/1406.2661 arXiv:1406.2661. · arXiv
- 10.1145/3025453.3025606 · DOI
- 10.1145/3173574.3173775 · DOI
- Martin Heidegger. 2005. The basic problems of phenomenology. Indiana University Press, Bloomington; Indianapolis. OCLC: 750615910.
- Martin Heidegger. 2010. Being and time. State University of New York Press, Albany. OCLC: ocn608297834.
- 10.1145/3290605.3300809 · DOI · Paper page
- Don Ihde. 1998. Expanding hermeneutics: visualism in science. Northwestern University Press, Evanston, Ill.
- 10.1145/2399016.2399019 · DOI
- Li Jönsson, Martín Tironi, Pablo Hermansen, and Alex Wilkie. 2022. Doing and Undoing Post-Anthropocentric Design. In DRS Biennial Conference Series. https://dl.designresearchsociety.org/drs-conference-papers/drs2022/editorials/17
- 10.2312/trvis.20191183 · DOI
- Friedrich Kittler. 2010. Optical media : Berlin lectures 1999. Polity, Cambridge, UK ; Malden, MA.
- 10.1145/2901790.2901840 · DOI
- 10.23987/sts.75323 · DOI
- 10.1086/203629 · DOI · Paper page
- 10.1109/MPRV.2012.57 · DOI
- 10.1145/3322276.3323701 · DOI
- Maria Lindh. 2016. As a Utility – Metaphors of Information Technologies. Human IT 13, 2 (2016), 47–80. http://urn.kb.se/resolve?urn=urn:nbn:se:hb:diva-10679 Publisher: Högskolan i Borås.
- 10.1145/3313831.3376792 · DOI · Paper page
- Joseph Galen Lindley, Paul Coulton, Haider Ali Akmal, and Franziska Louise Pilling. 2020. Signs of the Time: Making AI Legible. In DRS Biennial Conference Series. https://dl.designresearchsociety.org/drs-conference-papers/drs2020/researchpapers/63
- 10.1145/3196709.3196819 · doi:10.1145/3196709.3196819
- 10.51151/identities.v15i1-2.329 · DOI
- Davor Löffler. 2019. Generative Realitäten I Die Technologische Zivilisation als neue Achsenzeit und Zivilisationsstufe Eine Anthropologie des 21. Jahrhunderts. Velbrück Wissenschaft. OCLC: 1081348786. · doi:10.5771/9783748906582
- 10.1145/2901790.2901906 · DOI
- W. J. T Mitchell. 1995. Picture theory : essays on verbal and visual representation (paperback ed. ed.). Chicago [u.a.].
- Alexander Mordvintsev, Christopher Olah, and Mike Tyke. 2015. Inceptionism: Going Deeper into Neural Networks. http://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html
- Dave Murray-Rust Iohanna Nicenboim and Dan Lockton. 2022. Metaphors for designers working with AI. https://doi.org/10.21606/drs.2022.667 · DOI
- 10.1145/2377783.2377791 · DOI
- Erwin Panofsky. 1991. Perspective as Symbolic Form(1st ed ed.). Zone Books ; Distributed by the MIT Press, New York : Cambridge, Mass.
- 10.1145/2598510.2598525 · DOI
- 10.1145/3411764.3445406 · DOI · Paper page
- 10.1145/2702123.2702405 · DOI
- 10.1177/1086296X12457167 · doi:10.1177/1086296x12457167
- 10.1007/978-3-030-67070-2_14 · DOI
- 10.1007/s00146-020-01088-7 · DOI
- 10.1145/3301653 · doi:10.1145/3301653
- 10.1007/s11263-015-0816-y · DOI · Paper page
- Donald A. Schön. 1979. Generative metaphor: A perspective on problem-setting in social policy. In Metaphor and thought. Vol. 2. Cambridge University Press Cambridge, 137–163.
- 10.1145/3461778.3462163 · DOI · Paper page
- 10.1145/1978942.1979170 · DOI
- 10.21606/drs.2022.782 · DOI
- Francisco Varela and Jonathan Shear. 1999. First-Person Methodologies: What, Why, How?Journal of Consciousness Studies 6, 2-3 (1999), 1–14.
- 10.1145/2751314 · DOI · Paper page
- Ron Wakkary. 2021. Things We Could Design: For More Than Human-Centered Worlds. MIT Press, Cambridge, MA, USA.
- 10.1145/3064663.3064734 · DOI
- 10.1163/156916400746632 · doi:10.1163/156916400746632
- 10.1145/3290605.3300831 · DOI · Paper page
- Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, and Zhan Xu. 2020. Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting. arXiv:2005.09704 [cs] (May 2020). http://arxiv.org/abs/2005.09704 arXiv:2005.09704. · arXiv
- 10.1145/3322276.3323689 · DOI
Cited by
Benjamin, J. J., Lindley, J., Edwards, E., Rubegni, E., Korjakow, T., Grist, D., & Sharkey, R. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660677 · Cited by 32
Self-citationZhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
“Cremaschi et al. [40], for example, juxtapose a typewriter with modern AI to provoke thoughts on transparency, while Benjamin et al. [16] use a camera-based ‘entoptic metaphor’ enabling non-technical users to explore AI’s influence.”
InfluentialReflection Across AI-based Music Composition
Ford, C., Noel-Hirst, A., Cardinale, S., Loth, J., Sarmento, P., Wilson, E., Wolstanholme, L., Worrall, K., & Bryan-Kinns, N. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656185 · Cited by 22
Metadata
- Authors
- Benjamin, J. J., Biggs, H., Berger, A., Rukanskaitė, J., Heidt, M. B., Merrill, N., Pierce, J., & Lindley, J.
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-19
- OpenAlex
- W4317951178
- Cited by
- 35
- References
- 27
- FWCI
- 8.33
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- Funders
- NSF (National Science Foundation) (2142795 / 2230825)
- UK Research and Innovation (MR/T019220/1)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.