A composition algorithm based on crossmodal taste-music correspondences
Open accessAbstract
While there is broad consensus about the structural similarities between language and music, comparably less attention has been devoted to semantic correspondences between these two ubiquitous manifestations of human culture. We have investigated the relations between music and a narrow and bounded domain of semantics: the words and concepts referring to taste sensations. In a recent work, we found that taste words were consistently mapped to musical parameters. Bitter is associated with low-pitched and continuous music (legato), salty is characterized by silences between notes (staccato), sour is high pitched, dissonant and fast and sweet is consonant, slow and soft (Mesz et al., 2011). Here we extended these ideas, in a synergistic dialog between music and science, investigating whether music can be algorithmically generated from taste-words. We developed and implemented an algorithm that exploits a large corpus of classic and popular songs. New musical pieces were produced by choosing fragments from the corpus and modifying them to minimize their distance to the region in musical space that characterizes each taste. In order to test the capability of the produced music to elicit significant associations with the different tastes, musical pieces were produced and judged by a group of non-musicians. Results showed that participants could decode well above chance the taste-word of the composition. We also discuss how our findings can be expressed in a performance bridging music and cognitive science.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Specific auditory attributes are associated with basic tastes, both when presented as taste words and in the form of tasting solutions.
“Previously, it has been shown that specific auditory attributes are associated with basic tastes, both when presented as taste words (for instance, 'sweet'), and in the form of tasting solutions [80-88]”
From The Role of Intrinsic and Extrinsic Sensory Factors in Sweetness Perception of Food and Beverages: A Review (2019) · cites this paper as [84]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Psychoacoustics: facts and models
Fastl, H., & Zwicker, E. (1991) · Choice Reviews Online · doi:10.5860/choice.28-5706 · Cited by 2,413 · see citing works
A theory of magnitude: common cortical metrics of time, space and quantity
Walsh, V. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.09.002 · Cited by 2,044 · see citing works
When the brain plays music: auditory–motor interactions in music perception and production
Zatorre, R. J., Chen, J. L., & Penhune, V. B. (2007) · Nature reviews. Neuroscience · doi:10.1038/nrn2152 · Cited by 1,772 · see citing works
Vocal affect expression: A review and a model for future research.
Scherer, K. R. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.2.143 · Cited by 1,768 · see citing works
Crossmodal correspondences: A tutorial review
Spence, C. (2011) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-010-0073-7 · Cited by 1,618 · see citing works
(1999) · Cited by 1,435 · see citing works
Language, music, syntax and the brain
Patel, A. D. (2003) · Nature Neuroscience · doi:10.1038/nn1082 · Cited by 1,246 · see citing works
Fraisse, P. (1982) · Psychology of Music · doi:10.1016/b978-0-12-213562-0.50010-3 · Cited by 818 · see citing works
Tonal Consonance and Critical Bandwidth
Plomp, R., & Levelt, W. J. M. (1965) · The Journal of the Acoustical Society of America · doi:10.1121/1.1909741 · Cited by 786 · see citing works
Music, language and meaning: brain signatures of semantic processing
Koelsch, S., Kasper, E., Sammler, D., Schulze, K., Gunter, T. C., & Friederici, A. D. (2004) · Nature Neuroscience · doi:10.1038/nn1197 · Cited by 566 · see citing works
Fastl, H., & Zwicker, E. (2007) · Springer eBooks · doi:10.1007/978-3-540-68888-4 · Cited by 539 · see citing works
Human Facial Expressions in Response to Taste and Smell Stimulation
Steiner, J. (1979) · Advances in child development and behavior · doi:10.1016/s0065-2407(08)60349-3 · Cited by 535 · see citing works
Multisensory perception: Beyond modularity and convergence
Driver, J., & Spence, C. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00740-5 · Cited by 465 · see citing works
Natural cross-modal mappings between visual and auditory features
Evans, K. K., & Treisman, A. (2010) · Journal of Vision · doi:10.1167/10.1.6 · Cited by 403 · see citing works
The capacity for music: What is it, and what’s special about it?
Jackendoff, R. S., & Lerdahl, F. (2005) · Cognition · doi:10.1016/j.cognition.2005.11.005 · Cited by 361 · see citing works
Acoustic effects of variation in vocal effort by men, women, and children
Traunmüller, H., & Eriksson, A. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.429414 · Cited by 314 · see citing works
Crisinel, A.-S., Cosser, S., King, S., Jones, R., Petrie, J., & Spence, C. (2012) · Food Quality and Preference · doi:10.1016/j.foodqual.2011.08.009 · Cited by 231 · see citing works
An interaction between prosody and statistics in the segmentation of fluent speech.
Shukla, M., Nespor, M., & Mehler, J. (2007) · Cognitive Psychology · doi:10.1016/j.cogpsych.2006.04.002 · Cited by 196 · see citing works
Crisinel, A.-S., & Spence, C. (2010) · Attention, Perception & Psychophysics · doi:10.3758/app.72.7.1994 · Cited by 184 · see citing works
Hearing Colors, Tasting Shapes
Ramachandran, V. S., & Hubbard, E. M. (2003) · Scientific American · doi:10.1038/scientificamerican0503-52 · Cited by 179 · see citing works
What sound does that taste? Cross-modal mappings across gustation and audition
Simner, J., Cuskley, C., & Kirby, S. (2010) · Perception · doi:10.1068/p6591 · Cited by 174 · see citing works
Implicit association between basic tastes and pitch
Crisinel, A.-S., & Spence, C. (2009) · Neuroscience Letters · doi:10.1016/j.neulet.2009.08.016 · Cited by 141 · see citing works
Children with Specific Language Impairment Also Show Impairment of Music-syntactic Processing
Jentschke, S., Koelsch, S., Sallat, S., & Friederici, A. D. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20135 · Cited by 114 · see citing works
A sweet sound? Food names reveal implicit associations between taste and pitch
Crisinel, A.-S., & Spence, C. (2010) · Perception · doi:10.1068/p6574 · Cited by 108 · see citing works
Mesz, B., Trevisan, M. A., & Sigman, M. (2011) · Perception · doi:10.1068/p6801 · Cited by 108 · see citing works
Self-citationWhen coloured sounds taste sweet
Beeli, G., Esslen, M., & Jäncke, L. (2005) · Nature · doi:10.1038/434038a · Cited by 106 · see citing works
Synesthetic congruency modulates the temporal ventriloquism effect
Parise, C., & Spence, C. J. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2008.07.010 · Cited by 99 · see citing works
The multiple synaesthete E.S. — Neuroanatomical basis of interval-taste and tone-colour synaesthesia
Hänggi, J., Beeli, G., Oechslin, M. S., & Jäncke, L. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.07.018 · Cited by 86 · see citing works
Investigating the Relationship of Music and Language in Children
Jentschke, S., Koelsch, S., & Friederici, A. D. (2005) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1360.016 · Cited by 80 · see citing works
Luck, G., Toiviainen, P., Erkkilä, J., Lartillot, O., Riikkilä, K., Mäkelä, A., Pyhäluoto, K., Raine, H., Varkila, L., & Värri, J. (2007) · Psychology of Music · doi:10.1177/0305735607079714 · Cited by 47 · see citing works
Synaesthetic visuo-spatial forms: Viewing sequences in space
Simner, J. (2009) · Cortex · doi:10.1016/j.cortex.2009.07.001 · Cited by 33 · see citing works
Jentschke, S., Koelsch, S., & Friederici, A. D. (2005) · Max Planck Institute for Plasma Physics · Cited by 6 · see citing works
Flagged first, then by citations
2 of the 34 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Cariani URL: http://homepage.mac.com/cariani/CarianiNewWebsite/Publications_files/Cariani-ICMPC-8-2004-Proceedings.pdf
- Chomsky N. (1988). Language and Problems of Language. Cambridge, MA: MIT Press
Cited by
Sorted by citations
Crossmodal correspondences between sounds and tastes
Knöferle, K., & Spence, C. J. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0321-z · Cited by 209 · see citing works
Spence, C., Velasco, C., & Knoeferle, K. (2014) · Flavour · doi:10.1186/2044-7248-3-8 · Cited by 175 · see citing works
Wang, Q. J., Mielby, L. A., Junge, J. Y., Bertelsen, A. S., Kidmose, U., Spence, C., & Byrne, D. V. (2019) · Foods · doi:10.3390/foods8060211 · Cited by 168 · see citing works
Spence, C. J., Reinoso‐Carvalho, F., Velasco, C., & Wang, Q. J. (2019) · Multisensory Research · doi:10.1163/22134808-20191403 · Cited by 173 · see citing works
That Sounds Sweet: Using Cross‐Modal Correspondences to Communicate Gustatory Attributes
Knoeferle, K. M., Woods, A., Käppler, F., & Spence, C. (2015) · Psychology & Marketing · doi:10.1002/mar.20766 · Cited by 137 · see citing works
Is Multimedia Multisensorial? - A Review of Mulsemedia Systems
Covaci, A., Zou, L., Tal, I., Muntean, G., & Ghinea, G. (2018) · ACM Computing Surveys · doi:10.1145/3233774 · Cited by 110 · see citing works
The influence of soundscapes on the perception and evaluation of beers
Reinoso Carvalho, F., Wang, Q. (., Van Ee, R., & Spence, C. (2016) · Food Quality and Preference · doi:10.1016/j.foodqual.2016.03.009 · Cited by 88 · see citing works
Technology at the dining table
Spence, C. J., & Piqueras‐Fiszman, B. (2013) · Flavour · doi:10.1186/2044-7248-2-16 · Cited by 95 · see citing works
Wang, Q. J., Woods, A. T., & Spence, C. J. (2015) · i-Perception · doi:10.1177/2041669515622001 · Cited by 85 · see citing works
Composing with Cross-modal Correspondences: Music and Odors in Concert
Crisinel, A.-S., Jacquier, C., Deroy, O. D., & Spence, C. J. (2013) · Chemosensory Perception · doi:10.1007/s12078-012-9138-4 · Cited by 83 · see citing works
Spence, C. J., & Wang, Q. J. (2015) · Flavour · doi:10.1186/s13411-015-0043-z · Cited by 84 · see citing works
Looking for crossmodal correspondences between classical music and fine wine
Spence, C. J., Richards, L., Kjellin, E., Huhnt, A.-M., Daskal, V., Scheybeler, A., Velasco, C., & Deroy, O. D. (2013) · Flavour · doi:10.1186/2044-7248-2-29 · Cited by 72 · see citing works
On the shapes of flavours: A review of four hypotheses
Spence, C., & Deroy, O. (2014) · Theoria et Historia Scientiarum · doi:10.12775/ths-2013-0011 · Cited by 62 · see citing works
Crossmodal interactions between audition and taste: A systematic review and narrative synthesis
Guedes, D., Garrido, M. V., Lamy, E., Cavalheiro, B. P., & Prada, M. (2023) · Food Quality and Preference · doi:10.1016/j.foodqual.2023.104856 · Cited by 60 · see citing works
Wang, Q. J., Mielby, L. A., Thybo, A. K., Bertelsen, A. S., Kidmose, U., Spence, C., & Byrne, D. V. (2019) · Journal of Sensory Studies · doi:10.1111/joss.12492 · Cited by 49 · see citing works
Sweet and sour: music and taste associations
Kontukoski, M., Luomala, H., Mesz, B. A., Sigman, M., Trevisan, M. A., Rotola-Pukkila, M., & Hopia, A. (2015) · Nutrition & Food Science · doi:10.1108/nfs-01-2015-0005 · Cited by 48 · see citing works
When music is salty: The crossmodal associations between sound and taste
Guetta, R. E., & Loui, P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0173366 · Cited by 44 · see citing works
Wine and music (I): on the crossmodal matching of wine and music
Spence, C. J., & Wang, Q. J. (2015) · Flavour · doi:10.1186/s13411-015-0045-x · Cited by 42 · see citing works
The taste & affect music database: Subjective rating norms for a new set of musical stimuli
Guedes, D., Prada, M., Garrido, M. V., & Lamy, E. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01862-z · Cited by 38 · see citing works
Wine psychology: basic & applied
Spence, C. J. (2020) · Cognitive Research Principles and Implications · doi:10.1186/s41235-020-00225-6 · Cited by 36 · see citing works
Spence, C. J., Wang, Q. J., Reinoso‐Carvalho, F., & Keller, S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.740354 · Cited by 34 · see citing works
Gastrophysics: Getting creative with pairing flavours
Spence, C. J. (2021) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2021.100433 · Cited by 31 · see citing works
Sonic branding: A narrative review at the intersection of art and science
Spence, C. J., & Keller, S. (2024) · Psychology and Marketing · doi:10.1002/mar.21995 · Cited by 36 · see citing works
Sonic Seasoning and Other Multisensory Influences on the Coffee Drinking Experience
Spence, C. J. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.644054 · Cited by 29 · see citing works
Wine and music (III): so what if music influences the taste of the wine?
Spence, C. J., & Wang, Q. J. (2015) · Flavour · doi:10.1186/s13411-015-0046-9 · Cited by 27 · see citing works
Spence, C. J. (2015) · Flavour · doi:10.1186/s13411-015-0035-z · Cited by 20 · see citing works
Taste the Bass: Low Frequencies Increase the Perception of Body and Aromatic Intensity in Red Wine
Burzynska, J., Wang, Q. J., Spence, C. J., & Bastian, S. E. P. (2019) · Multisensory Research · doi:10.1163/22134808-20191406 · Cited by 20 · see citing works
The influence of music on the perception of taste
Campinho, J. D. S., de Sousa, P. H. M., & Mata, P. (2023) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2023.100669 · Cited by 19 · see citing works
‘Sounds salty!’ How a soundtrack affects the liking and perception of the salty balance in bread
Swahn, J., & Nilsen, A. N. (2023) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2023.100718 · Cited by 17 · see citing works
Tasting prosody: Crossmodal correspondences between voice quality and basic tastes
Motoki, K., Pathak, A., & Spence, C. J. (2022) · Food Quality and Preference · doi:10.1016/j.foodqual.2022.104621 · Cited by 14 · see citing works
Septianto, F. (2016) · Asia Pacific Journal of Marketing and Logistics · doi:10.1108/apjml-11-2015-0182 · Cited by 12 · see citing works
Radical Creative Semantic Anchoring: Creative-action metaphors and timbral interaction
Simurra, I., Messina, M., Aliel, L., & Keller, D. (2022) · Organised Sound · doi:10.1017/s1355771822000322 · Cited by 11 · see citing works
Park, C., & Kim, J. (2024) · Cognition · doi:10.1016/j.cognition.2024.105830 · Cited by 10 · see citing works
Sensitive to music? Examining the crossmodal effect of audition on sweet taste sensitivity
Guedes, D., Prada, M., Garrido, M. V., Caeiro, I., Simões, C. R. B., & Lamy, E. (2023) · Food Research International · doi:10.1016/j.foodres.2023.113256 · Cited by 7 · see citing works
Music attributes and the perception of orange juice
Istiani, N. F. F., Masullo, M., Ruggiero, G., Francini, M., & Maffei, L. (2024) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2024.100953 · Cited by 9 · see citing works
Mygdanis, Y., & Papazachariou-Christoforou, M. (2023) · Journal of Music Technology and Education · doi:10.1386/jmte_00062_1 · Cited by 6 · see citing works
Crossmodal congruency effects between sound and food pictures in a forced-choice task
Padulo, C., Mangone, M., Brancucci, A., Balsamo, M., & Fairfield, B. (2020) · Psychological Research · doi:10.1007/s00426-020-01406-0 · Cited by 7 · see citing works
Implicit Association Effects Between Sound and Food Images
Padulo, C., Tommasi, L., & Brancucci, A. (2018) · Multisensory Research · doi:10.1163/22134808-20181308 · Cited by 6 · see citing works
Multisensory interactions: The influence of context on beverage and soundtrack pairing
Vandenberghe‐Descamps, M., Mizener, B., Abdi, H., & Chollet, S. (2024) · Food Quality and Preference · doi:10.1016/j.foodqual.2024.105297 · Cited by 4 · see citing works
Perspectives on Multisensory Human-Food Interaction
Velasco, C., Obrist, M., Huisman, G., Nijholt, A., Spence, C., Motoki, K., & Narumi, T. (2022) · Frontiers research topics · doi:10.3389/978-2-88974-233-2 · Cited by 4 · see citing works
Sound Bites & Digital Seasoning
Spence, C. J. (2020) · array. the journal of the ICMA · doi:10.25370/array.v20152516 · Cited by 4 · see citing works
A multimodal symphony: integrating taste and sound through generative AI
Spanio, M., Zampini, M., Rodà, A., & Pierucci, F. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1575741 · Cited by 3 · see citing works
Music and Mathematics in Latin America: Major Developments in the Last 25 Years
Pareyón, G., Almada, C. D. L., Mathias, C., Saraiva, C., de Sousa, D. M., de Carvalho, H. T., Pitombeira, L., Gentil-Nunes, P., Mesz, B. A., Amster, P., et al. (2022) · MusMat Brazilian Journal of Music and Mathematics · doi:10.46926/musmat.2022v6n1.12-47 · Cited by 1 · see citing works
The Influence of Music on the Perception of Taste
Campinho, J. D. S. (2018) · Cited by 1 · see citing works
Technology at the Dining Table
Spence, C. J., & Piqueras‐Fiszman, B. (2014) · doi:10.1002/9781118491003.ch10
Emotions versus semantics: the cross-modal influence of songs on the sweetness of chocolate
Kambayashi, M. (2024) · Journal of High School Science · doi:10.64336/001c.122036
Deng, J., Li, M. S., Fan, Z., Wang, H., Bennett, P., Fang, Z., Lo, P., & Metatla, O. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812877
φιλοσοφία του maker movement σε μαθήματα «θεωρία της μουσικής» σε ένα ωδείο στον ελλαδικό χώρο
Mygdanis, Y., & Papazachariou-Christoforou, M. (2022) · Μουσικοπαιδαγωγικά · doi:10.12681/musped.46149
- Authors
- Mesz, B., Sigman, M., & Trevisan, M. A.
- Affiliations
- National University of Quilmes (AR)
- Universidad de Buenos Aires (AR)
- Corresponding author
- Trevisan, Marcos A.
- Venue
- Frontiers in Human Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2012
- Volume
- 6
- Pages
- 71-71
- ISSN
- 1662-5161 (ISSN-L)
- PMC
- PMC3338107
- OpenAlex
- W2049328350
- Cited by
- 48
- References
- 34
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 5 · 2023: 5 · 2022: 6 · 2021: 3 · 2020: 3 · 2019: 4 · 2018: 3 · 2017: 1 · 2016: 2 · 2015: 6 · 2014: 4 · 2013: 3 · 2012: 1
- FWCI
- 4.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- Frontiers in Human Neuroscience (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Repositorio Digital Institucional de la Universidad de Buenos Aires (Universidad de Buenos Aires) (publishedVersion, other-oa)
- Conicet (publishedVersion, cc-by)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Greater South Information System
- Funders
- Consejo Nacional de Investigaciones Científicas y Técnicas
- Universidad de Buenos Aires (CONICET)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Music Technology and Sound Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Keywords
- cross-modal correspondence, taste words, musical space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
