Navegar

Colección

Datos Estadísticas

Artículo

Estamos trabajando para incorporar este artículo al repositorio
Consulte el artículo en la página del editor
Consulte la política de Acceso Abierto del editor

Abstract:

Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias. © 2018 Taitz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Registro:

Documento: Artículo
Título:The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation
Autor:Taitz, A.; Florencia Assaneo, M.; Elisei, N.; Trípodi, M.; Cohen, L.; Sitt, J.D.; Trevisan, M.A.
Filiación:Department of Physics, IFIBA-University of Buenos Aires, Buenos Aires, Argentina
Department of Psychology, New York University, New York, United States
Medicine School, University of Buenos Aires, Buenos Aires, Argentina
Department of Linguistics, University of Buenos Aires, Buenos Aires, Argentina
INSERM U1127, Institut du Cerveau et de la Moelle Épinière, Paris, France
CNRS UMR 7225, Institut du Cerveau et de la Moelle Épinière, Paris, France
Sorbonne Universités, UPMC Univ Paris 06, Paris, France
AP-HP, Groupe Hospitalier Pitié-Salpêtrière, Departament of Neurology, Paris, France
Palabras clave:adult; article; classifier; consonant; controlled study; female; human; human experiment; lip; male; physics; sound; stimulus; tongue; vowel; hearing; language; middle aged; phonetics; physiology; sound; speech perception; theoretical model; vision; voice; young adult; Adult; Auditory Perception; Female; Humans; Language; Male; Middle Aged; Models, Theoretical; Phonetics; Physics; Sound; Speech Perception; Visual Perception; Voice; Young Adult
Año:2018
Volumen:13
Número:3
DOI: http://dx.doi.org/10.1371/journal.pone.0193466
Título revista:PLoS ONE
Título revista abreviado:PLoS ONE
ISSN:19326203
CODEN:POLNC
Registro:https://bibliotecadigital.exactas.uba.ar/collection/paper/document/paper_19326203_v13_n3_p_Taitz

Referencias:

  • Saussure, F.D.E., (2011) Course in General Linguistics, , New York City: Columbia University Press
  • Blasi, D.E., Wichmann, S., Hammarström, H., Stadler, P.F., Christiansen, M.H., Sound–meaning association biases evidenced across thousands of languages (2016) Proc Natl Acad Sci, 113, pp. 10818-10823. , https://doi.org/10.1073/pnas.1605782113, PMID: 27621455
  • Dingemanse, M., Blasi, D.E., Lupyan, G., Christiansen, M.H., Monaghan, P., Arbitrariness, Iconicity, and Sys-tematicity in Language (2015) Trends Cogn Sci, 19, pp. 603-615. , https://doi.org/10.1016/j.tics.2015.07.013, Elsevier Ltd; PMID: 26412098
  • Assaneo, M.F., Nichols, J.I., Trevisan, M.A., The Anatomy of Onomatopoeia (2011) PLoS One, 6, p. e28317. , https://doi.org/10.1371/journal.pone.0028317, Sigman M, editor. PMID: 22194825
  • Imai, M., Kita, S., Nagumo, M., Okada, H., Sound symbolism facilitates early verb learning (2008) Cognition, 109, pp. 54-65. , https://doi.org/10.1016/j.cognition.2008.07.015, Elsevier B.V.; PMID: 18835600
  • Oda, H., (2000) An Embodied Semantic Mechanism for Mimetic Words in Japanese, , Indiana University
  • Dingemanse, M., Ideophones and reduplication: Depiction, description, and the interpretation of repeated talk in discourse (2016) Stud Lang, 39, pp. 946-970. , https://doi.org/10.1075/sl.39.4.05din
  • Nuckolls, J.B., The case for sound symbolism (1999) Annu Rev Anthropol, 28, pp. 225-252. , https://doi.org/10.1146/annurev.anthro.28.1.225
  • Diffloth, G., (1972) Notes on Expressive Meaning, p. M4. , Chicago Linguistic Society
  • Doke, C.M., Early bantu literature the age of brusciotto (1935) Bantu Stud, 9, pp. 87-114. , Taylor & Francis
  • Dingemanse, M., Making new ideophones in Siwu: Creative depiction in conversation (2014) Pragmat Soc, 5, pp. 384-405. , https://doi.org/10.1075/ps.5.3.04din
  • Nuckolls, J.B., (1996) Sounds Like Life: Sound-Symbolic Grammar, Performance, and Cognition in Pastaza Quechua, , Oxford University Press
  • Köhler, W., Gestalt psychology (1929) Psychologische Probleme 1933], , New York Horace Liveright
  • Bremner, A.J., Caparos, S., Davidoff, J., De Fockert, J., Linnell, K.J., Spence, C., “Bouba” and “Kiki” in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners (2013) Cognition, 126, pp. 165-172. , https://doi.org/10.1016/j.cognition.2012.09.007, PMID: 23121711
  • Fort, M., Martin, A., Peperkamp, S., Consonants are more important than vowels in the bouba-kiki effect (2015) Lang Speech, 58, pp. 247-266. , https://doi.org/10.1177/0023830914534951, PMID: 26677645
  • Nielsen, A.K.S., Rendall, D., Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon (2013) Can J Exp Psychol, 67, pp. 153-163. , https://doi.org/10.1037/a0030553, PMID: 23205509
  • Calvert, G.A., Spence, C., Stein, B.E., (2004) The Handboox of Multisensory Processes, , Calvert GA, Spence C, Stein BE, editors. London: Cambridge, Massachusetts
  • Evans, K.K., Treisman, A., Natural cross-modal mappings between visual and auditory features (2010) J Vis, 10, pp. 6.1–12. , https://doi.org/10.1167/10.1.6, PMID: 20143899
  • Shintel, H., Nusbaum, H.C., Okrent, A., Analog acoustic expression in speech communication (2006) J Mem Lang, 55, pp. 167-177. , https://doi.org/10.1016/j.jml.2006.03.002
  • Maurer, D., Pathman, T., Mondloch, C.J., The shape of boubas: Sound–shape correspondences in toddlers and adults (2006) Dev Sci, 9, pp. 316-322. , https://doi.org/10.1111/j.1467-7687.2006.00495.x, Blackwell Publishing Ltd; PMID: 16669803
  • Changizi, M., (2011) Harnessed. How Language and Music Mimicked Nature and Transformed Ape to Man, , Dallas, TX 75231: BenBella Books
  • (1999) Handbook of The International Phonetic Association, , International Phonetic Association. Cambridge: Cambridge University Press
  • Dell, F., (1973) Les Règles Et Les Sons, , Hermann
  • Hubbard, E.M., Arman, A.C., Ramachandran, V.S., Boynton, G.M., Individual differences among grapheme-color synesthetes: Brain-behavior correlations (2005) Neuron, 45, pp. 975-985. , https://doi.org/10.1016/j.neuron.2005.02.008, PMID: 15797557
  • Bankieris, K., Simner, J., What is the link between synaesthesia and sound symbolism? (2015) Cognition, 136, pp. 186-195. , https://doi.org/10.1016/j.cognition.2014.11.013, Elsevier; PMID: 25498744
  • (2016) Cross-Linguistic Onomatopoeias—Wikipedia, The Free Encyclopedia [Internet], , https://en.wikipedia.org/w/index.php?title=Cross-linguistic_onomatopoeias&oldid=755875843, Wikipedia
  • Titze, I.R., (1994) Principles of Voice Production, , New Jersey: Prentice Hall
  • Hauser, M.D., Chomsky, N., Fitch, W.T., The faculty of language: What is it, who has it, and how did it evolve? (2002) Science, 298, pp. 1569-1579. , https://doi.org/10.1126/science.298.5598.1569, PMID: 12446899
  • Assaneo, M.F., Sitt, J., Varoquaux, G., Sigman, M., Cohen, L., Trevisan, M.A., Exploring the anatomical encoding of voice with a mathematical model of the vocal system (2016) Neuroimage, 141, pp. 31-39. , https://doi.org/10.1016/j.neuroimage.2016.07.033, Elsevier B.V.; PMID: 27436593
  • Beauchamp, M.S., Lee, K.E., Argall, B.D., Martin, A., Integration of auditory and visual information about objects in superior temporal sulcus (2004) Neuron, 41, pp. 809-823. , http://www.ncbi.nlm.nih.gov/pubmed/15813999, Available: PMID: 15003179
  • Hashimoto, T., Usui, N., Taira, M., Nose, I., Haji, T., Kojima, S., The neural mechanism associated with the processing of onomatopoeic sounds (2006) Neuroimage, 31, pp. 1762-1770. , https://doi.org/10.1016/j.neuroimage.2006.02.019, PMID: 16616863
  • Brainard, H.D., The psychophysics toolbox (1997) Spat Vis, 10, pp. 433-436. , PMID: 9176952
  • Boersma, P., Weenink, D., (2013) Praat: Doing Phonetics by Computer [Internet], , http://www.praat.org/
  • Powers, D.M.W., Evaluation: From precision, recall and F-Measure to roc, informedness, markedness & correlation (2011) J Mach Learn Technol, 2, pp. 37-63

Citas:

---------- APA ----------
Taitz, A., Florencia Assaneo, M., Elisei, N., Trípodi, M., Cohen, L., Sitt, J.D. & Trevisan, M.A. (2018) . The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation. PLoS ONE, 13(3).
http://dx.doi.org/10.1371/journal.pone.0193466
---------- CHICAGO ----------
Taitz, A., Florencia Assaneo, M., Elisei, N., Trípodi, M., Cohen, L., Sitt, J.D., et al. "The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation" . PLoS ONE 13, no. 3 (2018).
http://dx.doi.org/10.1371/journal.pone.0193466
---------- MLA ----------
Taitz, A., Florencia Assaneo, M., Elisei, N., Trípodi, M., Cohen, L., Sitt, J.D., et al. "The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation" . PLoS ONE, vol. 13, no. 3, 2018.
http://dx.doi.org/10.1371/journal.pone.0193466
---------- VANCOUVER ----------
Taitz, A., Florencia Assaneo, M., Elisei, N., Trípodi, M., Cohen, L., Sitt, J.D., et al. The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation. PLoS ONE. 2018;13(3).
http://dx.doi.org/10.1371/journal.pone.0193466