University of Canterbury Home
    • Admin
    UC Research Repository
    UC Library
    JavaScript is disabled for your browser. Some features of this site may not work without it.
    View Item 
    1. UC Home
    2. Library
    3. UC Research Repository
    4. UC Research Centres
    5. NZ Institute of Language, Brain and Behaviour
    6. NZILBB: Journal Articles
    7. View Item
    1. UC Home
    2.  > 
    3. Library
    4.  > 
    5. UC Research Repository
    6.  > 
    7. UC Research Centres
    8.  > 
    9. NZ Institute of Language, Brain and Behaviour
    10.  > 
    11. NZILBB: Journal Articles
    12.  > 
    13. View Item

    Tri-modal Speech: Audio-visual-tactile Integration in Speech Perception (2019)

    Thumbnail
    View/Open
    Derrick_J. Acoust. Soc. Am_2019_146_Tri-modal.pdf (1.710Mb)
    Type of Content
    Journal Article
    UC Permalink
    http://hdl.handle.net/10092/17714
    
    Publisher's DOI/URI
    https://doi.org/10.1121/1.5134064
    
    ISSN
    0001-4966
    Collections
    • NZILBB: Journal Articles [18]
    Authors
    Derrick, Donald cc
    Hansmann D
    Theys C
    show all
    Abstract

    Speech perception is a multi-sensory experience. Visual information enhances [Sumby and Pollack (1954). J. Acoust. Soc. Am. 25, 212–215] and interferes [McGurk and MacDonald (1976). Nature 264, 746–748] with speech perception. Similarly, tactile information, transmitted by puffs of air arriving at the skin and aligned with speech audio, alters [Gick and Derrick (2009). Nature 462, 502–504] auditory speech perception in noise. It has also been shown that aero-tactile information influences visual speech perception when an auditory signal is absent [Derrick, Bicevskis, and Gick (2019a). Front. Commun. Lang. Sci. 3(61), 1–11]. However, researchers have not yet identified the combined influence of aero-tactile, visual, and auditory information on speech perception. The effects of matching and mismatching visual and tactile speech on two-way forced-choice auditory syllable-in-noise classification tasks were tested. The results showed that both visual and tactile information altered the signal-to-noise threshold for accurate identification of auditory signals. Similar to previous studies, the visual component has a strong influence on auditory syllable-in-noise identification, as evidenced by a 28.04dB improvement in SNR between matching and mismatching visual stimulus presentations. In comparison, the tactile component had a small influence resulting in a 1.58dB SNR match-mismatch range. The effects of both the audio and tactile information were shown to be additive.

    Citation
    Donald Derrick, Doreen Hansmann, Catherine Theys. (2019) Tri-modal speech: Audio-visual-tactile integration in speech perception. The Journal of the Acoustical Society of America 146:5, 3495-3504.
    This citation is automatically generated and may be unreliable. Use as a guide only.
    ANZSRC Fields of Research
    17 - Psychology and Cognitive Sciences::1702 - Cognitive Science::170204 - Linguistic Processes (incl. Speech Production and Comprehension)

    Related items

    Showing items related by title, author, creator and subject.

    • Visual-tactile integration in speech perception : evidence for modality neutral speech primitives. 

      Bicevskis K; Derrick, Donald; Gick B (Acoustical Society of America (ASA), 2016)
      © 2016 Acoustical Society of America. Audio-visual [McGurk and MacDonald (1976). Nature 264, 746-748] and audio-tactile [Gick and Derrick (2009). Nature 462(7272), 502-504] speech stimuli enhance speech perception over ...
    • Visual-Tactile Speech Perception and the Autism Quotient 

      Derrick, Donald; Bicevskis K; Gick B (Frontiers Media SA, 2019)
      Multisensory information is integrated asymmetrically in speech perception: An audio signal can follow video by 240ms, but can precede video by only 60ms, without disrupting the sense of synchronicity (Munhall et al., ...
    • Audio-Visual-Tactile integration in speech perception. 

      Derrick, Donald; Hansmann D; Haws Z; Theys C (2018)
      Behavioural audio-visual research has shown enhancement1 and interference2 in speech perception, as has behavioural audio-tactile research3. However, to date, we have not encountered any experimental behavioural research ...
    Advanced Search

    Browse

    All of the RepositoryCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThesis DisciplineThis CollectionBy Issue DateAuthorsTitlesSubjectsThesis Discipline

    Statistics

    View Usage Statistics
    • SUBMISSIONS
    • Research Outputs
    • UC Theses
    • CONTACTS
    • Send Feedback
    • +64 3 369 3853
    • ucresearchrepository@canterbury.ac.nz
    • ABOUT
    • UC Research Repository Guide
    • Copyright and Disclaimer
    • SUBMISSIONS
    • Research Outputs
    • UC Theses
    • CONTACTS
    • Send Feedback
    • +64 3 369 3853
    • ucresearchrepository@canterbury.ac.nz
    • ABOUT
    • UC Research Repository Guide
    • Copyright and Disclaimer