Stereoelectroencephalography Reveals Neural Signatures of Multisensory Integration in the Human Superior Temporal Sulcus during Audiovisual Speech Perception

Human speech perception is inherently multisensory, relying on the integration of auditory cues from the talker's voice with visual stimuli from their facial movements. Previous studies employing blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) have implicated the superior temporal gyrus (STG) in auditory speech processing and the superior temporal sulcus (STS) in the integration of auditory and visual speech signals. However, fMRI, as an indirect hemodynamic measure, lacks the temporal resolution to capture the rapid neural computations fundamental to speech perception. To overcome these limitations, the present study utilized stereoelectroencephalography (sEEG) to directly record neural activity from the STG and STS regions in 42 epilepsy patients (25 female, 17 male). Participants were tasked with identifying single words presented in three sensory formats: auditory-only, visual-only, and audiovisual speech. These conditions were tested both with and without added auditory noise to evaluate the perceptual benefits of visual speech cues. Notably, observing the talker's face consistently enhanced participants' ability to perceive speech, especially in noisy environments. Neural recordings revealed a subset of electrodes, primarily located in the mid-posterior STG and STS, that responded to both auditory and visual speech signals. Auditory speech responses were detected at an average latency of 71 milliseconds, while visual speech responses occurred slightly later at 109 milliseconds. Crucially, multisensory enhancement was most pronounced in the upper bank of the STS. Compared to auditory-only speech, the STS exhibited a 40% faster response latency and an 18% increase in response amplitude when processing audiovisual speech. In contrast, the STG did not demonstrate significant differences in response speed or magnitude under multisensory conditions. An unexpected finding was that the STS demonstrated significantly faster response latencies for audiovisual speech than the STG (50 ms vs. 64 ms). This suggests a parallel processing model where the STG primarily supports auditory-only speech perception, whereas the STS plays a leading role in integrating multisensory speech information. These neural dynamics underscore the STS as a critical hub for audiovisual speech processing, complementing prior fMRI findings with direct electrophysiological evidence. Together, these results provide compelling support for the superior temporal sulcus as a neural substrate specialized for multisensory integration during speech perception. The application of sEEG allowed for precise temporal tracking of neural responses, revealing distinct and rapid processing pathways in the human temporal cortex. This research not only advances our understanding of the neural mechanisms underlying speech perception but also highlights the importance of multisensory integration in real-world communication scenarios. These insights hold potential implications for clinical interventions in populations with impaired speech perception and the development of neuroprosthetic devices designed to augment communication abilities.