scholarly journals Faces and voices in the brain: a modality-general person-identity representation in superior temporal sulcus

2018 ◽  
Author(s):  
Maria Tsantani ◽  
Nikolaus Kriegeskorte ◽  
Carolyn McGettigan ◽  
Lúcia Garrido
Keyword(s):  
Auditory Processing ◽  
Brain Activity ◽  
Activity Patterns ◽  
Superior Temporal Sulcus ◽  
Brain Regions ◽  
Similarity Analysis ◽  
Representational Similarity Analysis ◽  
Person Identity ◽  
Identity Representation ◽  
The Brain

AbstractFace-selective and voice-selective brain regions have been shown to represent face-identity and voice-identity, respectively. Here we investigated whether there are modality-general person-identity representations in the brain that can be driven by either a face or a voice, and that invariantly represent naturalistically varying face and voice tokens of the same identity. According to two distinct models, such representations could exist either in multimodal brain regions (Campanella and Belin, 2007) or in face-selective brain regions via direct coupling between face- and voice-selective regions (von Kriegstein et al., 2005). To test the predictions of these two models, we used fMRI to measure brain activity patterns elicited by the faces and voices of familiar people in multimodal, face-selective and voice-selective brain regions. We used representational similarity analysis (RSA) to compare the representational geometries of face- and voice-elicited person-identities, and to investigate the degree to which pattern discriminants for pairs of identities generalise from one modality to the other. We found no matching geometries for faces and voices in any brain regions. However, we showed crossmodal generalisation of the pattern discriminants in the multimodal right posterior superior temporal sulcus (rpSTS), suggesting a modality-general person-identity representation in this region. Importantly, the rpSTS showed invariant representations of face- and voice-identities, in that discriminants were trained and tested on independent face videos (different viewpoint, lighting, background) and voice recordings (different vocalizations). Our findings support the Multimodal Processing Model, which proposes that face and voice information is integrated in multimodal brain regions.Significance statementIt is possible to identify a familiar person either by looking at their face or by listening to their voice. Using fMRI and representational similarity analysis (RSA) we show that the right posterior superior sulcus (rpSTS), a multimodal brain region that responds to both faces and voices, contains representations that can distinguish between familiar people independently of whether we are looking at their face or listening to their voice. Crucially, these representations generalised across different particular face videos and voice recordings. Our findings suggest that identity information from visual and auditory processing systems is combined and integrated in the multimodal rpSTS region.

scholarly journals Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis

2016 ◽  
Author(s):  
Jörn Diedrichsen ◽  
Nikolaus Kriegeskorte
Keyword(s):  
Brain Activity ◽  
Activity Patterns ◽  
Similarity Analysis ◽  
Mathematical Framework ◽  
Advantages And Disadvantages ◽  
Representational Similarity Analysis ◽  
Powerful Test ◽  
Activity Measurements ◽  
The Brain ◽  
Representational Models

AbstractRepresentational models specify how activity patterns in populations of neurons (or, more generally, in multivariate brain-activity measurements) relate to sensory stimuli, motor responses, or cognitive processes. In an experimental context, representational models can be defined as hypotheses about the distribution of activity profiles across experimental conditions. Currently, three different methods are being used to test such hypotheses: encoding analysis, pattern component modeling (PCM), and representational similarity analysis (RSA). Here we develop a common mathematical framework for understanding the relationship of these three methods, which share one core commonality: all three evaluate the second moment of the distribution of activity profiles, which determines the representational geometry, and thus how well any feature can be decoded from population activity with any readout mechanism capable of a linear transform. Using simulated data for three different experimental designs, we compare the power of the methods to adjudicate between competing representational models. PCM implements a likelihood-ratio test and therefore provides the most powerful test if its assumptions hold. However, the other two approaches – when conducted appropriately – can perform similarly. In encoding analysis, the linear model needs to be appropriately regularized, which effectively imposes a prior on the activity profiles. With such a prior, an encoding model specifies a well-defined distribution of activity profiles. In RSA, the unequal variances and statistical dependencies of the dissimilarity estimates need to be taken into account to reach near-optimal power in inference. The three methods render different aspects of the information explicit (e.g. single-response tuning in encoding analysis and population-response representational dissimilarity in RSA) and have specific advantages in terms of computational demands, ease of use, and extensibility. The three methods are properly construed as complementary components of a single data-analytical toolkit for understanding neural representations on the basis of multivariate brain-activity data.Author SummaryModern neuroscience can measure activity of many neurons or the local blood oxygenation of many brain locations simultaneously. As the number of simultaneous measurements grows, we can better investigate how the brain represents and transforms information, to enable perception, cognition, and behavior. Recent studies go beyond showing that a brain region is involved in some function. They use representational models that specify how different perceptions, cognitions, and actions are encoded in brain-activity patterns. In this paper, we provide a general mathematical framework for such representational models, which clarifies the relationships between three different methods that are currently used in the neuroscience community. All three methods evaluate the same core feature of the data, but each has distinct advantages and disadvantages. Pattern component modelling (PCM) implements the most powerful test between models, and is analytically tractable and expandable. Representational similarity analysis (RSA) provides a highly useful summary statistic (the dissimilarity) and enables model comparison with weaker distributional assumptions. Finally, encoding models characterize individual responses and enable the study of their layout across cortex. We argue that these methods should be considered components of a larger toolkit for testing hypotheses about the way the brain represents information.

scholarly journals Stimulation-specific information is represented as local activity patterns across the brain

2019 ◽  
Author(s):  
Amirouche Sadoun ◽  
Tushar Chauhan ◽  
Samir Mameri ◽  
Yifan Zhang ◽  
Pascal Barone ◽  
...  
Keyword(s):  
Brain Activity ◽  
Activity Patterns ◽  
Three Dimensional ◽  
Correlation Method ◽  
Brain Regions ◽  
Specific Information ◽  
Spatially Correlated ◽  
Face Area ◽  
Spatial Configurations ◽  
The Brain

AbstractModern neuroimaging represents three-dimensional brain activity, which varies across brain regions. It remains unknown whether activity within brain regions is organized in spatial configurations to reflect perceptual and cognitive processes. We developed a rotational cross-correlation method allowing a straightforward analysis of spatial activity patterns for the precise detection of the spatially correlated distributions of brain activity. Using several statistical approaches, we found that the seed patterns in the fusiform face area were robustly correlated to brain regions involved in face-specific representations. These regions differed from the non-specific visual network meaning that activity structure in the brain is locally preserved in stimulation-specific regions. Our findings indicate spatially correlated perceptual representations in cerebral activity and suggest that the 3D coding of the processed information is organized in locally preserved activity patterns. More generally, our results provide the first demonstration that information is represented and transmitted as local spatial configurations of brain activity.

scholarly journals Faces and voices in the brain: A modality-general person-identity representation in superior temporal sulcus

NeuroImage ◽  
2019 ◽  
Vol 201 ◽  
pp. 116004 ◽  
Author(s):  
Maria Tsantani ◽  
Nikolaus Kriegeskorte ◽  
Carolyn McGettigan ◽  
Lúcia Garrido
Keyword(s):  
Superior Temporal Sulcus ◽  
Person Identity ◽  
Identity Representation ◽  
The Brain

Brain kernel: a new spatial covariance function for fMRI data

2021 ◽  
Author(s):  
Anqi Wu ◽  
Samuel A. Nastase ◽  
Christopher A Baldassano ◽  
Nicholas B Turk-Browne ◽  
Kenneth A. Norman ◽  
...  
Keyword(s):  
Covariance Function ◽  
Brain Activity ◽  
Activity Patterns ◽  
Bilateral Symmetry ◽  
Resting State Fmri ◽  
Brain Regions ◽  
Fmri Data ◽  
Spatial Covariance ◽  
Spatial Correlation Structure ◽  
The Brain

A key problem in functional magnetic resonance imaging (fMRI) is to estimate spatial activity patterns from noisy high-dimensional signals. Spatial smoothing provides one approach to regularizing such estimates. However, standard smoothing methods ignore the fact that correlations in neural activity may fall off at different rates in different brain areas, or exhibit discontinuities across anatomical or functional boundaries. Moreover, such methods do not exploit the fact that widely separated brain regions may exhibit strong correlations due to bilateral symmetry or the network organization of brain regions. To capture this non-stationary spatial correlation structure, we introduce the brain kernel, a continuous covariance function for whole-brain activity patterns. We define the brain kernel in terms of a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, parametrized with a Gaussian process (GP). The brain kernel specifies the prior covariance between voxels as a function of the distance between their locations in embedding space. The GP mapping warps the brain nonlinearly so that highly correlated voxels are close together in latent space, and uncorrelated voxels are far apart. We estimate the brain kernel using resting-state fMRI data, and we develop an exact, scalable inference method based on block coordinate descent to overcome the challenges of high dimensionality (10-100K voxels). Finally, we illustrate the brain kernel's usefulness with applications to brain decoding and factor analysis with multiple task-based fMRI datasets.

scholarly journals Microsleep is associated with brain activity patterns unperturbed by auditory inputs

2019 ◽  
Vol 122 (6) ◽  
pp. 2568-2575
Author(s):  
Zixin Yong ◽  
Joo Huang Tan ◽  
Po-Jang Hsieh
Keyword(s):  
Auditory Cortex ◽  
Brain Activity ◽  
Activity Patterns ◽  
Brain Regions ◽  
Auditory Stimulation ◽  
Simple Task ◽  
Highly Active ◽  
Early Afternoon ◽  
Mri Scans ◽  
The Brain

Microsleeps are brief episodes of arousal level decrease manifested through behavioral signs. Brain activity during microsleep in the presence of external stimulus remains poorly understood. In this study, we sought to understand neural responses to auditory stimulation during microsleep. We gave participants the simple task of listening to audios of different pitches and amplitude modulation frequencies during early afternoon functional MRI scans. We found the following: 1) microsleep was associated with cortical activations in broad motor and sensory regions and deactivations in thalamus, irrespective of auditory stimulation; 2) high and low pitch audios elicited different activity patterns in the auditory cortex during awake but not microsleep state; and 3) during microsleep, spatial activity patterns in broad brain regions were similar regardless of the presence or types of auditory stimulus (i.e., stimulus invariant). These findings show that the brain is highly active during microsleep but the activity patterns across broad regions are unperturbed by auditory inputs. NEW & NOTEWORTHY During deep drowsy states, auditory inputs could induce activations in the auditory cortex, but the activation patterns lose differentiation to high/low pitch stimuli. Instead of random activations, activity patterns across the brain during microsleep appear to be structured and may reflect underlying neurophysiological processes that remain unclear.

scholarly journals Meaningful patterns of information in the brain revealed through analysis of errors

2019 ◽  
Author(s):  
Alexandra Woolgar ◽  
Nadene Dermody ◽  
Soheil Afshar ◽  
Mark A. Williams ◽  
Anina N. Rich
Keyword(s):  
Visual Cortex ◽  
Brain Activity ◽  
Activity Patterns ◽  
Brain Regions ◽  
Stimulus Response ◽  
Correct Rule ◽  
Frontoparietal Cortex ◽  
Response Task ◽  
Incorrect Information ◽  
The Brain

SummaryGreat excitement has surrounded our ability to decode task information from human brain activity patterns, reinforcing the dominant view of the brain as an information processor. We tested a fundamental but overlooked assumption: that such decodable information is actually used by the brain to generate cognition and behaviour. Participants performed a challenging stimulus-response task during fMRI. Our novel analyses trained a pattern classifier on data from correct trials, and used it to examine stimulus and rule coding on error trials. There was a striking interaction in which frontoparietal cortex systematically represented incorrect rule but correct stimulus information when participants used the wrong rule, and incorrect stimulus but correct rule information on other types of errors. Visual cortex, by contrast, did not code correct or incorrect information on error. Thus behaviour was tightly linked to coding in frontoparietal cortex and only weakly linked to coding in visual cortex. Human behaviour may indeed result from information-like patterns of activity in the brain, but this relationship is stronger in some brain regions than in others. Testing for information coding on error can help establish which patterns constitute behaviourally-meaningful information.

scholarly journals Harmonic Brain Modes: A Unifying Framework for Linking Space and Time in Brain Dynamics

2017 ◽  
Vol 24 (3) ◽  
pp. 277-293 ◽  
Author(s):  
Selen Atasoy ◽  
Gustavo Deco ◽  
Morten L. Kringelbach ◽  
Joel Pearson
Keyword(s):  
Neural Activity ◽  
Brain Activity ◽  
Activity Patterns ◽  
Structural Connectivity ◽  
Building Blocks ◽  
Mental States ◽  
Brain Dynamics ◽  
Space And Time ◽  
Wide Range ◽  
The Brain

A fundamental characteristic of spontaneous brain activity is coherent oscillations covering a wide range of frequencies. Interestingly, these temporal oscillations are highly correlated among spatially distributed cortical areas forming structured correlation patterns known as the resting state networks, although the brain is never truly at “rest.” Here, we introduce the concept of harmonic brain modes—fundamental building blocks of complex spatiotemporal patterns of neural activity. We define these elementary harmonic brain modes as harmonic modes of structural connectivity; that is, connectome harmonics, yielding fully synchronous neural activity patterns with different frequency oscillations emerging on and constrained by the particular structure of the brain. Hence, this particular definition implicitly links the hitherto poorly understood dimensions of space and time in brain dynamics and its underlying anatomy. Further we show how harmonic brain modes can explain the relationship between neurophysiological, temporal, and network-level changes in the brain across different mental states ( wakefulness, sleep, anesthesia, psychedelic). Notably, when decoded as activation of connectome harmonics, spatial and temporal characteristics of neural activity naturally emerge from the interplay between excitation and inhibition and this critical relation fits the spatial, temporal, and neurophysiological changes associated with different mental states. Thus, the introduced framework of harmonic brain modes not only establishes a relation between the spatial structure of correlation patterns and temporal oscillations (linking space and time in brain dynamics), but also enables a new dimension of tools for understanding fundamental principles underlying brain dynamics in different states of consciousness.

scholarly journals Prefrontal High Gamma in ECoG tags periodicity of musical rhythms in perception and imagination

2019 ◽  
Author(s):  
S. A. Herff ◽  
C. Herff ◽  
A. J. Milne ◽  
G. D. Johnson ◽  
J. J. Shih ◽  
...  
Keyword(s):  
Auditory Processing ◽  
Right Hemisphere ◽  
Brain Activity ◽  
Activity Patterns ◽  
Superior Temporal Gyrus ◽  
Gamma Activity ◽  
Rhythm Perception ◽  
High Gamma ◽  
The Right ◽  
Musical Rhythms

AbstractRhythmic auditory stimuli are known to elicit matching activity patterns in neural populations. Furthermore, recent research has established the particular importance of high-gamma brain activity in auditory processing by showing its involvement in auditory phrase segmentation and envelope-tracking. Here, we use electrocorticographic (ECoG) recordings from eight human listeners, to see whether periodicities in high-gamma activity track the periodicities in the envelope of musical rhythms during rhythm perception and imagination. Rhythm imagination was elicited by instructing participants to imagine the rhythm to continue during pauses of several repetitions. To identify electrodes whose periodicities in high-gamma activity track the periodicities in the musical rhythms, we compute the correlation between the autocorrelations (ACC) of both the musical rhythms and the neural signals. A condition in which participants listened to white noise was used to establish a baseline. High-gamma autocorrelations in auditory areas in the superior temporal gyrus and in frontal areas on both hemispheres significantly matched the autocorrelation of the musical rhythms. Overall, numerous significant electrodes are observed on the right hemisphere. Of particular interest is a large cluster of electrodes in the right prefrontal cortex that is active during both rhythm perception and imagination. This indicates conscious processing of the rhythms’ structure as opposed to mere auditory phenomena. The ACC approach clearly highlights that high-gamma activity measured from cortical electrodes tracks both attended and imagined rhythms.

scholarly journals Neural evidence for the prediction of animacy features during language comprehension: Evidence from MEG and EEG Representational Similarity Analysis

2019 ◽  
Author(s):  
Lin Wang ◽  
Edward Wlotko ◽  
Edward Alexander ◽  
Lotte Schoot ◽  
Minjae Kim ◽  
...  
Keyword(s):  
Spatial Patterns ◽  
Language Comprehension ◽  
Neural Activity ◽  
Semantic Processing ◽  
Sentence Comprehension ◽  
Brain Activity ◽  
Coarse Grained ◽  
Similarity Analysis ◽  
Semantic Features ◽  
Representational Similarity Analysis

AbstractIt has been proposed that people can generate probabilistic predictions at multiple levels of representation during language comprehension. We used Magnetoencephalography (MEG) and Electroencephalography (EEG), in combination with Representational Similarity Analysis (RSA), to seek neural evidence for the prediction of animacy features. In two studies, MEG and EEG activity was measured as human participants (both sexes) read three-sentence scenarios. Verbs in the final sentences constrained for either animate or inanimate semantic features of upcoming nouns, and the broader discourse context constrained for either a specific noun or for multiple nouns belonging to the same animacy category. We quantified the similarity between spatial patterns of brain activity following the verbs until just before the presentation of the nouns. The MEG and EEG datasets revealed converging evidence that the similarity between spatial patterns of neural activity following animate constraining verbs was greater than following inanimate constraining verbs. This effect could not be explained by lexical-semantic processing of the verbs themselves. We therefore suggest that it reflected the inherent difference in the semantic similarity structure of the predicted animate and inanimate nouns. Moreover, the effect was present regardless of whether a specific word could be predicted, providing strong evidence for the prediction of coarse-grained semantic features that goes beyond the prediction of individual words.Significance statementLanguage inputs unfold very quickly during real-time communication. By predicting ahead we can give our brains a “head-start”, so that language comprehension is faster and more efficient. While most contexts do not constrain strongly for a specific word, they do allow us to predict some upcoming information. For example, following the context, “they cautioned the…”, we can predict that the next word will be animate rather than inanimate (we can caution a person, but not an object). Here we used EEG and MEG techniques to show that the brain is able to use these contextual constraints to predict the animacy of upcoming words during sentence comprehension, and that these predictions are associated with specific spatial patterns of neural activity.

scholarly journals Resting-state brain activity can predict target-independent aptitude in fMRI-neurofeedback training

2021 ◽  
Author(s):  
Takashi Nakano ◽  
Masahiro Takamura ◽  
Haruki Nishimura ◽  
Maro Machizawa ◽  
Naho Ichikawa ◽  
...  
Keyword(s):  
Resting State ◽  
Brain Connectivity ◽  
Brain Activity ◽  
Resting State Fmri ◽  
Brain Regions ◽  
Posterior Cingulate Cortex ◽  
Fmri Data ◽  
Independent Test ◽  
The Individual ◽  
The Brain

AbstractNeurofeedback (NF) aptitude, which refers to an individual’s ability to change its brain activity through NF training, has been reported to vary significantly from person to person. The prediction of individual NF aptitudes is critical in clinical NF applications. In the present study, we extracted the resting-state functional brain connectivity (FC) markers of NF aptitude independent of NF-targeting brain regions. We combined the data in fMRI-NF studies targeting four different brain regions at two independent sites (obtained from 59 healthy adults and six patients with major depressive disorder) to collect the resting-state fMRI data associated with aptitude scores in subsequent fMRI-NF training. We then trained the regression models to predict the individual NF aptitude scores from the resting-state fMRI data using a discovery dataset from one site and identified six resting-state FCs that predicted NF aptitude. Next we validated the prediction model using independent test data from another site. The result showed that the posterior cingulate cortex was the functional hub among the brain regions and formed predictive resting-state FCs, suggesting NF aptitude may be involved in the attentional mode-orientation modulation system’s characteristics in task-free resting-state brain activity.

Sign in / Sign up

Export Citation Format

Share Document