Inter-individual deep image reconstruction

The sensory cortex is characterized by general organizational principles such as topography and hierarchy. However, measured brain activity given identical input exhibits substantially different patterns across individuals. While anatomical and functional alignment methods have been proposed in functional magnetic resonance imaging (fMRI) studies, it remains unclear whether and how hierarchical and fine-grained representations can be converted between individuals while preserving the encoded perceptual contents. In this study, we evaluated machine learning models called neural code converters that predict one's brain activity pattern (target) from another's (source) given the same stimulus by the decoding of hierarchical visual features and the reconstruction of perceived images. The training data for converters consisted of fMRI data obtained with identical sets of natural images presented to pairs of individuals. Converters were trained using the whole visual cortical voxels from V1 through the ventral object areas, without explicit labels of visual areas. We decoded the converted brain activity patterns into hierarchical visual features of a deep neural network (DNN) using decoders pre-trained on the target brain and then reconstructed images via the decoded features. Without explicit information about visual cortical hierarchy, the converters automatically learned the correspondence between the visual areas of the same levels. DNN feature decoding at each layer showed higher decoding accuracies from corresponding levels of visual areas, indicating that hierarchical representations were preserved after conversion. The viewed images were faithfully reconstructed with recognizable silhouettes of objects even with relatively small amounts of data for converter training. The conversion also allows pooling data across multiple individuals, leading to stably high reconstruction accuracy compared to those converted between individuals. These results demonstrate that the conversion learns hierarchical correspondence and preserves the fine-grained representations of visual features, enabling visual image reconstruction using decoders trained on other individuals.

Wrist and finger motor representations embedded in the cerebral and cerebellar resting-state activation

Several functional magnetic resonance imaging (fMRI) studies have demonstrated that resting-state brain activity consists of multiple components, each corresponding to the spatial pattern of brain activity induced by performing a task. Especially in a movement task, such components have been shown to correspond to the brain activity pattern of the relevant anatomical region, meaning that the voxels of pattern that are cooperatively activated while using a body part (e.g., foot, hand, and tongue) also behave cooperatively in the resting state. However, it is unclear whether the components involved in resting-state brain activity correspond to those induced by the movement of discrete body parts. To address this issue, in the present study, we focused on wrist and finger movements in the hand, and a cross-decoding technique trained to discriminate between the multi-voxel patterns induced by wrist and finger movement was applied to the resting-state fMRI. We found that the multi-voxel pattern in resting-state brain activity corresponds to either wrist or finger movements in the motor-related areas of each hemisphere of the cerebrum and cerebellum. These results suggest that resting-state brain activity in the motor-related areas consists of the components corresponding to the elementary movements of individual body parts. Therefore, the resting-state brain activity possibly has a finer structure than considered previously.

supFunSim: Spatial Filtering Toolbox for EEG

Brain activity pattern recognition from EEG or MEG signal analysis is one of the most important method in cognitive neuroscience. The supFunSim library is a new Matlab toolbox which generates accurate EEG forward model and implements a collection of spatial filters for EEG source reconstruction, including the linearly constrained minimum-variance (LCMV), eigenspace LCMV, nulling (NL), and minimum-variance pseudo-unbiased reduced-rank (MV-PURE) filters in various versions. It also enables source-level directed connectivity analysis using partial directed coherence (PDC) measure. The supFunSim library is based on the well-known FieldTrip toolbox for EEG and MEG analysis and is written using object-oriented programming paradigm. The resulting modularity of the toolbox enables its simple extensibility. This paper gives a complete overview of the toolbox from both developer and end-user perspectives, including description of the installation process and use cases.

The neural representation of self is recapitulated in the brains of friends: A round-robin fMRI study

Humans continually form and update impressions of each other’s identities based on the disclosure of thoughts, feelings, and beliefs. At the same time, individuals also have specific beliefs and knowledge about their own self-concept. Over a decade of social neuroscience research has shown that retrieving information about the self and about other persons recruits similar areas of the medial prefrontal cortex (MPFC), however it remains unclear if an individual’s neural representation of self is reflected in the brains of well-known others or if instead the two representations share no common relationship. Here we examined this question in a tight-knit network of friends as they engaged in a round-robin trait evaluation task in which each participant was both perceiver and target for every other participant and in addition also evaluated their self. Using functional magnetic resonance imaging and a multilevel modeling approach, we show that multivoxel brain activity patterns in the MPFC during a person’s self-referential thought are correlated with those of friends when thinking of that same person. Moreover, the similarity of neural self/other patterns was itself positively associated with the similarity of self/other trait judgments ratings as measured behaviorally in a separate session. These findings suggest that accuracy in person perception may be predicated on the degree to which the brain activity pattern associated with an individual thinking about their own self-concept is similarly reflected in the brains of others.