scholarly journals Exploring spatio-temporal neural dynamics of the human visual cortex

2018 ◽  
Author(s):  
Ying Yang ◽  
Michael J. Tarr ◽  
Robert E. Kass ◽  
Elissa M. Aminoff
Keyword(s):  
Neural Network ◽  
Visual Cortex ◽  
Information Flow ◽  
Residual Layer ◽  
Low Level ◽  
Early Visual Cortex ◽  
The Common ◽  
Spatio Temporal ◽  
High Level ◽  
Common Components

AbstractThe human visual cortex is organized in a hierarchical manner. Although a significant body of evidence has been accumulated in support of this hypothesis, specific details regarding the spatial and temporal information flow remain open. Here we present detailed spatio-temporal correlation profiles of neural activity with low-level and high-level features derived from a “deep” (8-layer) neural network pre-trained for object recognition. These correlation profiles indicate an early-to-late shift from low-level features to high-level features and from low-level regions to higher-level regions along the visual hierarchy, consistent with feedforward information flow. To refine our understanding of information flow, we computed three sets of features from the low-and high-level features provided by the neural network: object-category-relevant low-level features (the common components between low-level and high-level features), low-level features roughly orthogonal to high-level features (the residual Layer 1 features), and unique high-level features that were roughly orthogonal to low-level features (the residual Layer 7 features). Contrasting the correlation effects of the common components and the residual Layer 1 features, we observed that the early visual cortex exhibits a similar amount of correlation with the two feature sets early in time (60 to 120 ms), but in a later time window, the early visual cortex exhibits a higher and longer correlation effect with the common components/low-level task-relevant features as compared to the low-level residual features—an effect unlikely to arise from purely feedforward information flow. Overall, our results indicate that non-feedforward processes, for example, top-down influences from mental representations of categories, may facilitate differentiation between these two types of low-level features within the early visual cortex.

scholarly journals Multivariate pattern analysis of MEG and EEG: a comparison of representational structure in time and space

2016 ◽  
Author(s):  
Radoslaw Martin Cichy ◽  
Dimitrios Pantazis
Keyword(s):  
Visual Cortex ◽  
Visual Processing ◽  
Pattern Analysis ◽  
Temporal Dynamics ◽  
Multivariate Pattern Analysis ◽  
Eeg Data ◽  
Early Visual Cortex ◽  
Spatio Temporal ◽  
High Level ◽  
Multivariate Pattern

1AbstractMultivariate pattern analysis of magnetoencephalography (MEG) and electroencephalography (EEG) data can reveal the rapid neural dynamics underlying cognition. However, MEG and EEG have systematic differences in sampling neural activity. This poses the question to which degree such measurement differences consistently bias the results of multivariate analysis applied to MEG and EEG activation patterns. To investigate, we conducted a concurrent MEG/EEG study while participants viewed images of everyday objects. We applied multivariate classification analyses to MEG and EEG data, and compared the resulting time courses to each other, and to fMRI data for an independent evaluation in space. We found that both MEG and EEG revealed the millisecond spatio-temporal dynamics of visual processing with largely equivalent results. Beyond yielding convergent results, we found that MEG and EEG also captured partly unique aspects of visual representations. Those unique components emerged earlier in time for MEG than for EEG. Identifying the sources of those unique components with fMRI, we found the locus for both MEG and EEG in high-level visual cortex, and in addition for MEG in early visual cortex. Together, our results show that multivariate analyses of MEG and EEG data offer a convergent and complimentary view on neural processing, and motivate the wider adoption of these methods in both MEG and EEG research.

scholarly journals PedNet: A Spatio-Temporal Deep Convolutional Neural Network for Pedestrian Segmentation

2018 ◽  
Vol 4 (9) ◽  
pp. 107 ◽  
Author(s):  
Mohib Ullah ◽  
Ahmed Mohammed ◽  
Faouzi Alaya Cheikh
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Performance Metrics ◽  
State Of The Art ◽  
Temporal Information ◽  
Feature Maps ◽  
Current Frame ◽  
Low Level ◽  
Art Methods ◽  
Spatio Temporal

Articulation modeling, feature extraction, and classification are the important components of pedestrian segmentation. Usually, these components are modeled independently from each other and then combined in a sequential way. However, this approach is prone to poor segmentation if any individual component is weakly designed. To cope with this problem, we proposed a spatio-temporal convolutional neural network named PedNet which exploits temporal information for spatial segmentation. The backbone of the PedNet consists of an encoder–decoder network for downsampling and upsampling the feature maps, respectively. The input to the network is a set of three frames and the output is a binary mask of the segmented regions in the middle frame. Irrespective of classical deep models where the convolution layers are followed by a fully connected layer for classification, PedNet is a Fully Convolutional Network (FCN). It is trained end-to-end and the segmentation is achieved without the need of any pre- or post-processing. The main characteristic of PedNet is its unique design where it performs segmentation on a frame-by-frame basis but it uses the temporal information from the previous and the future frame for segmenting the pedestrian in the current frame. Moreover, to combine the low-level features with the high-level semantic information learned by the deeper layers, we used long-skip connections from the encoder to decoder network and concatenate the output of low-level layers with the higher level layers. This approach helps to get segmentation map with sharp boundaries. To show the potential benefits of temporal information, we also visualized different layers of the network. The visualization showed that the network learned different information from the consecutive frames and then combined the information optimally to segment the middle frame. We evaluated our approach on eight challenging datasets where humans are involved in different activities with severe articulation (football, road crossing, surveillance). The most common CamVid dataset which is used for calculating the performance of the segmentation algorithm is evaluated against seven state-of-the-art methods. The performance is shown on precision/recall, F 1 , F 2 , and mIoU. The qualitative and quantitative results show that PedNet achieves promising results against state-of-the-art methods with substantial improvement in terms of all the performance metrics.

scholarly journals Reconstructing feedback representations in ventral visual pathway with a generative adversarial autoencoder

2020 ◽  
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh
Keyword(s):  
Neural Network ◽  
Visual Information ◽  
Visual Pathway ◽  
Functional Magnetic Resonance ◽  
Low Level ◽  
Ventral Visual Pathway ◽  
High Level ◽  
Feedback Connections ◽  
The Brain ◽  
Insight Into

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.

scholarly journals A High-Density Crowd Counting Method Based on Convolutional Feature Fusion

2018 ◽  
Vol 8 (12) ◽  
pp. 2367 ◽  
Author(s):  
Hongling Luo ◽  
Jun Sang ◽  
Weiqun Wu ◽  
Hong Xiang ◽  
Zhili Xiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Large Scale ◽  
Feature Fusion ◽  
High Density ◽  
Counting Problem ◽  
Crowd Counting ◽  
Low Level ◽  
Density Map ◽  
High Level

In recent years, the trampling events due to overcrowding have occurred frequently, which leads to the demand for crowd counting under a high-density environment. At present, there are few studies on monitoring crowds in a large-scale crowded environment, while there exists technology drawbacks and a lack of mature systems. Aiming to solve the crowd counting problem with high-density under complex environments, a feature fusion-based deep convolutional neural network method FF-CNN (Feature Fusion of Convolutional Neural Network) was proposed in this paper. The proposed FF-CNN mapped the crowd image to its crowd density map, and then obtained the head count by integration. The geometry adaptive kernels were adopted to generate high-quality density maps which were used as ground truths for network training. The deconvolution technique was used to achieve the fusion of high-level and low-level features to get richer features, and two loss functions, i.e., density map loss and absolute count loss, were used for joint optimization. In order to increase the sample diversity, the original images were cropped with a random cropping method for each iteration. The experimental results of FF-CNN on the ShanghaiTech public dataset showed that the fusion of low-level and high-level features can extract richer features to improve the precision of density map estimation, and further improve the accuracy of crowd counting.

scholarly journals Active Shock Absorber Control Based on Time-Delay Neural Network

Energies ◽  
2020 ◽  
Vol 13 (5) ◽  
pp. 1091
Author(s):  
Alexander Alyukov ◽  
Yuri Rozhdestvenskiy ◽  
Sergei Aliukov
Keyword(s):  
Neural Network ◽  
Time Delay ◽  
Shock Absorber ◽  
Learning Ability ◽  
Control Input ◽  
Low Level ◽  
External Data ◽  
Good Learning ◽  
Suspension Model ◽  
High Level

A controlled suspension usually consists of a high-level and a low-level controller. The purpose the high-level controller is to analyze external data on vehicle conditions and make decisions on the required value of the force on the shock absorber rod, while the purpose of the low-level controller is to ensure the implementation of the desired force value by controlling the actuators. Many works have focused on the design of high-level controllers of active suspensions, in which it is considered that the shock absorber can instantly and absolutely accurately implement a given control input. However, active shock absorbers are complex systems that have hysteresis. In addition, electro-pneumatic and hydraulic elements are often used in the design, which have a long response time and often low accuracy. The application of methods of control theory in such systems is often difficult due to the complexity of constructing their mathematical models. In this article, the authors propose an effective low-level controller for an active shock absorber based on a time-delay neural network. Neural networks in this case show good learning ability. The low-level controller is implemented in a simplified suspension model and the simulation results are presented for a number of typical cases.

scholarly journals Neural Correlates of Motion-induced Blindness in the Human Brain

2010 ◽  
Vol 22 (6) ◽  
pp. 1235-1243 ◽  
Author(s):  
Marieke L. Schölvinck ◽  
Geraint Rees
Keyword(s):  
Visual Cortex ◽  
Visual Processing ◽  
Target Location ◽  
Visual Awareness ◽  
Neural Correlates ◽  
Low Level ◽  
Bistable Perception ◽  
Visual Phenomenon ◽  
High Level ◽  
Visual Cortical

Motion-induced blindness (MIB) is a visual phenomenon in which highly salient visual targets spontaneously disappear from visual awareness (and subsequently reappear) when superimposed on a moving background of distracters. Such fluctuations in awareness of the targets, although they remain physically present, provide an ideal paradigm to study the neural correlates of visual awareness. Existing behavioral data on MIB are consistent both with a role for structures early in visual processing and with involvement of high-level visual processes. To further investigate this issue, we used high field functional MRI to investigate signals in human low-level visual cortex and motion-sensitive area V5/MT while participants reported disappearance and reappearance of an MIB target. Surprisingly, perceptual invisibility of the target was coupled to an increase in activity in low-level visual cortex plus area V5/MT compared with when the target was visible. This increase was largest in retinotopic regions representing the target location. One possibility is that our findings result from an active process of completion of the field of distracters that acts locally in the visual cortex, coupled to a more global process that facilitates invisibility in general visual cortex. Our findings show that the earliest anatomical stages of human visual cortical processing are implicated in MIB, as with other forms of bistable perception.

scholarly journals InteractionNN: A Neural Network for Learning Hidden Features in Sparse Prediction

2019 ◽  
Author(s):  
Xiaowang Zhang ◽  
Qiang Gao ◽  
Zhiyong Feng
Keyword(s):  
Neural Network ◽  
Nonlinear Interaction ◽  
Feature Interaction ◽  
Feed Forward Neural Network ◽  
Low Level ◽  
Feature Interactions ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
High Level ◽  
Dense Features

In this paper, we present a neural network (InteractionNN) for sparse predictive analysis where hidden features of sparse data can be learned by multilevel feature interaction. To characterize multilevel interaction of features, InteractionNN consists of three modules, namely, nonlinear interaction pooling, layer-lossing, and embedding. Nonlinear interaction pooling (NI pooling) is a hierarchical structure and, by shortcut connection, constructs low-level feature interactions from basic dense features to elementary features. Layer-lossing is a feed-forward neural network where high-level feature interactions can be learned from low-level feature interactions via correlation of all layers with target. Moreover, embedding is to extract basic dense features from sparse features of data which can help in reducing our proposed model computational complex. Finally, our experiment evaluates on the two benchmark datasets and the experimental results show that InteractionNN performs better than most of state-of-the-art models in sparse regression.

scholarly journals Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition

2017 ◽  
Author(s):  
Xinge Zhu ◽  
Liang Li ◽  
Weigang Zhang ◽  
Tianrong Rao ◽  
Min Xu ◽  
...  
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Feature Fusion ◽  
Feature Representation ◽  
Low Level ◽  
Learning Framework ◽  
Independent Entity ◽  
Internet Images ◽  
High Level ◽  
Different Levels

Visual emotion recognition aims to associate images with appropriate emotions. There are different visual stimuli that can affect human emotion from low-level to high-level, such as color, texture, part, object, etc. However, most existing methods treat different levels of features as independent entity without having effective method for feature fusion. In this paper, we propose a unified CNN-RNN model to predict the emotion based on the fused features from different levels by exploiting the dependency among them. Our proposed architecture leverages convolutional neural network (CNN) with multiple layers to extract different levels of features with in a multi-task learning framework, in which two related loss functions are introduced to learn the feature representation. Considering the dependencies within the low-level and high-level features, a new bidirectional recurrent neural network (RNN) is proposed to integrate the learned features from different layers in the CNN model. Extensive experiments on both Internet images and art photo datasets demonstrate that our method outperforms the state-of-the-art methods with at least 7% performance improvement.

Sign in / Sign up

Export Citation Format

Share Document