scholarly journals Spontaneous generation of face recognition in untrained deep neural networks

2019 ◽  
Author(s):  
Seungdae Baek ◽  
Min Song ◽  
Jaeson Jang ◽  
Gwangsu Kim ◽  
Se-Bum Paik
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Visual Experience ◽  
Spontaneous Generation ◽  
Biologically Inspired ◽  
Weight Variation ◽  
Statistical Variation ◽  
Face Selectivity ◽  
Point Invariant ◽  
The Brain

AbstractFace-selective neurons are observed in the primate visual pathway and are considered the basis of facial recognition in the brain. However, it is debated whether this neuronal selectivity can arise spontaneously, or requires training from visual experience. Here, we show that face-selective neurons arise spontaneously in random feedforward networks in the absence of learning. Using biologically inspired deep neural networks, we found that face-selective neurons arise under three different network conditions: one trained using non-face natural images, one randomized after being trained, and one never trained. We confirmed that spontaneously emerged face-selective neurons show the biological view-point-invariant characteristics observed in monkeys. Such neurons suddenly vanished when feedforward weight variation declined to a certain level. Our results suggest that innate face-selectivity originates from statistical variation of the feedforward projections in hierarchical neural networks.

scholarly journals Spontaneous generation of innate number sense in untrained deep neural networks

2019 ◽  
Author(s):  
Gwangsu Kim ◽  
Jaeson Jang ◽  
Seungdae Baek ◽  
Min Song ◽  
Se-Bum Paik
Keyword(s):  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Number Sense ◽  
Visual Pathway ◽  
Natural Images ◽  
Spontaneous Generation ◽  
Biologically Inspired ◽  
Weight Variation ◽  
Statistical Variation ◽  
The Brain

AbstractNumber-selective neurons are observed in numerically naïve animals, but it was not understood how this innate function emerges in the brain. Here, we show that neurons tuned to numbers can arise in random feedforward networks, even in the complete absence of learning. Using a biologically inspired deep neural network, we found that number tuning arises in three cases of networks: one trained to non-numerical natural images, one randomized after trained, and one never trained. Number-tuned neurons showed characteristics that were observed in the brain following the Weber-Fechner law. These neurons suddenly vanished when the feedforward weight variation decreased to a certain level. These results suggest that number tuning can develop from the statistical variation of bottom-up projections in the visual pathway, initializing innate number sense.

scholarly journals Face detection in untrained deep neural networks

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Seungdae Baek ◽  
Min Song ◽  
Jaeson Jang ◽  
Gwangsu Kim ◽  
Se-Bum Paik
Keyword(s):  
Neural Networks ◽  
Face Detection ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Visual Experience ◽  
Visual Pathway ◽  
Visual Stream ◽  
Face Selectivity ◽  
Ventral Visual Stream ◽  
The Brain

AbstractFace-selective neurons are observed in the primate visual pathway and are considered as the basis of face detection in the brain. However, it has been debated as to whether this neuronal selectivity can arise innately or whether it requires training from visual experience. Here, using a hierarchical deep neural network model of the ventral visual stream, we suggest a mechanism in which face-selectivity arises in the complete absence of training. We found that units selective to faces emerge robustly in randomly initialized networks and that these units reproduce many characteristics observed in monkeys. This innate selectivity also enables the untrained network to perform face-detection tasks. Intriguingly, we observed that units selective to various non-face objects can also arise innately in untrained networks. Our results imply that the random feedforward connections in early, untrained deep neural networks may be sufficient for initializing primitive visual selectivity.

scholarly journals Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

2020 ◽  
Author(s):  
Soma Nonaka ◽  
Kei Majima ◽  
Shuntaro C. Aoki ◽  
Yukiyasu Kamitani
Keyword(s):  
Neural Networks ◽  
Image Recognition ◽  
High Performance ◽  
Deep Neural Networks ◽  
Recognition Performance ◽  
Brain Activity ◽  
Spatial Integration ◽  
Hierarchical Processing ◽  
Visual Areas ◽  
The Brain

SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.

Evolving deep neural networks for Time Series Forecasting

2021 ◽  
Vol 18 (2) ◽  
pp. 40-55
Author(s):  
Lídio Mauro Lima Campos ◽  
◽  
Jherson Haryson Almeida Pereira ◽  
Danilo Souza Duarte ◽  
Roberto Célio Limão Oliveira ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Neural Networks ◽  
Time Series ◽  
Deep Neural Networks ◽  
Good Prediction ◽  
Biologically Inspired ◽  
Price Prediction ◽  
Error Measures ◽  
Good Ability ◽  
Proposed Model

The aim of this paper is to introduce a biologically inspired approach that can automatically generate Deep Neural networks with good prediction capacity, smaller error and large tolerance to noises. In order to do this, three biological paradigms were used: Genetic Algorithm (GA), Lindenmayer System and Neural Networks (DNNs). The final sections of the paper present some experiments aimed at investigating the possibilities of the method in the forecast the price of energy in the Brazilian market. The proposed model considers a multi-step ahead price prediction (12, 24, and 36 weeks ahead). The results for MLP and LSTM networks show a good ability to predict peaks and satisfactory accuracy according to error measures comparing with other methods.

scholarly journals Lessons From Deep Neural Networks for Studying the Coding Principles of Biological Neural Networks

2021 ◽  
Vol 14 ◽  
Author(s):  
Hyojin Bae ◽  
Sang Jeong Kim ◽  
Chang-Eop Kim
Keyword(s):  
Neural Networks ◽  
Neural Coding ◽  
Deep Neural Networks ◽  
Neural Response ◽  
Feature Representation ◽  
Network Feature ◽  
Biological Neural Networks ◽  
Stimulus Features ◽  
The Comparative Study ◽  
The Brain

One of the central goals in systems neuroscience is to understand how information is encoded in the brain, and the standard approach is to identify the relation between a stimulus and a neural response. However, the feature of a stimulus is typically defined by the researcher's hypothesis, which may cause biases in the research conclusion. To demonstrate potential biases, we simulate four likely scenarios using deep neural networks trained on the image classification dataset CIFAR-10 and demonstrate the possibility of selecting suboptimal/irrelevant features or overestimating the network feature representation/noise correlation. Additionally, we present studies investigating neural coding principles in biological neural networks to which our points can be applied. This study aims to not only highlight the importance of careful assumptions and interpretations regarding the neural response to stimulus features but also suggest that the comparative study between deep and biological neural networks from the perspective of machine learning can be an effective strategy for understanding the coding principles of the brain.

scholarly journals Understanding a Deep Learning Technique through a Neuromorphic System a Case Study with SpiNNaker Neuromorphic Platform

2018 ◽  
Vol 164 ◽  
pp. 01015
Author(s):  
Indar Sugiarto ◽  
Felix Pasila
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Cellular Level ◽  
System A ◽  
Learning Technique ◽  
Neuromorphic System ◽  
The Brain ◽  
Potential Use

Deep learning (DL) has been considered as a breakthrough technique in the field of artificial intelligence and machine learning. Conceptually, it relies on a many-layer network that exhibits a hierarchically non-linear processing capability. Some DL architectures such as deep neural networks, deep belief networks and recurrent neural networks have been developed and applied to many fields with incredible results, even comparable to human intelligence. However, many researchers are still sceptical about its true capability: can the intelligence demonstrated by deep learning technique be applied for general tasks? This question motivates the emergence of another research discipline: neuromorphic computing (NC). In NC, researchers try to identify the most fundamental ingredients that construct intelligence behaviour produced by the brain itself. To achieve this, neuromorphic systems are developed to mimic the brain functionality down to cellular level. In this paper, a neuromorphic platform called SpiNNaker is described and evaluated in order to understand its potential use as a platform for a deep learning approach. This paper is a literature review that contains comparative study on algorithms that have been implemented in SpiNNaker.

The architecture challenge: Future artificial-intelligence systems will require sophisticated architectures, and knowledge of the brain might guide their construction

2017 ◽  
Vol 40 ◽  
Author(s):  
Gianluca Baldassarre ◽  
Vieri Giuliano Santucci ◽  
Emilio Cartoni ◽  
Daniele Caligiore
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Human Cognition ◽  
Important Means ◽  
The Brain ◽  
Artificial Intelligence Systems

AbstractIn this commentary, we highlight a crucial challenge posed by the proposal of Lake et al. to introduce key elements of human cognition into deep neural networks and future artificial-intelligence systems: the need to design effective sophisticated architectures. We propose that looking at the brain is an important means of facing this great challenge.

scholarly journals Video Super-Resolution Based on Generative Adversarial Network and Edge Enhancement

Electronics ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 459
Author(s):  
Jialu Wang ◽  
Guowei Teng ◽  
Ping An
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Visual Experience ◽  
Super Resolution ◽  
Ground Truth ◽  
Intermediate Result ◽  
Edge Enhancement ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Sr Reconstruction

With the help of deep neural networks, video super-resolution (VSR) has made a huge breakthrough. However, these deep learning-based methods are rarely used in specific situations. In addition, training sets may not be suitable because many methods only assume that under ideal circumstances, low-resolution (LR) datasets are downgraded from high-resolution (HR) datasets in a fixed manner. In this paper, we proposed a model based on Generative Adversarial Network (GAN) and edge enhancement to perform super-resolution (SR) reconstruction for LR and blur videos, such as closed-circuit television (CCTV). The adversarial loss allows discriminators to be trained to distinguish between SR frames and ground truth (GT) frames, which is helpful to produce realistic and highly detailed results. The edge enhancement function uses the Laplacian edge module to perform edge enhancement on the intermediate result, which helps further improve the final results. In addition, we add the perceptual loss to the loss function to obtain a higher visual experience. At the same time, we also tried training network on different datasets. A large number of experiments show that our method has advantages in the Vid4 dataset and other LR videos.

scholarly journals Brain Age Prediction With Morphological Features Using Deep Neural Networks: Results From Predictive Analytic Competition 2019

2021 ◽  
Vol 11 ◽  
Author(s):  
Angela Lombardi ◽  
Alfonso Monaco ◽  
Giacinto Donvito ◽  
Nicola Amoroso ◽  
Roberto Bellotti ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Morphological Changes ◽  
Structure Characterization ◽  
Feature Importance ◽  
Mri Scans ◽  
Magnetic Resonance Imaging Mri ◽  
Brain Age ◽  
The Brain ◽  
Age Prediction

Morphological changes in the brain over the lifespan have been successfully described by using structural magnetic resonance imaging (MRI) in conjunction with machine learning (ML) algorithms. International challenges and scientific initiatives to share open access imaging datasets also contributed significantly to the advance in brain structure characterization and brain age prediction methods. In this work, we present the results of the predictive model based on deep neural networks (DNN) proposed during the Predictive Analytic Competition 2019 for brain age prediction of 2638 healthy individuals. We used FreeSurfer software to extract some morphological descriptors from the raw MRI scans of the subjects collected from 17 sites. We compared the proposed DNN architecture with other ML algorithms commonly used in the literature (RF, SVR, Lasso). Our results highlight that the DNN models achieved the best performance with MAE = 4.6 on the hold-out test, outperforming the other ML strategies. We also propose a complete ML framework to perform a robust statistical evaluation of feature importance for the clinical interpretability of the results.

scholarly journals Do deep neural networks see the way we do?

2019 ◽  
Author(s):  
Georgin Jacob ◽  
R. T. Pramod ◽  
Harish Katti ◽  
S. P. Arun
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Relative Size ◽  
Training Data ◽  
Object Representations ◽  
Multiple Dimensions ◽  
Deep Networks ◽  
The Brain ◽  
Global Advantage ◽  
The Way

ABSTRACTDeep neural networks have revolutionized computer vision, and their object representations match coarsely with the brain. As a result, it is widely believed that any fine scale differences between deep networks and brains can be fixed with increased training data or minor changes in architecture. But what if there are qualitative differences between brains and deep networks? Do deep networks even see the way we do? To answer this question, we chose a deep neural network optimized for object recognition and asked whether it exhibits well-known perceptual and neural phenomena despite not being explicitly trained to do so. To our surprise, many phenomena were present in the network, including the Thatcher effect, mirror confusion, Weber’s law, relative size, multiple object normalization and sparse coding along multiple dimensions. However, some perceptual phenomena were notably absent, including processing of 3D shape, patterns on surfaces, occlusion, natural parts and a global advantage. Our results elucidate the computational challenges of vision by showing that learning to recognize objects suffices to produce some perceptual phenomena but not others and reveal the perceptual properties that could be incorporated into deep networks to improve their performance.

Sign in / Sign up

Export Citation Format

Share Document