scholarly journals Training the brain using neural-network models

Nature ◽  
1988 ◽  
Vol 333 (6172) ◽  
pp. 401-401 ◽  
Author(s):  
JOHN R. SKOYLES
Keyword(s):  
Neural Network ◽  
Network Models ◽  
Neural Network Models ◽  
The Brain

scholarly journals Modeling the online social transmission through the theory of complex adaptive systems and neural network models based on the brain dynamics

2015 ◽  
Author(s):  
Ηλίας Λυμπερόπουλος
Keyword(s):  
Neural Network ◽  
Complex Adaptive Systems ◽  
Adaptive Systems ◽  
Network Models ◽  
Social Transmission ◽  
Brain Dynamics ◽  
Neural Network Models ◽  
Complex Adaptive ◽  
The Brain

Η μοντελοποίηση δυναμικών κοινωνικών διαδικασιών που λαμβάνουν χώρα στο διαδίκτυο αποτελεί ένα απαιτητικό εγχείρημα για τους παρακάτω λόγους: Πρώτον, τα πρόσωπα που αλληλεπιδρούν είναι ετερογενή και το καθένα ξεχωριστά αποτελεί ένα πολύπλοκο σύστημα. Δεύτερον, οι αλληλεπιδράσεις μεταξύ χρηστών επηρεάζονται από θόρυβο και τυχαιότητα, ενώ παράλληλα το δίκτυο των διαπροσωπικών επικοινωνιών είναι εξαιρετικά πολύπλοκο. Τρίτον, τα κοινωνικά συστήματα δεν βρίσκονται σε κατάσταση ισορροπίας καθώς η δυναμική τους επηρεάζεται από εξωτερικές διαταραχές των οποίων η κατανομή, συσχέτιση με ένα κοινωνικό σύστημα, καθώς και η μη στασιμότητά τους, είναι δύσκολο να καθοριστούν και να συμπεριληφθούν σ’ ένα δυναμικό κοινωνικό μοντέλο. Η επιτυχής μοντελοποίηση της online κοινωνικής μετάδοσης απαιτεί μια προσέγγιση ικανή να ανταπεξέλθει στις παραπάνω προκλήσεις. Γι’ αυτό το σκοπό αναπτύσσω και εφαρμόζω ένα πλαίσιο υλοποίησης βασισμένο στην θεωρία των πολύπλοκων προσαρμοστικών συστημάτων}. Μέσω μια τέτοιας μεθοδολογίας μπορεί να μελετηθεί η δυναμική φύση των αλληλεπιδράσεων μεταξύ χρηστών καθώς και οι μακροσκοπικές ιδιότητες της δραστηριότητας τους υπό την παρουσία εξωτερικών επιρροών. Ένα εξαιρετικά σύνθετο πολύπλοκο προσαρμοστικό σύστημα είναι αυτό του ανθρώπινου εγκεφάλου. Τα κοινωνικά δίκτυα είναι ακόμα πιο πολύπλοκα καθώς ουσιαστικά αποτελούνται από διασυνδεδεμένους εγκεφάλους. Ως αποτέλεσμα η μοντελοποίηση δυναμικών διαδικασιών που λαμβάνουν χώρα στα online κοινωνικά δίκτυα αποτελεί ένα υπερβολικά περίπλοκο έργο. Για την αντιμετώπιση των προκλήσεων μιας τέτοιας προσπάθειας εξετάζω τις online κοινωνικές διεργασίες μέσα από την προοπτική της νευροεπιστήμης θεωρώντας τη δυναμική των online κοινωνικών δικτύων ανάλογη με την δυναμική δικτύων αποτελούμενων από νευρώνες ολοκλήρωσης και πυροδότησης. Μέσω αυτού του ισομορφισμού εισάγω ένα νέο μοντέλο για την online κοινωνική μετάδοση το οποίο βασίζεται σε τρεις πηγές θετικής ή αρνητικής επιρροής: Την αυτοπαραγόμενη, τη διαπροσωπική και την εξωτερική. Το προτεινόμενο μοντέλο εξηγεί την ανάπτυξη της online δραστηριότητας καθώς και τις μορφές μετάδοσής της σε συνάρτηση με το δίκτυο των αλληλεπιδράσεων, την ενδογενή και εξωγενή επιρροή καθώς και τον μηχανισμό ενεργοποίησης των χρηστών. Μέσω πειραμάτων εξομοίωσης και ελέγχων εγκυρότητας των παραγόμενων αποτελεσμάτων μετά από σύγκριση με πραγματικά δεδομένα από το κοινωνικό δίκτυο Twitter, δείχνω ότι το μοντέλο αναπαράγει με ακρίβεια πρότυπα συλλογικής δραστηριότητας προερχόμενα από την απόκριση των χρηστών σε διαφόρων μορφών ερεθίσματα. Η συγκριτική αξιολόγηση των επιδόσεων του προτεινόμενου μοντέλου σε συνάρτηση με αυτή μοντέλων αναφοράς δείχνει ότι αυτό υπερτερεί σημαντικά στην ακρίβεια αναπαραγωγής πραγματικών προτύπων online δραστηριότητας. Μια ακόμα διαδικασία online κοινωνικής μετάδοσης την οποία μοντελοποιώ με καινοτόμο τρόπο σε αυτή τη Διδακτορική Διατριβή αφορά στη μετάδοση online πληροφορίας. Τη δυναμική αυτής της διαδικασίας την αναπαράγω μέσω ενός δικτυακού δυναμικού συστήματος αποτελούμενο από νευρώνες ολοκλήρωσης και πυροδότησης με θορυβώδη εισροή πληροφορίας. Μέσω του συνδυασμού ντετερμινιστικών και στοχαστικών συνιστωσών το προτεινόμενο μοντέλο αναπαράγει με ακρίβεια τα πρότυπα μετάδοσης online πληροφορίας, υποδηλώνοντας ότι αυτά εξαρτώνται από την χρονική δομή, ισχύ, καθώς και το λόγο σήματος-θορύβου των ερεθισμάτων που επηρεάζουν τους διασυνδεδεμένους χρήστες. Ο προτεινόμενος μηχανισμός ενσωματώνει τις έννοιες της ``απλής'' και ``πολύπλοκης'' μετάδοσης και επεκτείνει τις υπάρχουσες προσεγγίσεις καθώς συμπεριλαμβάνει σε ένα ενιαίο μοντέλο ενδογενείς και εξωγενείς, θετικές και αρνητικές, ντετερμινιστικές και στοχαστικές πηγές επιρροής. Τα προτεινόμενα μοντέλα νευρωνικών δικτύων είναι εύκολα προσαρμόσιμα και κατάλληλα για τη μελέτη ενός μεγάλου αριθμού από online και offline κοινωνικές δυναμικές διαδικασίες που αφορούν στη διάδοση συμπεριφορών, τάσεων και φημών, καθώς και στη διάχυση και προώθηση νέων προϊόντων. Τελικά, τη γνώση που προέκυψε από την μοντελοποίηση των προτύπων της online κοινωνικής δραστηριότητας την αξιοποιώ περαιτέρω με την ανάπτυξη ενός προβλεπτικού μοντέλου ικανού να παράγει ακριβείς προβλέψεις σχετικά με τη διάδοση online περιεχομένου.

The Role of Computer Simulation in Neurolinguistics

1993 ◽  
Vol 16 (2) ◽  
pp. 153-169 ◽  
Author(s):  
Hans-Jürgen Eikmeyer ◽  
Ulrich Schade
Keyword(s):  
Neural Network ◽  
Computer Simulation ◽  
Cognitive Science ◽  
Language Production ◽  
Network Models ◽  
Neural Network Models ◽  
Cognitive Capacities ◽  
The Brain ◽  
Structure And Functions

As a result of present-day technological standards, the technique of computer simulation is constantly gaining influence in cognitive science. Neurolinguistics is a special branch of this field in which cognitive capacities connected with language are related to the structure and functions of the brain. It is argued that computer simulation is a useful technique for evaluating neurolinguistic models. This is demonstrated with respect to neural network models of the process of language production.

Ergodicity of Spike Trains: When Does Trial Averaging Make Sense?

2003 ◽  
Vol 15 (6) ◽  
pp. 1341-1372 ◽  
Author(s):  
Naoki Masuda ◽  
Kazuyuki Aihara
Keyword(s):  
Neural Network ◽  
Network Models ◽  
Spike Trains ◽  
Neural Network Models ◽  
Large Variability ◽  
Synaptic Inputs ◽  
Synchronous Firing ◽  
High Membrane Potential ◽  
External Stimuli ◽  
The Brain

Neuronal information processing is often studied on the basis of spiking patterns. The relevant statistics such as firing rates calculated with the peri-stimulus time histogram are obtained by averaging spiking patterns over many experimental runs. However, animals should respond to one experimental stimulation in real situations, and what is available to the brain is not the trial statistics but the population statistics. Consequently, physiological ergodicity, namely, the consistency between trial averaging and population averaging, is implicitly assumed in the data analyses, although it does not trivially hold true. In this letter, we investigate how characteristics of noisy neural network models, such as single neuron properties, external stimuli, and synaptic inputs, affect the statistics of firing patterns. In particular, we show that how high membrane potential sensitivity to input fluctuations, inability of neurons to remember past inputs, external stimuli with large variability and temporally separated peaks, and relatively few contributions of synaptic inputs result in spike trains that are reproducible over many trials. The reproducibility of spike trains and synchronous firing are contrasted and related to the ergodicity issue. Several numerical calculations with neural network examples are carried out to support the theoretical results.

scholarly journals A Deep Predictive Coding Network for Learning Latent Representations

2018 ◽  
Author(s):  
Shirin Dora ◽  
Cyriel Pennartz ◽  
Sander Bohte
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Predictive Coding ◽  
External Environment ◽  
Network Models ◽  
The Novel ◽  
Neural Network Models ◽  
Architectural Constraints ◽  
Latent Representations ◽  
The Brain

AbstractIt has been argued that the brain is a prediction machine that continuously learns how to make better predictions about the stimuli received from the external environment. It builds a model of the world around us and uses this model to infer the external stimulus. Predictive coding has been proposed as a mechanism through which the brain might be able to build such a model of the external environment. However, it is not clear how predictive coding can be used to build deep neural network models of the brain while complying with the architectural constraints imposed by the brain. In this paper, we describe an algorithm to build a deep generative model using predictive coding that can be used to infer latent representations about the stimuli received from external environment. Specifically, we used predictive coding to train a deep neural network on real-world images in a unsupervised learning paradigm. To understand the capacity of the network with regards to modeling the external environment, we studied the latent representations generated by the model on images of objects that are never presented to the model during training. Despite the novel features of these objects the model is able to infer the latent representations for them. Furthermore, the reconstructions of the original images obtained from these latent representations preserve the important details of these objects.

scholarly journals Neural classifiers with limited connectivity and recurrent readouts

2017 ◽  
Author(s):  
Lyudmila Kushnir ◽  
Stefano Fusi
Keyword(s):  
Neural Network ◽  
Long Range ◽  
Network Models ◽  
Classification Performance ◽  
Network Architectures ◽  
Huge Number ◽  
Neural Network Models ◽  
Random Patterns ◽  
Scalable Network ◽  
The Brain

AbstractFor many neural network models in which neurons are trained to classify inputs like perceptrons, the number of inputs that can be classified is limited by the connectivity of each neuron, even when the total number of neurons is very large. This poses the problem of how the biological brain can take advantage of its huge number of neurons given that the connectivity is sparse. One solution is to combine multiple perceptrons together, as in committee machines. The number of classifiable random patterns would then grow linearly with the number of perceptrons, even when each perceptron has limited connectivity. However, the problem is moved to the downstream readout neurons, which would need a number of connections that is as large as the number of perceptrons. Here we propose a different approach in which the readout is implemented by connecting multiple perceptrons in a recurrent attractor neural network. We prove analytically that the number of classifiable random patterns can grow unboundedly with the number of perceptrons, even when the connectivity of each perceptron remains finite. Most importantly, both the recurrent connectivity and the connectivity of downstream readouts also remain finite. Our study shows that feed-forward neural classifiers with numerous long range afferent connections can be replaced by recurrent networks with sparse long range connectivity without sacrificing the classification performance. Our strategy could be used to design more general scalable network architectures with limited connectivity, which resemble more closely the brain neural circuits which are dominated by recurrent connectivity.

Material Demands for Optical Neural Networks

MRS Bulletin ◽  
1988 ◽  
Vol 13 (8) ◽  
pp. 30-35 ◽  
Author(s):  
Dana Z. Anderson
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Speech Processing ◽  
Visual Processing ◽  
Network Models ◽  
Human Memory ◽  
Information Storage ◽  
Physical Damage ◽  
Neural Network Models ◽  
The Brain

From the time of their conception, holography and holograms have evolved as a metaphor for human memory. Holograms can be made so that the information they contain is distributed throughout the holographic medium—destroy part of the hologram and the stored information remains wholly intact, except for a loss of detail. In this property holograms evidently have something in common with human memory, which is to some extent resilient against physical damage to the brain. There is much more to the metaphor than simply that information is stored in a distributed manner.Research in the optics community is now looking to holography, in particular dynamic holography, not only for information storage, but for information processing as well. The ideas are based upon neural network models. Neural networks are models for processing that are inspired by the apparent architecture of the brain. This is a processing paradigm that is new to optics. From within this network paradigm we look to build machines that can store and recall information associatively, play back a chain of recorded events, undergo learning and possibly forgetting, make decisions, adapt to a particular environment, and self-organize to evolve some desirable behavior. We hope that neural network models will give rise to optical machines for memory, speech processing, visual processing, language acquisition, motor control, and so on.

scholarly journals Engineering recurrent neural networks from task-relevant manifolds and dynamics

2019 ◽  
Author(s):  
Eli Pollock ◽  
Mehrdad Jazayeri
Keyword(s):  
Neural Network ◽  
Working Memory ◽  
Recurrent Neural Network ◽  
Network Dynamics ◽  
Memory Task ◽  
Network Models ◽  
Network Activity ◽  
Mathematical Framework ◽  
Neural Network Models ◽  
The Brain

AbstractMany cognitive processes involve transformations of distributed representations in neural populations, creating a need for population-level models. Recurrent neural network models fulfill this need, but there are many open questions about how their connectivity gives rise to dynamics that solve a task. Here, we present a method for finding the connectivity of networks for which the dynamics are specified to solve a task in an interpretable way. We apply our method to a working memory task by synthesizing a network that implements a drift-diffusion process over a ring-shaped manifold. We also use our method to demonstrate how inputs can be used to control network dynamics for cognitive flexibility and explore the relationship between representation geometry and network capacity. Our work fits within the broader context of understanding neural computations as dynamics over relatively low-dimensional manifolds formed by correlated patterns of neurons.Author SummaryNeurons in the brain form intricate networks that can produce a vast array of activity patterns. To support goal-directed behavior, the brain must adjust the connections between neurons so that network dynamics can perform desirable computations on behaviorally relevant variables. A fundamental goal in computational neuroscience is to provide an understanding of how network connectivity aligns the dynamics in the brain to the dynamics needed to track those variables. Here, we develop a mathematical framework for creating recurrent neural network models that can address this problem. Specifically, we derive a set of linear equations that constrain the connectivity to afford a direct mapping of task-relevant dynamics onto network activity. We demonstrate the utility of this technique by creating and analyzing a set of network models that can perform a simple working memory task. We then extend the approach to show how additional constraints can furnish networks whose dynamics are controlled flexibly by external inputs. Finally, we exploit the flexibility of this technique to explore the robustness and capacity limitations of recurrent networks. This network synthesis method provides a powerful means for generating and validating hypotheses about how task-relevant computations can emerge from network dynamics.

scholarly journals H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks

2020 ◽  
Author(s):  
Thomas Limbacher ◽  
Robert Legenstein
Keyword(s):  
Neural Network ◽  
Synaptic Plasticity ◽  
Question Answering ◽  
Network Models ◽  
Neural Network Models ◽  
The Past ◽  
Hebbian Plasticity ◽  
Long Time ◽  
Simple Neural Network ◽  
The Brain

AbstractThe ability to base current computations on memories from the past is critical for many cognitive tasks such as story understanding. Hebbian-type synaptic plasticity is believed to underlie the retention of memories over medium and long time scales in the brain. However, it is unclear how such plasticity processes are integrated with computations in cortical networks. Here, we propose Hebbian Memory Networks (H-Mems), a simple neural network model that is built around a core hetero-associative network subject to Hebbian plasticity. We show that the network can be optimized to utilize the Hebbian plasticity processes for its computations. H-Mems can one-shot memorize associations between stimulus pairs and use these associations for decisions later on. Furthermore, they can solve demanding question-answering tasks on synthetic stories. Our study shows that neural network models are able to enrich their computations with memories through simple Hebbian plasticity processes.

Sign in / Sign up

Export Citation Format

Share Document