Weight statistics controls dynamics in recurrent neural networks.

08:00 EDT 9th April 2019 | BioPortfolio

Summary of "Weight statistics controls dynamics in recurrent neural networks."

Recurrent neural networks are complex non-linear systems, capable of ongoing activity in the absence of driving inputs. The dynamical properties of these systems, in particular their long-time attractor states, are determined on the microscopic level by the connection strengths wij between the individual neurons. However, little is known to which extent network dynamics is tunable on a more coarse-grained level by the statistical features of the weight matrix. In this work, we investigate the dynamics of recurrent networks of Boltzmann neurons. In particular we study the impact of three statistical parameters: density (the fraction of non-zero connections), balance (the ratio of excitatory to inhibitory connections), and symmetry (the fraction of neuron pairs with wij = wji). By computing a 'phase diagram' of network dynamics, we find that balance is the essential control parameter: Its gradual increase from negative to positive values drives the system from oscillatory behavior into a chaotic regime, and eventually into stationary fixed points. Only directly at the border of the chaotic regime do the neural networks display rich but regular dynamics, thus enabling actual information processing. These results suggest that the brain, too, is fine-tuned to the 'edge of chaos' by assuring a proper balance between excitatory and inhibitory neural connections.


Journal Details

This article was published in the following journal.

Name: PloS one
ISSN: 1932-6203
Pages: e0214541


DeepDyve research library

PubMed Articles [17992 Associated PubMed Articles listed on BioPortfolio]

Global Synchronization of Coupled Fractional-Order Recurrent Neural Networks.

This paper presents new theoretical results on the global synchronization of coupled fractional-order recurrent neural networks. Under the assumptions that the coupled fractional-order recurrent neura...

Deep neural networks in psychiatry.

Machine and deep learning methods, today's core of artificial intelligence, have been applied with increasing success and impact in many commercial and research settings. They are powerful tools for l...

The Vapnik-Chervonenkis dimension of graph and recursive neural networks.

The Vapnik-Chervonenkis dimension (VC-dim) characterizes the sample learning complexity of a classification model and it is often used as an indicator for the generalization capability of a learning m...

Contrasting the effects of adaptation and synaptic filtering on the timescales of dynamics in recurrent networks.

Neural activity in awake behaving animals exhibits a vast range of timescales that can be several fold larger than the membrane time constant of individual neurons. Two types of mechanisms have been p...

Emergent mechanisms of evidence integration in recurrent neural networks.

Recent advances in machine learning have enabled neural networks to solve tasks humans typically perform. These networks offer an exciting new tool for neuroscience that can give us insight in the eme...

Clinical Trials [9069 Associated Clinical Trials listed on BioPortfolio]

Resting-State Neural Connectivity in Patients With Subjective Tinnitus Without Bother

Tinnitus is the occurrence of an auditory sensation without the presence of an acoustic stimulus. Approximately, 50 million people in the United States experience chronic tinnitus and 15 m...

Neural Dynamics Technique for the Treatment of Chronic Radicular Low Back Pain

Low back pain ( LBP) is a significant health problem which affects about 70%-80% of the population during the course of their life . The majority of LBP is resolved within 3 months; howev...

Neural Mechanisms Underlying the Antidepressant Effects of Sleep Deprivation

From 40 to 60% of patients with depression experience a rapid and significant improvement of mood with one night of sleep deprivation (SD). The neural mechanisms underlying this effect hav...

Contributions From the Analysis of Graphs for Identification of Neural Cliques

The aim of the study is to demonstrate that our semantic knowledge (elements of our long-term memory and the process we use them) respond to a graphic organisation and gather together foll...

Responses to Appetite and Taste in the Brain Circuits That Control Eating Behaviour

The aim of this study is to gain insight into brain structure and the neural networks that control taste and eating behaviour in patients with severe obesity undergoing a primary sleeve ga...

Medical and Biotech [MESH] Definitions

A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.

Used for general articles concerning statistics of births, deaths, marriages, etc.

An early embryonic developmental process of CHORDATES that is characterized by morphogenic movements of ECTODERM resulting in the formation of the NEURAL PLATE; the NEURAL CREST; and the NEURAL TUBE. Improper closure of the NEURAL GROOVE results in congenital NEURAL TUBE DEFECTS.

An indicator of body density as determined by the relationship of BODY WEIGHT to BODY HEIGHT. BMI=weight (kg)/height squared (m2). BMI correlates with body fat (ADIPOSE TISSUE). Their relationship varies with age and gender. For adults, BMI falls into these categories: below 18.5 (underweight); 18.5-24.9 (normal); 25.0-29.9 (overweight); 30.0 and above (obese). (National Center for Health Statistics, Centers for Disease Control and Prevention)

The physical or mechanical action of the LUNGS; DIAPHRAGM; RIBS; and CHEST WALL during respiration. It includes airflow, lung volume, neural and reflex controls, mechanoreceptors, breathing patterns, etc.

Quick Search


DeepDyve research library

Searches Linking to this Article