Learning in the machine: Recirculation is random backpropagation.

08:00 EDT 27th September 2018 | BioPortfolio

Summary of "Learning in the machine: Recirculation is random backpropagation."

Learning in physical neural systems must rely on learning rules that are local in both space and time. Optimal learning in deep neural architectures requires that non-local information be available to the deep synapses. Thus, in general, optimal learning in physical neural systems requires the presence of a deep learning channel to communicate non-local information to deep synapses, in a direction opposite to the forward propagation of the activities. Theoretical arguments suggest that for circular autoencoders, an important class of neural architectures where the output layer is identical to the input layer, alternative algorithms may exist that enable local learning without the need for additional learning channels, by using the forward activation channel as the deep learning channel. Here we systematically identify, classify, and study several such local learning algorithms, based on the general idea of recirculating information from the output layer to the hidden layers. We show through simulations and mathematical derivations that these algorithms are robust and converge to critical points of the global error function. In most cases, we show that these recirculation algorithms are very similar to an adaptive form of random backpropagation, where each hidden layer receives a linearly transformed, slowly-varying, version of the output error.


Journal Details

This article was published in the following journal.

Name: Neural networks : the official journal of the International Neural Network Society
ISSN: 1879-2782
Pages: 479-494


DeepDyve research library

PubMed Articles [5348 Associated PubMed Articles listed on BioPortfolio]

Backpropagation through time and the brain.

It has long been speculated that the backpropagation-of-error algorithm (backprop) may be a model of how the brain learns. Backpropagation-through-time (BPTT) is the canonical temporal-analogue to bac...

Contrastive Hebbian learning with random feedback weights.

Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's ru...

Is It Ethical to Use Prognostic Estimates from Machine Learning to Treat Psychosis?

Machine learning is a method for predicting clinically relevant variables, such as opportunities for early intervention, potential treatment response, prognosis, and health outcomes. This commentary e...

RNNbow: Visualizing Learning Via Backpropagation Gradients in RNNs.

We present RNNbow, an interactive tool for visualizing the gradient flow during backpropagation in training of recurrent neural networks. By visualizing the gradient, as opposed to activations, RNNbow...

Can Machine-learning Techniques Be Used for 5-year Survival Prediction of Patients With Chondrosarcoma?

Several studies have identified prognostic factors for patients with chondrosarcoma, but there are few studies investigating the accuracy of computationally intensive methods such as machine learning....

Clinical Trials [2057 Associated Clinical Trials listed on BioPortfolio]

Blood Recirculation and vvECMO

A prospective, observational study will be performed measuring recirculation of blood through a veno-venous Extracoporeal Membrane Oxygenation (vvECMO) in patients with acute respiratory d...

Measurement of Recirculation During Venovenous Extracorporeal Membrane Oxygenation (VV ECMO)

This study is to test the usefulness of ultrasound dilution recirculation measurements in patients receiving venovenous extracorporeal membrane oxygenation as therapy. The ultrasound dilut...

Machine Learning From Fetal Flow Waveforms to Predict Adverse Perinatal Outcomes

The aim of this study is to get a proof of concept for using a computational model of fetal haemodynamics, combined with machine learning based on Doppler patterns of the fetal cardiovascu...

Machine Learning-Based Risk Profile Classification of Patients Undergoing Elective Heart Valve Surgery

Machine learning methods potentially provide a highly accurate and detailed assessment of expected individual patient risk before elective cardiac surgery. Correct anticipation of this ris...

The CIRE Study (CItrate REcirculation Study)

The aim of this study is to determine the effects of AV port reversal (AVPR) on recirculation, clearance, post-filter ionized calcium and subsequently citrate dosing.

Medical and Biotech [MESH] Definitions

A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of unlabeled paired input-output training (sample) data.

A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of labeled paired input-output training (sample) data.

SUPERVISED MACHINE LEARNING algorithm which learns to assign labels to objects from a set of training examples. Examples are learning to recognize fraudulent credit card activity by examining hundreds or thousands of fraudulent and non-fraudulent credit card activity, or learning to make disease diagnosis or prognosis based on automatic classification of microarray gene expression profiles drawn from hundreds or thousands of samples.

A type of ARTIFICIAL INTELLIGENCE that enable COMPUTERS to independently initiate and execute LEARNING when exposed to new data.

Process in which individuals take the initiative, in diagnosing their learning needs, formulating learning goals, identifying resources for learning, choosing and implementing learning strategies and evaluating learning outcomes (Knowles, 1975)

Quick Search


DeepDyve research library

Searches Linking to this Article