Track topics on Twitter Track topics that are important to you
Learning in physical neural systems must rely on learning rules that are local in both space and time. Optimal learning in deep neural architectures requires that non-local information be available to the deep synapses. Thus, in general, optimal learning in physical neural systems requires the presence of a deep learning channel to communicate non-local information to deep synapses, in a direction opposite to the forward propagation of the activities. Theoretical arguments suggest that for circular autoencoders, an important class of neural architectures where the output layer is identical to the input layer, alternative algorithms may exist that enable local learning without the need for additional learning channels, by using the forward activation channel as the deep learning channel. Here we systematically identify, classify, and study several such local learning algorithms, based on the general idea of recirculating information from the output layer to the hidden layers. We show through simulations and mathematical derivations that these algorithms are robust and converge to critical points of the global error function. In most cases, we show that these recirculation algorithms are very similar to an adaptive form of random backpropagation, where each hidden layer receives a linearly transformed, slowly-varying, version of the output error.
This article was published in the following journal.
Name: Neural networks : the official journal of the International Neural Network Society
It has long been speculated that the backpropagation-of-error algorithm (backprop) may be a model of how the brain learns. Backpropagation-through-time (BPTT) is the canonical temporal-analogue to bac...
Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's ru...
Machine learning is a method for predicting clinically relevant variables, such as opportunities for early intervention, potential treatment response, prognosis, and health outcomes. This commentary e...
We present RNNbow, an interactive tool for visualizing the gradient flow during backpropagation in training of recurrent neural networks. By visualizing the gradient, as opposed to activations, RNNbow...
Several studies have identified prognostic factors for patients with chondrosarcoma, but there are few studies investigating the accuracy of computationally intensive methods such as machine learning....
A prospective, observational study will be performed measuring recirculation of blood through a veno-venous Extracoporeal Membrane Oxygenation (vvECMO) in patients with acute respiratory d...
This study is to test the usefulness of ultrasound dilution recirculation measurements in patients receiving venovenous extracorporeal membrane oxygenation as therapy. The ultrasound dilut...
The aim of this study is to get a proof of concept for using a computational model of fetal haemodynamics, combined with machine learning based on Doppler patterns of the fetal cardiovascu...
Machine learning methods potentially provide a highly accurate and detailed assessment of expected individual patient risk before elective cardiac surgery. Correct anticipation of this ris...
The aim of this study is to determine the effects of AV port reversal (AVPR) on recirculation, clearance, post-filter ionized calcium and subsequently citrate dosing.
A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of unlabeled paired input-output training (sample) data.
A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of labeled paired input-output training (sample) data.
SUPERVISED MACHINE LEARNING algorithm which learns to assign labels to objects from a set of training examples. Examples are learning to recognize fraudulent credit card activity by examining hundreds or thousands of fraudulent and non-fraudulent credit card activity, or learning to make disease diagnosis or prognosis based on automatic classification of microarray gene expression profiles drawn from hundreds or thousands of samples.
A type of ARTIFICIAL INTELLIGENCE that enable COMPUTERS to independently initiate and execute LEARNING when exposed to new data.
Process in which individuals take the initiative, in diagnosing their learning needs, formulating learning goals, identifying resources for learning, choosing and implementing learning strategies and evaluating learning outcomes (Knowles, 1975)