On Kernel Method-Based Connectionist Models and Supervised Deep Learning without Backpropagation.

07:00 EST 8th November 2019 | BioPortfolio

Summary of "On Kernel Method-Based Connectionist Models and Supervised Deep Learning without Backpropagation."

We propose a novel family of connectionist models based on kernel machines and consider the problem of learning layer by layer a compositional hypothesis class (i.e., a feedforward, multilayer architecture) in a supervised setting. In terms of the models, we present a principled method to "kernelize" (partly or completely) any neural network (NN). With this method, we obtain a counterpart of any given NN that is powered by kernel machines instead of neurons. In terms of learning, when learning a feedforward deep architecture in a supervised setting, one needs to train all the components simultaneously using backpropagation (BP) since there are no explicit targets for the hidden layers (Rumelhart, Hinton, & Williams, 1986). We consider without loss of generality the two-layer case and present a general framework that explicitly characterizes a target for the hidden layer that is optimal for minimizing the objective function of the network. This characterization then makes possible a purely greedy training scheme that learns one layer at a time, starting from the input layer. We provide instantiations of the abstract framework under certain architectures and objective functions. Based on these instantiations, we present a layer-wise training algorithm for an -layer feedforward network for classification, where can be arbitrary. This algorithm can be given an intuitive geometric interpretation that makes the learning dynamics transparent. Empirical results are provided to complement our theory. We show that the kernelized networks, trained layer-wise, compare favorably with classical kernel machines as well as other connectionist models trained by BP. We also visualize the inner workings of the greedy kernelized models to validate our claim on the transparency of the layer-wise algorithm.


Journal Details

This article was published in the following journal.

Name: Neural computation
ISSN: 1530-888X
Pages: 1-39


DeepDyve research library

PubMed Articles [30017 Associated PubMed Articles listed on BioPortfolio]

Ensemble-based kernel learning for a class of data assimilation problems with imperfect forward simulators.

Simulator imperfection, often known as model error, is ubiquitous in practical data assimilation problems. Despite the enormous efforts dedicated to addressing this problem, properly handling simulato...

Self-supervised learning for medical image analysis using image context restoration.

Machine learning, particularly deep learning has boosted medical image analysis over the past years. Training a good model based on deep learning requires large amount of labelled data. However, it is...

Deep learning-enabled accurate normalization of reconstruction kernel effects on emphysema quantification in low-dose CT.

Lung densitometry is being frequently adopted in CT-based emphysema quantification, yet known to be affected by the choice of reconstruction kernel. This study presents a two-step deep learning archit...

Deep neural-kernel blocks.

This paper introduces novel deep architectures using the hybrid neural-kernel core model as the first building block. The proposed models follow a combination of a neural networks based architecture a...

A deep learning method for image-based subject-specific local SAR assessment.

Local specific absorption rate (SAR) cannot be measured and is usually evaluated by offline numerical simulations using generic body models that of course will differ from the patient's anatomy. An ad...

Clinical Trials [9750 Associated Clinical Trials listed on BioPortfolio]

Quality Measurement of Esophagogastroduodenoscopy Using Deep Learning Models

The aim of our study was to develop DCNN-based models and assess feasibility of the models in the quality measurement of EGD.

Deep-learning Based Location of Spine CT

CT-guided epidural steroid injection (ESI) thrives among interventional radiologists. Although X-ray fluoroscopy still remains as the gold standard to guide ESI, CT guidance has the merits...

Diagnostic Yield of Deep Learning Based Denoising MRI in Cushing's Disease

Negative MRI findings may occur in up to 40% of cases of ACTH producing microadenomas. The aim of the study is to evaluate if detection of ACTH producing microadenomas can be increased usi...

Video-Based Social Learning or Didactics for Car Seat Education

This study compares a video-based social learning method to the traditional didactic method as a new way to teach caregivers about child passenger safety, including how to install a car se...

Intelligent ICU of the Future

The objective of this project is to create deep learning and machine learning models capable of recognizing patient visual cues, including facial expressions such as pain and functional ac...

Medical and Biotech [MESH] Definitions

A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of labeled paired input-output training (sample) data.

Usually refers to the use of mathematical models in the prediction of learning to perform tasks based on the theory of probability applied to responses; it may also refer to the frequency of occurrence of the responses observed in the particular study.

SUPERVISED MACHINE LEARNING algorithm which learns to assign labels to objects from a set of training examples. Examples are learning to recognize fraudulent credit card activity by examining hundreds or thousands of fraudulent and non-fraudulent credit card activity, or learning to make disease diagnosis or prognosis based on automatic classification of microarray gene expression profiles drawn from hundreds or thousands of samples.

A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.

Process in which individuals take the initiative, in diagnosing their learning needs, formulating learning goals, identifying resources for learning, choosing and implementing learning strategies and evaluating learning outcomes (Knowles, 1975)

Quick Search

DeepDyve research library

Searches Linking to this Article