Track topics on Twitter Track topics that are important to you
Catastrophic forgetting and capacity saturation are the central challenges of any parametric lifelong learning system. In this work, we study these challenges in the context of sequential supervised learning with an emphasis on recurrent neural networks. To evaluate the models in the lifelong learning setting, we propose a curriculum-based, simple, and intuitive benchmark where the models are trained on tasks with increasing levels of difficulty. To measure the impact of catastrophic forgetting, the model is tested on all the previous tasks as it completes any task. As a step toward developing true lifelong learning systems, we unify gradient episodic memory (a catastrophic forgetting alleviation approach) and Net2Net (a capacity expansion approach). Both models are proposed in the context of feedforward networks, and we evaluate the feasibility of using them for recurrent networks. Evaluation on the proposed benchmark shows that the unified model is more suitable than the constituent models for lifelong learning setting.
This article was published in the following journal.
Name: Neural computation
Cell segmentation in microscopy images is a common and challenging task. In recent years, deep neural networks achieved remarkable improvements in the field of computer vision. The dominant paradigm i...
Learning long-term dependences (LTDs) with recurrent neural networks (RNNs) is challenging due to their limited internal memories. In this paper, we propose a new external memory architecture for RNNs...
Transfer learning has achieved a lot of success in deep neural networks to reuse useful knowledge from source domains. However, most of the existing transfer learning strategies on neural networks are...
We seek to (i) characterize the learning architectures exploited in biological neural networks for training on very few samples, and (ii) port these algorithmic structures to a machine learning contex...
An increasing number of neural memory networks have been developed, leading to the need for a systematic approach to analyze and compare their underlying memory structures. Thus, in this paper, we fir...
The basal ganglia play an important role in motor learning, especially during the consolidation phase of motor learning. This raises the question whether it is possible to sustain learning...
The study is about the role of cellular neural networks-genetic algorithm in the diagnosis of periprosthetic hip infections. A retrospective case series of septic and aseptic loosening of ...
We aim to experiment and implement various deep learning architectures in order to achieve human-level accuracy in Computer-aided diagnosis (CAD) systems. In particular, we are interested ...
All the studies underlined the high frequency of co-morbid associations in specific learning disorders. Understanding the reasons for these associations could enable us to determine the ce...
PET images are based on detecting two annihilation 511 KeV photons that are produced by positron emitting isotopes. The longer the acquisition time, the more photons are detected and proce...
A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.
Diagnosed when there are specific deficits in an individual’s ability to perceive or process information efficiently and accurately. This disorder first manifests during the years of formal schooling and is characterized by persistent and impairing difficulties with learning foundational academic skills in reading, writing, and/or math. The individual’s performance of the affected academic skills is well below average for age, or acceptable performance levels are achieved only with extraordinary effort. Specific learning disorder may occur in individuals identified as intellectually gifted and manifest only when the learning demands or assessment procedures (e.g., timed tests) pose barriers that cannot be overcome by their innate intelligence and compensatory strategies. For all individuals, specific learning disorder can produce lifelong impairments in activities dependent on the skills, including occupational performance. (from DSM-V)
Process in which individuals take the initiative, in diagnosing their learning needs, formulating learning goals, identifying resources for learning, choosing and implementing learning strategies and evaluating learning outcomes (Knowles, 1975)
A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of labeled paired input-output training (sample) data.
A MACHINE LEARNING paradigm used to make predictions about future instances based on a given set of unlabeled paired input-output training (sample) data.