Attention-Aware Multi-Task Convolutional Neural Networks.

08:00 EDT 4th October 2019 | BioPortfolio

Summary of "Attention-Aware Multi-Task Convolutional Neural Networks."

Multi-task deep learning methods learn multiple tasks simultaneously and share representations amongst them, so information from related tasks improves learning within one task. The generalization capabilities of the produced models are substantially enhanced. Typical multi-task deep learning models usually share representations of different tasks in lower layers of the network, and separate representations of different tasks in higher layers. However, different groups of tasks always have different requirements for sharing representations, so the required design criterion does not necessarily guarantee that the obtained network architecture is optimal. In addition, most existing methods ignore the redundancy problem and lack the pre-screening process for representations before they are shared. Here, we propose a model called Attention-aware Multi-task Convolutional Neural Network, which automatically learns appropriate sharing through end-to-end training. The attention mechanism is introduced into our architecture to suppress redundant contents contained in the representations. The shortcut connection is adopted to preserve useful information. We evaluate our model by carrying out experiments on different task groups and different datasets. Our model demonstrates an improvement over existing techniques in many experiments, indicating the effectiveness and the robustness of the model. We also demonstrate the importance of attention mechanism and shortcut connection in our model.


Journal Details

This article was published in the following journal.

Name: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
ISSN: 1941-0042


DeepDyve research library

PubMed Articles [14250 Associated PubMed Articles listed on BioPortfolio]

Self Multi-Head Attention-based Convolutional Neural Networks for fake news detection.

With the rapid development of the internet, social media has become an essential tool for getting information, and attracted a large number of people join the social media platforms because of its low...

Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization.

Convolutional neural networks (CNNs) have proven successful as function approximators and have therefore been used for classification problems including electroencephalography (EEG) signal decoding fo...

Automatic classification of dopamine transporter SPECT: deep convolutional neural networks can be trained to be robust with respect to variable image characteristics.

This study investigated the potential of deep convolutional neural networks (CNN) for automatic classification of FP-CIT SPECT in multi-site or multi-camera settings with variable image characteristic...

GRUU-Net: Integrated convolutional and gated recurrent neural network for cell segmentation.

Cell segmentation in microscopy images is a common and challenging task. In recent years, deep neural networks achieved remarkable improvements in the field of computer vision. The dominant paradigm i...

Dynamic Pixel-wise Weighting-based Fully Convolutional Neural Networks for Left Ventricle Segmentation in Short-axis MRI.

Left ventricle (LV) segmentation in cardiac MRI is an essential procedure for quantitative diagnosis of various cardiovascular diseases. In this paper, we present a novel fully automatic left ventricl...

Clinical Trials [6558 Associated Clinical Trials listed on BioPortfolio]

Renal Cancer Detection Using Convolutional Neural Networks

We aim to experiment and implement various deep learning architectures in order to achieve human-level accuracy in Computer-aided diagnosis (CAD) systems. In particular, we are interested ...

Assessment of Stability of Behavioral and Neural Measures of Attention Networks Across Multiple Sessions

The purpose of this research study is to learn more about the stability over multiple recording sessions of brainwave activity during a computerized task that measures different types of a...

Neural Mechanisms of Attention Lapses in Adult ADHD

ADHD is a common disorder, leading to a significant disability that often persists in adulthood. ADHD is characterized by attentional disturbances that are difficult to asses with standard...

Diagnostic Efficacy of CNN in Differentiation of Visual Field

Glaucoma is currently the leading cause of irreversible blindness in the world. The multi-center study is designed to evaluate the efficacy of the convolutional neural network based algori...

Early Discrimination of Periprosthetic Hip Infections Using Neural Networks (SEPTIC-ANNR)

The study is about the role of cellular neural networks-genetic algorithm in the diagnosis of periprosthetic hip infections. A retrospective case series of septic and aseptic loosening of ...

Medical and Biotech [MESH] Definitions

A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.

Single or multi-sheet notices made to attract attention to events, activities, causes, goods, or services. They are for display, usually in a public place and are chiefly pictorial.

The observation and analysis of movements in a task with an emphasis on the amount of time required to perform the task.

An early embryonic developmental process of CHORDATES that is characterized by morphogenic movements of ECTODERM resulting in the formation of the NEURAL PLATE; the NEURAL CREST; and the NEURAL TUBE. Improper closure of the NEURAL GROOVE results in congenital NEURAL TUBE DEFECTS.

Loss of higher cortical functions with retained awareness due to multiple cortical or subcortical CEREBRAL INFARCTION. Memory, judgment, attention span, and impulse control are often impaired, and may be accompanied by PSEUDOBULBAR PALSY; HEMIPARESIS; reflex abnormalities, and other signs of localized neurologic dysfunction. (From Adams et al., Principles of Neurology, 6th ed, p1060)

Quick Search

DeepDyve research library

Searches Linking to this Article