Deep Coupled ISTA Network for Multi-modal Image Super-Resolution.

08:00 EDT 3rd October 2019 | BioPortfolio

Summary of "Deep Coupled ISTA Network for Multi-modal Image Super-Resolution."

Given a low-resolution (LR) image, multi-modal image super-resolution (MISR) aims to find the high-resolution (HR) version of this image with the guidance of an HR image from another modality. In this paper, we use a model-based approach to design a new deep network architecture for MISR. We first introduce a novel joint multi-modal dictionary learning (JMDL) algorithm to model cross-modality dependency. In JMDL, we simultaneously learn three dictionaries and two transform matrices to combine the modalities. Then, by unfolding the iterative shrinkage and thresholding algorithm (ISTA), we turn the JMDL model into a deep neural network, called deep coupled ISTA network. Since the network initialization plays an important role in deep network training, we further propose a layer-wise optimization algorithm (LOA) to initialize the parameters of the network before running back-propagation strategy. Specifically, we model the network initialization as a multi-layer dictionary learning problem, and solve it through convex optimization. The proposed LOA is demonstrated to effectively decrease the training loss and increase the reconstruction accuracy. Finally, we compare our method with other state-of-the-art methods in the MISR task. The numerical results show that our method consistently outperforms others both quantitatively and qualitatively at different upscaling factors for various multi-modal scenarios.


Journal Details

This article was published in the following journal.

Name: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
ISSN: 1941-0042


DeepDyve research library

PubMed Articles [14533 Associated PubMed Articles listed on BioPortfolio]

Adversarial learning for mono- or multi-modal registration.

This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration networ...

Adaptive Transform Domain Image Super-resolution Via Orthogonally Regularized Deep Networks.

Deep learning methods, in particular, trained Convolutional Neural Networks (CNN) have recently been shown to produce compelling results for single image Super-Resolution (SR). Invariably, a CNN is le...

Deep learning-based super-resolution in coherent imaging systems.

We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolut...

Deep Inception-Residual Laplacian Pyramid Networks for Accurate Single-Image Super-Resolution.

With exploiting contextual information over large image regions in an efficient way, the deep convolutional neural network has shown an impressive performance for single-image super-resolution (SR). I...

Face Hallucination by Attentive Sequence Optimization with Reinforcement Learning.

Face hallucination is a domain-specific super-resolution problem that aims to generate a high-resolution (HR) face image from a low-resolution (LR) input. In contrast to the existing patch-wise super-...

Clinical Trials [6432 Associated Clinical Trials listed on BioPortfolio]

Study on the Brain Network of Angelman Syndrome

The aims of study on Angelman syndrome: (1)establish the Angelman syndrome database (2)to study the brain Network of Angelman Syndrome Based on Multi-modal Brain Image and Neural-EEG Data

B-slim, a Multi-source Digital Super Coach for Sustainable Weight Loss

The purpose of this study is to carry out a field trial to evaluate the effectiveness and added value of the b-SLIM digital Super Coach in a convenient sample of overweight and obese adult...

Multi-modal Exercise Program in Older Aduls With Knee Osteoarthritis

Aim: To evaluate the short-term effects of a multi-modal exercise program on physical performance variables in older adults with knee osteoarthritis.

Multi-modal Exercise Program in Older Adults

Aim: To evaluate the short-term effects of a multi-modal exercise program on physical performance variables in older adults.

Design of a Non-invasive Multi-modal Neonatal Monitoring System

This is a preliminary study whose objectives are to define the clinical use cases and the constraints of the implementation of a multi-sensor image-sound system.

Medical and Biotech [MESH] Definitions

A method of delineating blood vessels by subtracting a tissue background image from an image of tissue plus intravascular contrast material that attenuates the X-ray photons. The background image is determined from a digitized image taken a few moments before injection of the contrast material. The resulting angiogram is a high-contrast image of the vessel. This subtraction technique allows extraction of a high-intensity signal from the superimposed background information. The image is thus the result of the differential absorption of X-rays by different tissues.

The production of an image obtained by cameras that detect the radioactive emissions of an injected radionuclide as it has distributed differentially throughout tissues in the body. The image obtained from a moving detector is called a scan, while the image obtained from a stationary camera device is called a scintiphotograph.

A visual image which is recalled in accurate detail. It is a sort of projection of an image on a mental screen.

A ubiquitously expressed G-protein-coupled receptor kinase subtype that has specificity for the agonist-occupied form of BETA-ADRENERGIC RECEPTORS and a variety of other G-PROTEIN-COUPLED RECEPTORS. Although it is highly homologous to G-PROTEIN-COUPLED RECEPTOR KINASE 2, it is not considered to play an essential role in regulating myocardial contractile response.

X-ray image-detecting devices that make a focused image of body structures lying in a predetermined plane from which more complex images are computed.

Quick Search

DeepDyve research library

Relevant Topics

Arthritis Fibromyalgia Gout Lupus Rheumatic Rheumatology is the medical specialty concerned with the diagnosis and management of disease involving joints, tendons, muscles, ligaments and associated structures (Oxford Medical Diction...

Antiretroviral therapy
Standard antiretroviral therapy (ART) consists of the combination of at least three antiretroviral (ARV) drugs to maximally suppress the HIV virus and stop the progression of HIV disease. Huge reductions have been seen in rates of death and suffering whe...

Searches Linking to this Article