Track topics on Twitter Track topics that are important to you
Given a low-resolution (LR) image, multi-modal image super-resolution (MISR) aims to find the high-resolution (HR) version of this image with the guidance of an HR image from another modality. In this paper, we use a model-based approach to design a new deep network architecture for MISR. We first introduce a novel joint multi-modal dictionary learning (JMDL) algorithm to model cross-modality dependency. In JMDL, we simultaneously learn three dictionaries and two transform matrices to combine the modalities. Then, by unfolding the iterative shrinkage and thresholding algorithm (ISTA), we turn the JMDL model into a deep neural network, called deep coupled ISTA network. Since the network initialization plays an important role in deep network training, we further propose a layer-wise optimization algorithm (LOA) to initialize the parameters of the network before running back-propagation strategy. Specifically, we model the network initialization as a multi-layer dictionary learning problem, and solve it through convex optimization. The proposed LOA is demonstrated to effectively decrease the training loss and increase the reconstruction accuracy. Finally, we compare our method with other state-of-the-art methods in the MISR task. The numerical results show that our method consistently outperforms others both quantitatively and qualitatively at different upscaling factors for various multi-modal scenarios.
This article was published in the following journal.
Name: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration networ...
Deep learning methods, in particular, trained Convolutional Neural Networks (CNN) have recently been shown to produce compelling results for single image Super-Resolution (SR). Invariably, a CNN is le...
We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolut...
With exploiting contextual information over large image regions in an efficient way, the deep convolutional neural network has shown an impressive performance for single-image super-resolution (SR). I...
Face hallucination is a domain-specific super-resolution problem that aims to generate a high-resolution (HR) face image from a low-resolution (LR) input. In contrast to the existing patch-wise super-...
The aims of study on Angelman syndrome: (1)establish the Angelman syndrome database (2)to study the brain Network of Angelman Syndrome Based on Multi-modal Brain Image and Neural-EEG Data
The purpose of this study is to carry out a field trial to evaluate the effectiveness and added value of the b-SLIM digital Super Coach in a convenient sample of overweight and obese adult...
Aim: To evaluate the short-term effects of a multi-modal exercise program on physical performance variables in older adults with knee osteoarthritis.
Aim: To evaluate the short-term effects of a multi-modal exercise program on physical performance variables in older adults.
This is a preliminary study whose objectives are to define the clinical use cases and the constraints of the implementation of a multi-sensor image-sound system.
A method of delineating blood vessels by subtracting a tissue background image from an image of tissue plus intravascular contrast material that attenuates the X-ray photons. The background image is determined from a digitized image taken a few moments before injection of the contrast material. The resulting angiogram is a high-contrast image of the vessel. This subtraction technique allows extraction of a high-intensity signal from the superimposed background information. The image is thus the result of the differential absorption of X-rays by different tissues.
The production of an image obtained by cameras that detect the radioactive emissions of an injected radionuclide as it has distributed differentially throughout tissues in the body. The image obtained from a moving detector is called a scan, while the image obtained from a stationary camera device is called a scintiphotograph.
A visual image which is recalled in accurate detail. It is a sort of projection of an image on a mental screen.
A ubiquitously expressed G-protein-coupled receptor kinase subtype that has specificity for the agonist-occupied form of BETA-ADRENERGIC RECEPTORS and a variety of other G-PROTEIN-COUPLED RECEPTORS. Although it is highly homologous to G-PROTEIN-COUPLED RECEPTOR KINASE 2, it is not considered to play an essential role in regulating myocardial contractile response.
X-ray image-detecting devices that make a focused image of body structures lying in a predetermined plane from which more complex images are computed.
Arthritis Fibromyalgia Gout Lupus Rheumatic Rheumatology is the medical specialty concerned with the diagnosis and management of disease involving joints, tendons, muscles, ligaments and associated structures (Oxford Medical Diction...
Standard antiretroviral therapy (ART) consists of the combination of at least three antiretroviral (ARV) drugs to maximally suppress the HIV virus and stop the progression of HIV disease. Huge reductions have been seen in rates of death and suffering whe...