Fully complex conjugate gradient-based neural networks using Wirtinger calculus framework: Deterministic convergence and its application.

08:00 EDT 26th March 2019 | BioPortfolio

Summary of "Fully complex conjugate gradient-based neural networks using Wirtinger calculus framework: Deterministic convergence and its application."

Conjugate gradient method has been verified to be one effective strategy for training neural networks due to its low memory requirements and fast convergence. In this paper, we propose an efficient conjugate gradient method to train fully complex-valued network models in terms of Wirtinger differential operator. Two ways are adopted to enhance the training performance. One is to construct a sufficient descent direction during training by designing a fine tuning conjugate coefficient. Another technique is to pursue the optimal learning rate instead of a fixed constant in each iteration which is determined by employing a generalized Armijo search. In addition, we rigorously prove its weak and strong convergence results, i.e., the gradient norms of objective function with respect to weights approach zero along with the increasing iterations and the weight sequence tends to the optimal point. To verify the effectiveness and rationality of the proposed method, four illustrated simulations have been performed on both typical regression and classification problems.


Journal Details

This article was published in the following journal.

Name: Neural networks : the official journal of the International Neural Network Society
ISSN: 1879-2782
Pages: 50-64


DeepDyve research library

PubMed Articles [21777 Associated PubMed Articles listed on BioPortfolio]

Gradient based hyperparameter optimization in Echo State Networks.

Like most machine learning algorithms, Echo State Networks possess several hyperparameters that have to be carefully tuned for achieving best performance. For minimizing the error on a specific task, ...

Ensemble Neural Networks (ENN): A gradient-free stochastic method.

In this study, an efficient stochastic gradient-free method, the ensemble neural networks (ENN), is developed. In the ENN, the optimization process relies on covariance matrices rather than derivative...

Stochastic Conjugate Gradient Algorithm With Variance Reduction.

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance...

Fixed-time synchronization of inertial memristor-based neural networks with discrete delay.

This paper is concerned with the fixed-time synchronization control of inertial memristor-based neural networks with discrete delay. We design four different kinds of feedback controllers, under which...

Clone-Based Encoded Neural Networks to Design Efficient Associative Memories.

In this paper, we introduce a neural network (NN) model named clone-based neural network (CbNN) to design associative memories. Neurons in CbNN can be cloned statically or dynamically which allows to ...

Clinical Trials [5346 Associated Clinical Trials listed on BioPortfolio]

Computerized Intervention Targeting Cognitive Control Deficits in Depressed Adults

Project:EVO is a video-game based intervention that targets neural networks associated with cognitive control. The same networks have been implicated in poor treatment response in middle a...

Resting-State Neural Connectivity in Patients With Subjective Tinnitus Without Bother

Tinnitus is the occurrence of an auditory sensation without the presence of an acoustic stimulus. Approximately, 50 million people in the United States experience chronic tinnitus and 15 m...

Contributions From the Analysis of Graphs for Identification of Neural Cliques

The aim of the study is to demonstrate that our semantic knowledge (elements of our long-term memory and the process we use them) respond to a graphic organisation and gather together foll...

Brain-Based Biomarkers in Response to TMS in MDD

The overarching goal of this research program is to elucidate causal and directional neural network- level abnormalities in depression, and how they are modulated by an individually-tailor...

Comparison of RVOT Gradient Under Anaesthesia With Post-operative Gradient in Patients Undergoing TOF Repair

The primary objective of the study will be to compare intraoperative post TOF repair RVOT gradient under two different anaesthetic depths. Secondary objectives will be to follow up change ...

Medical and Biotech [MESH] Definitions

A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.

Complex sets of enzymatic reactions connected to each other via their product and substrate metabolites.

Separation of particles according to density by employing a gradient of varying densities. At equilibrium each particle settles in the gradient at a point equal to its density. (McGraw-Hill Dictionary of Scientific and Technical Terms, 4th ed)

A technique used to separate particles according to their densities in a continuous density gradient. The sample is usually mixed with a solution of known gradient materials and subjected to centrifugation. Each particle sediments to the position at which the gradient density is equal to its own. The range of the density gradient is usually greater than that of the sample particles. It is used in purifying biological materials such as proteins, nucleic acids, organelles, and cell types.

An early embryonic developmental process of CHORDATES that is characterized by morphogenic movements of ECTODERM resulting in the formation of the NEURAL PLATE; the NEURAL CREST; and the NEURAL TUBE. Improper closure of the NEURAL GROOVE results in congenital NEURAL TUBE DEFECTS.

Quick Search


DeepDyve research library

Searches Linking to this Article