Track topics on Twitter Track topics that are important to you
Multimodal emotion understanding enables AI systems to interpret human emotions. With accelerated video surge, emotion understanding remains challenging due to inherent data ambiguity and diversity of video content. Although deep learning has made a considerable progress in big data feature learning, they are viewed as deterministic models used in a "black-box" manner which does not have capabilities to represent inherent ambiguities with data. Since the possibility theory of fuzzy logic focuses on knowledge representation and reasoning under uncertainty, we intend to incorporate the concepts of fuzzy logic into deep learning framework. This paper presents a novel convolutional neuro-fuzzy network, which is an integration of convolutional neural networks in fuzzy logic domain to extract high-level emotion features from text, audio, and visual modalities. The feature sets extracted by fuzzy convolutional layers are compared with those of convolutional neural networks at the same level using t-distributed Stochastic Neighbor Embedding. This paper demonstrates a multimodal emotion understanding framework with an adaptive neural fuzzy inference system that can generate new rules to classify emotions. For emotion understanding of movie clips, we concatenate audio, visual, and text features extracted using the proposed convolutional neuro-fuzzy network to train adaptive neural fuzzy inference system. In this paper, we go one step further to explain how deep learning arrives at a conclusion that can guide us to an interpretable AI. To identify which visual/text/audio aspects are important for emotion understanding, we use direct linear non-Gaussian additive model to explain the relevance in terms of causal relationships between features of deep hidden layers. The critical features extracted are input to the proposed multimodal framework to achieve higher accuracy.
This article was published in the following journal.
Name: Neural networks : the official journal of the International Neural Network Society
The Lung nodules are very important to indicate the lung cancer, and its early detection enables timely treatment and increases the survival rate of patient. Even though lots of works are done in this...
Adaptive neuro-fuzzy inference system (ANFIS) was employed for the prediction of oxidative stability of virgin olive oil (VOO) during storage as a function of time, storage temperature, total polyphen...
In this study, the binding of allergens to antibody-receptor complexes was investigated. This process is important in understanding the allergic response. A BioNetGen model that simulates this process...
The modelling of biological systems is accompanied with epistemic uncertainties that range from structural uncertainty to parametric uncertainty due to such limitations as insufficient understanding o...
Schizophrenia is associated with an increased violence risk, particularly homicide. One possible, but scarcely explored, contributor to the increased violence risk is social cognitive impairment. Indi...
Glaucoma is currently the leading cause of irreversible blindness in the world. The multi-center study is designed to evaluate the efficacy of the convolutional neural network based algori...
This study evaluates the benefit of Neuro 1 sound processor upgrade in speech perfomance in adults. Half of participants will be tested with Neuro 1 first and Neuro 2, while the other half...
The study will examine the neural and behavioral correlates of emotion regulation in adolescents engaging in binge eating and/or purging and healthy adolescents. Furthermore, it will look ...
For centuries the term "blood curling" has been used to describe extreme frightening situations. However, the origin of this ancient theory has never been studied and it remains unknown if...
The purpose of the study is to measure the interest of a movie explaining the path of the children in surgery, in order to reduce the anxiety of the children and his parents. This study i...
Approximate, quantitative reasoning that is concerned with the linguistic ambiguity which exists in natural or synthetic language. At its core are variables such as good, bad, and young as well as modifiers such as more, less, and very. These ordinary terms represent fuzzy sets in a particular problem. Fuzzy logic plays a key role in many medical expert systems.
The act or fact of grasping the meaning, nature, or importance of; understanding. (American Heritage Dictionary, 4th ed) Includes understanding by a patient or research subject of information disclosed orally or in writing.
The process of helping patients to effectively and efficiently use the health care system when faced with one or more of these challenges: (1) choosing, understanding, and using health coverage or applying for assistance when uninsured; (2) choosing, using, and understanding different types of health providers and services; (3) making treatment decisions; and (4) managing care received by multiple providers.
The use of combination of imaging techniques or platforms (e.g., MRI SCAN and PET SCAN) encompassing aspects of anatomical, functional, or molecular imaging methods.
The circulation of the BLOOD through the MICROVASCULAR NETWORK.
Neurology - Central Nervous System (CNS)
Alzheimer's Disease Anesthesia Anxiety Disorders Autism Bipolar Disorders Dementia Epilepsy Multiple Sclerosis (MS) Neurology Pain Parkinson's Disease Sleep Disorders Neurology is the branch of me...