Call For Paper

IEEE Transactions on Autonomous Mental Development Special Issue on 

Multimodal Modeling and Analysis Informed by Brain Imaging

A challenging problem facing multimedia content analysis is the semantic gap between the high-level perception and cognition in the human brain and the low-level features embedded in digital contents. The human brain is the ultimate recipient and assessor of multimedia content and semantics. Deep understanding of the brain responses to multimedia will fundamentally advance the computational strategies for multimodal representation, classification and retrieval.

We envision a future with a seamless integration between cognitive neuroscience, a discipline related to the principle and mechanisms of the brain, and computer science, a discipline designing automated digital algorithms. Examples of such integration include neural network algorithms, which could reduce the semantic gap by mimicking the neural processes in the brain. Conversely, applications of automated computer algorithms have advanced our understanding of the brain. In the recent years, we have witnessed the emergence of novel brain-guided or brain-informed techniques in multimedia analysis and modeling, including computational visual attention models, sparse representation techniques, and deep learning techniques. These techniques have been applied to object recognition, image categorization, image/video compression, image/video retrieval, and video summarization.

The remarkable development of neuroimaging techniques such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG), has enabled us to probe the human brain in natural settings such as free viewing of multimedia contents. This development is leading a new trend that applies neuroscience and neuroimaging to assisting multimodal analysis and modeling. This new methodology has considerably narrowed the gaps between the low-level multimedia features and the high-level semantics. In parallel, neuroimaging combined with naturalistic stimuli such as films and music also provide neuroscientist an intriguing opportunity to examine the brain circuitry underlying natural experience. We believe that the integration of neuroimaging and multimedia will significantly advance our understanding on how the human brain perceive, process and assess multimodal contents, as well as developing effective algorithms for multimedia analysis.

This special issue will focus on the synergistic combinations of cognitive brain science, brain imaging, and multimedia analysis. It aims to capture the latest advances in the research community working on brain imaging-informed multimedia analysis, as well as computational model of the brain processes driven by multimedia contents. We are soliciting original contributions for:

Brain encoding and decoding models under natural multimedia (image/video/audio) via fMRI and EEG New development of brain computer interface (BCI) Models of brain functional interaction under natural multimedia stimulus Brain imaging informed multimedia content representation Brain imaging informed computational models for multimedia applications Brain imaging informed multimedia applications, including object recognition, image/video/audio categorization, image/video/audio retrieval and summarization, image/video/audio emotion or effective computing, and image/video/audio recommendation Clinical application of multimedia stimulus in brain disorders


Three kinds of submissions are possible:

Instructions for authors:
We are accepting submissions through Manuscript Central at (please select “Multimodal Modeling and Analysis Informed by Brain Imaging” as the submission type)
When submitting your manuscript, please also cc it to the editors.


10 November 2014:Deadline for paper submission (extended)
15 February 2015:Notification of acceptance
28 February 2015:Final manuscript
April 2015:Tentative publication date

Related Events

Real-life cognition challenge

Research groups and individuals are invited to participate in a contest to master following two challenges: 

The best three contributions will receive an award (3000, 1500, and 500 Euro respectively) sponsored by the Center for Behavioral Brain Sciences, Magdeburg, Germany. [To learn More]