Call for papers
The D-META Grand Challenge aims to set up the basis for comparison, analysis, and further improvement of multimodal data annotations and multimodal interactive systems. The main goal of this Grand Challenge is to foster research and development in multimodal communication and to further elaborate algorithms and techniques for building various multimodal applications. Held by two coupled pillars, method benchmarking and annotation evaluation, the D-META challenge envisions a starting point for transparent and publicly available application and annotation evaluation on multimodal data sets.
Scope
The D-META challenged is organized by tasks. Each tasks to be solved has one (or more) associated data set. In order to evaluate the performance on each task, a Golden Standard will be provided. Such standard consists of (partial) annotations and sets the ground truth over the data set concerning the targetted task. Authors may submit their methods/benchmarks for applications and/or their systems/comparisons for annotations and evaluation. We expect papers covering areas such as: (i) applications of an algorithm to a data set(s) to solve precise tasks, (ii) benchmark of several algorithms using the same data set(s), (iii) extensions of the annotation scheme with new relevant features, (iv) applications of the data to an automatic system, (v) discussions on ecologically valid data sets and (vi) position papers of how to organise the next challenge. In the following, the tasks are outlined, further in the text, they are detailed.
- AVRGR Recognize gestures addressed to the robot by means of the vision and the audio.
- AVSR Detect, localize and track multiple speakers using audio-visual information.
- CEP Estimate the level of engagement in a video-mediated communication.
- AVCGR Recognize conversational gestures in first encouter dialogues.
- AVFGR Recognize feedback gestures in first encouter dialogues.
Important dates
The schedule has the following important dates:
19-Mar-2012 Data set annotation is released
31-Jul-2012 Paper deadline
24-Aug-2012 Author notification
14-Sep-2012 Camera-ready
Oct-2012 Work presented at D-META’12
Format
The papers should be formatted following the ACM/IEEE format as in the ICMI 2012 proceedings. No more than six pages long, the manuscripts should contain the motivation, a brief description of the benchmarked methods, and an extensive discussion of the obtained results. No description of the data sets is needed, but the citation to the reference papers. A summary paper collecting the results presented by the authors will be published together with the ICMI proceedings.
