Version 5 (Anonymous, 02/09/2010 09:59 am)

1 1
h1. SAIBA - Multimodal Behavior Generation Framework 
2 1
3 4
Lead by: Hannes Vilhjalmsson, Norman Badler, Lewis Johnson, Stefan Kopp, Brigitte Krenn, Stacy Marsella, Andrew N. Marshall, Catherine Pelachaud, Hannes Pirker, Kristinn R. Thorisson
4 1
5 1
The generation of natural multimodal output for embodied conversational agents requires a time-critical production process with high flexibility. To scaffold this production process and encourage sharing and collaboration, a working group of ECA researchers has introduced the SAIBA framework (Situation, Agent, Intention, Behavior, Animation). The framework specifies multimodal generation at a macro-scale, consisting of processing stages on three different levels: (1) planning of a communicative intent, (2) planning of a multimodal realization of this intent, and (3) realization of the planned behaviors.
6 2
7 2
8 2
9 2
10 4
h2. Introduction
11 2
12 2
The overall goal of this international effort is to unify a multimodal behavior generation framework for Embodied Conversational Agents (ECAs) so that people in the field can more easily work together and share resources.
13 2
14 2
So far the following research centers and institutions actively participate in the effort (alphabetical):
15 2
16 2
Articulab, Northwestern University, USA
17 2
Artificial Intelligence Group, University of Bielefeld, Germany
18 2
Austrian Research Institute for AI (OFAI), Vienna, Austria
19 2
Center for Analysis and Design of Intelligent Agents (CADIA), Reykjavik University, Iceland
20 2
Center for Human Modeling and Simulation, University of Pennsylvania, USA
21 2
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKA), Germany
22 2
Human Media Interaction, University of Twente, The Netherlands
23 2
Human-Oriented Technology Lab, University of Zagreb, Croatia
24 2
Information Sciences Institute (ISI), University of Southern California, USA
25 2
Institute for Creative Technologies (ICT), University of Southern California, USA
26 1
Intelligent Agents and Synthetic Characters Group at INESC, Lisbon, Portugal
27 1
IUT de Montreuil, University de Paris 8, France
28 2
29 4
h2. Overview
30 4
31 2
The first step towards a unifying representational framework for multimodal generation has been to lay down the general planning stages and knowledge structures that are involved in the creation of multimodal communicative behavior. We do not want to impose a particular micro-architecture. Yet, as our goal is to define representation languages that can serve as clear interfaces at separate levels of abstraction—building upon our experiences from previous ECA systems—we need to modularize the problem.
32 2
33 2
We aim for the representation languages to be:
34 2
35 2
Independent of a particular application or domain
36 2
Independent of the employed graphics and sound player model
37 2
Represent a clear-cut separation between information types (function-related versus process-related specification of behavior)
38 2
The generation of natural multimodal output requires a time-critical production process with high flexibility. To scaffold this production process we introduced the SAIBA framework (Situation, Agent, Intention, Behavior, Animation), and specify the macro-scale multimodal generation consisting of processing stages on three different levels:
39 2
40 2
Planning of a communicative intent
41 2
Planning of multimodal behaviors that carry out this intent
42 2
Realization of the planned behaviors
43 1
These processing stages are depicted below:
44 1
45 1
46 4
47 4
The interface between stages (1) and (2) — Intent Planning and Behavior Planning — describes communicative and expressive intent without any reference to physical behavior. We call the language that we propose for specifying such information the Function Markup Language (FML). It is meant to provide a semantic description that accounts for the aspects that are relevant and influential in the planning of verbal and nonverbal behavior.
48 4
49 4
The interface between (2) and (3) — Behavior Planning and Behavior Realization — describes multimodal behaviors as they are to be realized by the final stage of the generation process. We propose the Behavior Markup Language (BML) for this purpose. It provides a general, player-independent description of multimodal behavior that can be used to control an embodied agent. Nevertheless, it needs to provide a sufficient level of detail in describing behavior, from the mere occurrence and the relative timing of the involved actions, to the detailed definition of a behavior’s form.
50 4
51 4
h2. Workshops 
52 4
53 5
54 4
Amsterdam, The Netherlands, September 13, 2009
55 4
56 5
57 4
Budapest, Hungary, May 12, 2009
58 1
59 5
The 3rd BML Workshop, hosted by "MITRE Corporation":
60 4
Boston, USA, June 2-3, 2008
61 5
62 5
63 4
64 4