Virtual & Augmented Worlds

Multimodal Knowledge

Perception

Decision

Action & Motor Control

Architecture

Virtual World

Augmented World

• Typical

"An autonomous or semi-autonomous program that acts on behalf of someone or something else."

Focus on: Delegation

• Our

"Any (partially or fully) autonomous system that approximates how people or animals behave, in that it has (1) a clearly distinguished perceptual apparatus to get information from the world, a (2) mind-like program for understanding and planning in the world, and (3) communicative & manipulative apparatus for conveying information to and manipulating the world.

Focus on: Perception, thought & communication

Embodiment = Physical Appearance

Embodiment connects to the outside world
- via its perceptual and motor apparatus

Example: Gandalf

Machine intelligence (A.I.) -> Communication

Communication -> Human-like communication

Humanoid communication -> Mutlimodal, Real-time, Embodied Humanoids

Psychological Models -> Machine intelligence (A.I.)

The "Star Wars™ Bar Problem":

How do you communicate with these characters?

Graphically drawn "world"
- fools the senses into believing in some alternative to the real world

Related: Virtual Reality, Synthetic Reality,

Example: LEGO Castle World

Reality with a "graphics overlay"

- can include virtual worlds

Examples:

Artifiical Intelligence

Multimodal: Speech, gesture, co-spatiality & co-temorality

Eyes: Attentional and deictic functions, both during speaking and listening.

Speech: Prosody — the timing and sequence of speech-related sounds and intonation

Speech content: What we say
(not how we say it -- but beware of interactions!)

Body: Direction of head and trunk — e.g. when user turns away to talk to someone else, and position of hands in body space (hand position relative to trunk and head).

Hand gestures: Deictic, iconic, emblematic, pantomimic, beat.

Turn-taking signals: Various feature-based analysis of combinations of co-occurring multimodal events, such as intonation, gaze, hand position and head direction.

Real-time: Timely, Interruptable, Sensitive to outside stimuli

Relative to human real-time

- Turn-taking

- Full perception-action loop

From Handbook of A.I.: "Artificial Intelligence is a field of science and engineering concerned with the computational understanding of what is commonly called intelligent behavior, and with the creation of artifacts that exhibit such behavior."

Minsky: Typically, 'Intelligence' is everything that people can do and machines cannot do, at any point in time

Alpha-Beta, A* - not A.I.!

Future of A.I.: Large, "ecological"-like systems with multiple knowledge bases, representations, planning systems, perception schemes, motor control schemes, and communication means.