EMBODIED AGENTS

IN AUGMENTED & VIRTUAL REALITIES

Course E6998-004, Dept. of Computer Science, Columbia University, Fall 2002
Prof. Kris R. Thórisson, Ph.D.
 
 
 

 

LECTURE NOTES

LECTURE 3 PART 2

Embodiment, Action & Motor Control

September 26, 2002

 

  
     









15

Action Scheduling in Ymir

  

 

Two parts:

  • Motor Composer

  • Behavior Lexicon

  
 

Motor Composer: Takes Behavior Requests and prioritizes, composes a ‘muscle plan’ using information from the ...

  

 

Behavior Lexicon, which stores motor plans in a hierarchical fashion

  
     










16

Motor Composer

  

 

Receives Behavior Requests

  
 

Uses the Behavior Lexicon

 

 

Prioritization: depending on which layer the Request originiated from

  
 

  • Reactive Layer = high priority

  
 

  • Process Control Layer = medium priority

  
 

  • Content Layer = low priority

  
 

Resource allocation: Behavior morphologies are selected based on the amount of overlap that the required motors have with any currently executing behaviors

  
     










17

Motor Composer: Real-Time

  

Anytime algorithm

  
 

‘Do the best it can’

  

 

Checks for “time-out” (Expected Lifetime) based on...

 
 

...Date of Behavior Request

  
 

EL: Effective Lifetime

  
 

If EL is about to be reached, execute current morphology

Cancel [A] IF (Time-Now > time-stamp + EL )

where A is a behavior request, Time-Now is the time shown by the real-time clock, time-stamp is the time when the request was decided, and EL is behavior A’s Expected Lifetime

  
     










18

Behavior Lexicon

  

Behaviors are indexed functionally and morphologically, e.g. nod vs. show-I-agree

Nodding is done by moving the head back and forth. This is a morphological description (or label) of a behavior because it describes a movement

There are many ways, besides nodding, that you can use to ‘show you agree’. The same goes for ‘showing happiness’: These are indexed functionally, because the description ‘show happiness’ describes the function of a set of behaviors










19

Behavior Lexicon

  

Stored postures of acts/behaviors
Rosenbaum et al.: “Knowledge Model” of behavior control

  

Hierarchy: Behaviors are composed of smaller behavior segments; leafs are individual motors

  
 
 
 

‘Knowledge Model”
levels activated earlier provide information spanning longer stretches of time, e.g. the global act of moving your arm/hand/finger to enter the expression “27 + 9 + 3” into a calculator. Levels actuated later provide smaller and smaller constituents for that behavior, e.g. individual key presses

  
     










20

Behavior Lexicon

  
   
     









21

Behavior Lexicon

  

Show-Taking-Turn:

  
 

Glance-to-the-side

  • Move-motor-plh [pos-x, time]

  • Move-motor-plv [pos-y, time]

  • Move-motor-prh [pos-x, time]

  • Move-motor-prv [pos-y, time]

  
 

Lift-eyebrows

  • Move-motor-bll [pos-y, time]

  • Move-motor-blc [pos-y, time]

  • Move-motor-blm [pos-y, time]

  • Move-motor-brl [pos-y, time]

  • Move-motor-brc [pos-y, time]

  • Move-motor-brm [pos-y, time]

  
     












22

Face Control

  
 
 
 

brm = brow, right, medial

brc = brow, right, central

brl = brow, right, lateral

  
 
 
  







23

Action Selection

  
 

Two-phase:
1. Decision to behave (Deciders)
2. Behavior morphology selection (Action Scheduler)

  
 

Behavior Lexicon can be separated from decision mechanism

  
 

Exact look (morphology) of the action is delayed until the latest possible moment

  
 

Benefits:
added reactivity
Easy conflict resolution between degrees of freedom

  
     







24

Conflict Resolution
— ways to select between conflicting behavior morphologies

  
 

“Minimum Overlap”: select a morphology that has the least overlap in time or df with currently executing behaviors (implemented)

  
 

“Maximum Conspicuity”: select the most ‘obvious’ way for doing it, based on social norms, amount of arm movement, etc.

  
 

“Minimum Effort”: select a morphology that minimizes the amount of ‘energy’ needed to move the muscles

  
     





25

Spatial Behavior

  
 

Q: How does the agent coordinate its movemets with the outside world?

  
 

A: Recognize landmark objects, compute geometric deltas, and apply deltas to motors

  
     














26

Spatial Behavior

  
 

 
 
 













27

Spatial Behavior

  
 

 
 

Link Action Scheduler to a spatial knowledge base

  
 

Another advantage of delaying the composition of behavior morphology until the last minute: It will always be based on the most recently updated perceptual data

  
 
 










 

28

Behavior Lexicon Example

  
 
(setf *behavior-lexicon*
;GENERAL LAYOUT: (<list-of-acts> (<first-act>(<first-act-element>)(<second-act-element>))
; (<second-act (<...>))
; )
;ACT TEMPLATE: (name class (((act-name-of-option-1 delay exec-time)(act-name delay exec-time) etc*)
; (etc*)))
;MOTORS: (motor-name class delay exec-time pos/data)
'(
; MORPHOLOGICAL DEFINITIONS
;Features
;neutral
(face-neutral act (((mouth-neutral 100 400) (eyes-neutral 0 300)
(brows-neutral 0 500))))
(brows-neutral act (((left-brow-neutral 0 400)(right-brow-neutral 0 400))))
(left-brow-neutral mot-lev (((Bll 0 400 30)
(Blc 0 400 30) ;Brow, left, central
(Blm 0 400 30)))) ;Brow, left, medial
(right-brow-neutral mot-lev (((Brm 0 400 30)
(Brc 0 400 30)
(Brl 0 400 30))))
(eyes-neutral act (((upper-lids-neutral 0 100)(lower-lids-neutral 0 100))))
(upper-lids-neutral mot-lev (((Eru 0 100 75)(Elu 0 100 80))))
(upper-lids-open-wide mot-lev (((Eru 0 100 89)(Elu 0 100 94))))
(lids-neutral act (((upper-lids-neutral 0 300)(lower-lids-neutral 0 200))))
(mouth-neutral mot-lev (((Mb 0 200 15) ;Mouth, bottom
(Mlv 0 200 60) ;Mouth, left, vertical
(Mlh 0 200 40)
(Mrv 0 200 60)
(Mrh 0 200 40))))
(mouth-in-n-shape mot-lev (((Mb 0 200 15)(Mlv 0 40)(Mrv 0 40)(Mlh 0 50)(Mrh 0 50))))
(head-at-zero-zero mot-lev (((Hh 0 800 0)(Hv 0 150 0))))
(head-diag-up-left mot-lev (((Hh 0 1000 20)(Hv 0 1000 20)))) ;for debugging
(gaze-at-zero-zero mot-lev (((Plv 0 50 0)(Plh 0 50 0)(Prv 0 50 0)(Prh 0 50 0))))
;actions
(raise-brows mot-lev (((Bll 0 400 90)(Blc 0 300 100)(Blm 0 400 90)
(Brl 0 400 90)(Brc 0 300 100)(Brm 0 400 90))))
(lower-brows mot-lev (((Bll 0 400 5)(Blc 0 400 5)(Blm 0 400 5) (Brl 0 400 5)(Brc 0 400 5)(Brm 0 400 5))))
(brows-in-v-shape mot-lev (((Bll 0 400 90)(Blc 0 300 40)(Blm 0 400 10)
(Brl 0 400 90)(Brc 0 300 40)(Brm 0 400 10))))
(brows-in-roof-shape mot-lev (((Bll 0 400 10)(Blc 0 300 50)(Blm 0 400 90)
(Brl 0 400 10)(Brc 0 300 50)(Brm 0 400 70))))
(brows-in-n-shape mot-lev (((Bll 0 400 50)(Blc 0 300 90)(Blm 0 400 50)
(Brl 0 400 50)(Brc 0 300 90)(Brm 0 400 50))))
(brows-in-pensive-shape mot-lev (((Bll 0 400 95)(Blc 0 300 40)(Blm 0 400 40)
(Brl 0 400 50)(Brc 0 300 10)(Brm 0 400 5))))
(squint mot-lev (((Elu 0 300 60)(Eru 0 300 60)(Ell 0 300 20)(Erl 0 300 20))))
(half-closed-eyes mot-lev (((Elu 0 500 50)(Eru 0 500 60))))
(lower-lids-neutral mot-lev (((Erl 0 300 30)(Ell 0 300 50))))
(lower-lids-up mot-lev (((Ell 0 300 0)(Erl 0 400 0))))
(pull-l-mouth-corner mot-lev (((Mlh 0 500 90))))
(quickly-glance-sideways-and-back act (((gaze-right 0 100)(gaze-at-zero-zero 100 100))))

  
 

Part of Gandalf's Behavior Lexicon

  
     

 

 

 



<- PART 1—

2002©K.R.Thórisson