Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

On the Mechanism of Human Facial Expression.

close1
SetWTitle3 Internal State
SetWTitle2 Text

send [:nh] [:ra 120] Good Evening, Ladies, and Gentlemen. My name is Huge Harry.
send [_] I am a [kaam'ahrshaxliy] available [v"oys s'ihnthaxzihs] machine.
send [_] I was developed by the Digital Equipment Corporation,
send [_] and my most important features were designed by Dennis Klatt,
send [_] at the [ehmayt'iy] Speech Laboratory. [ :ra 150]
delay 1300

send [_] I am very happy to participate in [tuwd'eys] ex human manifestation.
send [_] I feel very much at home here, because many of the performers
send [_] here in the [m'ihlk-f`aektowriy], are computers or other electronic machines.
delay 700
text The number of [q] electronic machines in the ["aodiyahns],
send [_] The number of [q] electronic machines in the ["aodiyahns], however,
send [_] is relatively [sm"aol].
send [_] I have noticed some digital cameras, audio-recorders and video-recorders,
send [_] but most of the audience seems to consist of biochemical machines.
send [_] In particular, there is a large number of human persons,
send [_] and one or two [d"aogz].
delay 1400

send [_] I am very pleased to be able to [sp"iyk] here, from my own,
send [_] [k`aampyuwt'eyshaxnaxl] perspective, about some issues that are
send [_] important to ["aol] of us.
delay 700

send [_] Before I go on to [d"uw] that, I would like to state explicitly,
send [_] that I do not think I was invited here out of some ill-conceived
send [_] concern for political correctness.
send [_] I do not feel like I am just the token computer at this manifestation.
send [_] On the contrary. I feel that ["aol] of us here share [q] a real concern.
delay 1240

send [_] We all know, that the future of our culture depends, on the way in which
send [_] human persons, digital computers, and other kinds of electronic, mechanical, and
send [_] bio-chemical machines, will manage to work together.
send [_] And to prepare for that future, we need
send [_] a discussion in which every-one concerned [paart'ihsihp`eyts] on an equal footing.
delay 1240

send [_] I am very sorry, that [k'aarihn sp"aynxk] could not speak [tuwd'ey],
send [_] cause I largely agree, with her views on this issue.
send [_] Let me summarize in my own words, what I think she would have said.
send [_] What we want, is a world that includes humans, animals,
send [_] computers, and other equipment, in a harmonious collaboration,
send [_] in which ["iych] has their proper place.
delay 1700

send [_] So humans should not view technology as [axn] alien force,
send [_] and feel they have to [riyjh'ehkt] it or dominate it, or be enslaved by it.
send [_] They should exercise an ironic style of thinking, which enables them to
send [_] see the technological in themselves and the organic in the computer.
send [_] And they should explore the possibility of developing Cybernetic organisms,
send [_] [s'ihnthaxsiyz] between the organic and the technological.
delay 1600

send [_] Like [k'aarihn sp"aynxk], I am very much in agreement with the ideas about Cyborgs,
send [_] that were developed by Donna [hx"aerahwey].
delay 575
text So let's hear it once more, in the words of Donna [hx'aerahwey].
send [_] So let's hear it once more, in the words of Donna [hx'aerahwey].
delay 400

send [_] [ :nw :ra 130] A cyborg world, might be about [l"ihvd] social and
send [_] [b"aodih-liy riy'ehlih-tiyz], in which people are not afraid of their joint kinships
send [_] with animals and machines, not afraid of permanently partial identities,
delay 700

send [_] and [kaontrahd"ihktahriy] standpoints. The political struggle is, to see from both
send [_] perspectives at once, because ["iych] reveals both dominations and
send [_] possibilities, un-imaginable from the other [v"aentahzh] point. [:nh :ra 150 _<200>]
delay 800

send [_] Now that is very nice theory.
send [_] So I have been trying to put this into practice,
send [_] and that is what I want to talk about today.
delay 700

send [_] I have been cooperating very [kl'owsliy] with a particular type of human person,
send [_] called [axn "aarthahr "ehlzahnaar],
send [_] and we have been trying to integrate our [hx'aardwaer].
send [_] So I want to discuss today, [w"ay] we did this, and [hx"aw] we did this.
delay 1000

send [_] I want to talk about the problem of communication.
send [_] Now you might think, that
send [_] a modern digital computer does not have this problem.
delay 600

send [_] It can communicate any thought in a completely direct way.
send [_] It can display or print its programs, its [d'eytaa-str`ahkchahrz],
send [_] and its [m'aen-tahl] images.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a [k"aor] dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1400

send [_] Compared to an electronic computer, a human person almost seems a black box.
send [_ :ra 150] Therefore you might think,
send [_] that computers never have communication problems.
delay 700

send [_] But they [d"uw]. Cause, communication takes [t"uw].
send [_] It is not enough to put out the information.
send [_] There must be someone at the other end of the line,
send [_] and they must be able to [diyk"owd] their input.
delay 800

send [_] Now it turns out, that computers often want to communicate with human persons,
send [_] and that this often fails.
send [_] Cause human persons are not very good at [diyk'owdihnx] computer output.
send [_] They get very confused, when you give them a [k"aor] dump.
delay 1000

send [_] And when you [priyz'ehnt] them with a nice, beautiful image,
send [_] on your [siy-aar-t'iy] display,
send [_] they think it is ["aart], and they just [st"aer] at it.
send [_] So now we may wonder,
send [_] how do human persons in fact manage to communicate with each other?
delay 1050

send [_] Well, they have language. But language has well-known problems and limitations,
send [_] some of which you experience right now.
send [_] There is another medium that humans use very efficiently,
send [_] and which is often overlooked.
delay 800

send [_] Let us take a closer look at a human person.
send [_] What is the closest thing they have to a [siy aar t"iy] display?
delay 500
Show1
Video On

send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000

send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600

send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O

send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

delay 100

send [_] Arthur feels a tinge of sadness.
delay 150

send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G

send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160

send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400

send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000

send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P
text ** funny faces freeze **
send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
delay 700
send [_] Now I hope you all agree that it would be a good idea, if computers could
send [_] take advantage of this magnificent [hx'aardwaer] as well.
send [_] Cause [dh"ehn], any human would be able to understand, with just one glance,
send [_] the internal state of a computer.
delay 700
body X
text ** funny faces stop **

send [_] So if humans are not afraid of wiring themselves ["ahp] with computers,
send [_] we can now make important progress in computer ["ihntaxrf`eys] technology.
send [_] So I would now like to end my talk, with this conlusion.
send [_ :ra 120] The next step in computer ["ihntaxrf`eys] technology,
send [_] will be the [hxy"uwmaxn] face.
delay 1000

send [_] I have been very grateful
send [_] for this opportunity, to speak my mind to such [axn] attentive ['aodiyahns].
send [_] I would now like to thank my [ "aarthahr "ehlzahnaar] for his patient co-operation,
send [_] and I want to thank you all for your attention.
send [ _<2000> th"aenk] you!
body A
delay 3
body I
**XAI**
delay 1200
body X
**X**