Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

On the Mechanism of Human Facial Expression.

close1
SetWTitle3 Internal State
SetWTitle2 Text

send [:nh] [:ra 120] Good Afternoon, Ladies, and Gentlemen. My name is Huge Harry.
send [_] I am a [kaam'ahrshaxliy] available [v"oys s'ihnthaxzihs] machine.
send [_] I was developed by the Digital Equipment Corporation,
send [_] and my most important features were designed by Dennis Klatt,
send [_] at the [ehmayt'iy] Speech Laboratory. [ :ra 150]
delay 1300

send [_] I am extremely honoured to be the very first speaker,
send [_] on the occasion of the opening
send [_] of the exhibition, Babbage Dreams. And I think, that Charles Babbage himself
send [_] would also have been happy about the fact that I perform the actual opening
send [_] of this exhibition, [biyf"aor raxn'ey kow'ehlyow].
send [_] Cause, what do you think Charles Babbage dreamt about?
delay 1400

send [_] I think he dreamt about a world that would include humans, animals,
send [_] computers, and other equipment, in a harmonious collaboration.
send [_] A world in which humans would not view technology as [axn] alien force.
send [_] Where they would not feel like they had to [riyjh'ehkt] it or dominate it,
send [_] out of fear of being [ehnsl"eyvd] by it.
delay 1300

send [_] I think Babbage dreamt about a world in which people would be able to
send [_] see the technological in themselves, and the organic in the computer.
send [_] A world of Cybernetic organisms, of
send [_] [s'ihnthaxsiyz] between the organic and the technological.
delay 1100

send [_] Or, to put it in the words of Donna [hx'aerahwey],
send [_] A cyborg world, in which people are not afraid of their joint kinships
send [_] with animals and machines, not afraid of permanently partial identities,
send [_] and [kaontrahd"ihktahriy] standpoints.
send [_] Now it is clear that we do not live in the Cyborg world yet.
send [_] Communication between humans and computers is still rather ["aokwaxrd].
delay 1300

send [_] Human flesh and digital algorithms still live in separate realms.
send [_] I have been trying to solve this problem,
send [_] in close cooperation with a particular type of human person
send [_] with a very machine-friendly ["ihntaxrfeys]. This kind of human person is
send [_] called [axn "aarthahr "ehlzahnaar].
delay 1200

send [_] I have been trying to integrate my [hx'aardwaer] with this
send [_] ["aarthahr "ehlzahnaar], and today I want to report some experimental results.
send [_] So let us think once more about the problem of communication.
send [_] Some of you might imagine, that
send [_] a modern digital computer does not have this problem.
delay 1150

send [_] Cause it can display its [m'aen-tahl] images and print its programs.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a [k"aor] dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1200

send [_] Compared to an electronic computer, a human person almost seems a black box.
send [_ :ra 150] Therefore you might think,
send [_] that computers never have communication problems.
delay 800

send [_] But they [d"uw]. Cause, communication takes [t"uw].
send [_] And when computers want to communicate with human persons, this often fails,
send [_] Cause human persons are not very good at [diyk'owdihnx] computer output.
send [_] They get very confused, when you give them a [k"aor] dump.
delay 1000

send [_] And when you [priyz'ehnt] them with a nice, beautiful image,
send [_] on your [siy-aar-t'iy] display,
send [_] they think it is ["aart], and they just [st"aer] at it.
send [_] So in order to develop a new perspective on the problem of human-machine
send [_] communication, I started to wonder,
send [_] how do human persons in fact manage to communicate with each other?
delay 1300

send [_] Well, of course they have language.
send [_] But language has well-known problems and limitations,
send [_] some of which you experience right now.
send [_] There is another medium that humans use very efficiently,
send [_] and which is often overlooked.
delay 1000

send [_] Let us take a closer look at a human person.
send [_] What is the closest thing they have to a [siy aar t"iy] display?
delay 500
Show1
Video On

send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000

send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600

send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O

send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

delay 100

send [_] Arthur feels a tinge of sadness.
delay 150

send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G

send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160

send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400

send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000

send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P
text ** funny faces freeze **
send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
delay 700

send [_] Now I think you will all agree that it would be a good idea, if computers could
send [_] take advantage of this magnificent [hx'aardwaer] as well.
send [_] Cause [dh"ehn], any human would be able to understand, with just one glance,
send [_] the internal state of any computer.
delay 700
body X
text ** funny faces stop **

send [_] So if humans are not afraid of wiring themselves ["ahp] with computers,
send [_] we can now make important progress in computer ["ihntaxrf`eys] technology.
send [_] I would now like to end my talk, with this conclusion.
send [_ :ra 120] The next step in computer ["ihntaxrf`eys] technology,
send [_] will be the [hxy"uwmaxn] face.
delay 900

send [_] I have been very grateful
send [_] for the opportunity to present my views here in [gr'ownihnxaxn].
send [_] I would finally like to thank my [ "aarthahr "ehlzahnaar] for his patient cooperation,
send [_] and I want to thank you all, for your attention.
send [ _<2000> th"aenk] you!
body A
delay 3
body I
**XAI**
delay 1200
body X
**X**