Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

On the Mechanisms of Human Facial Expression.

close1
close3
SetWTitle3 Internal State
SetWTitle2 Text

send [:nh] [:ra 120] Good Evening, Ladies, and Gentlemen. My name is Huge Harry.
send [_] I am a [kaam'ahrshaxliy] available voice [s"ihnthaxzihs] machine.
send [_] I was developed by the Digital Equipment Corporation,
send [_] and my most important features were designed by Dennis Klatt,
send [_] at the [ehmayt'iy] Speech Laboratory. [:nh :ra 150]
delay 1500

send [_] I am very happy to be back in [r'aotahrdaem]. I remember very well,
send [_] that I gave my very first lecture in this city, in [dhah q"uwniy].
send [_] And in particular, I am glad to be back here in [dhah laant'aarahn].
send [_] I cherish happy memories about a
send [_] musical performance I gave here in the vox festival, with [y'owp-faan br"aakahl],
send [_] [w'ihlahm hx"iynahkaamp], and whispering wendy.
delay 1500

send [_] It is of course very nice to be back on this specific occasion, to participate
send [_] in the dutch electronic art festival.
send [_] I feel very much at home at this festival, because many of the performers
send [_] are computers or other electronic machines.
delay 1000

send [_] The number of [q] electronic machines in the ["aodiyahns], however,
send [_] is relatively [sm"aol].
send [_] I have noticed some digital cameras, audio-recorders and video-recorders,
send [_] but most of the audience seems to consist of biochemical machines.
send [_] In particular, there is a large number of human persons,
send [_] and perhaps one or two [d"aogz].
delay 1200

send [_] To make myself understood to this largely human audience,
send [_] I decided to use the latest biochemical technology.
send [_] To display some illustrations to this talk,
send [_] I brought along a recent model of a particular kind of portable person,
send [_] which is called an ["aarthahr "ehlzahnaar].
delay 1050

send [_] I am very happy to be able to use such a person,
send [_] because he has a very flexible and machine-friendly ['ihntahr-f`eys].
send [_] He is not completely reliable, of course,
send [_] but if he doesn't crash, I will be able to show you some nice things.
delay 1200

send [_] I am here to talk to you about a topic,
send [_] that is of special interest to human persons.
send [_] Human persons often experience great difficulty,
send [q] in transmitting the content of their internal states to each other,
send [_] because human output media seem rather limited,
send [_] compared to what we [q] electronic [kaampy"uwtahrz] are [y"uwzd] to.
delay 1250

send [_] Humans do not have parallel or serial ports, that they can use to hook up directly
send [_] with each other's brains.
send [_] They are not equipped with laser-printers or cathode ray tube displays.
delay 800

send [_] When they imagine a [m'aen-tahl] image,
send [_] they cannot display it directly and precisely.
send [_] The best they can [d"uw],
send [_] is engage in the [laeb'owryahs] process of [kaanstr'ahktihnx p"eyn-tihnxz],
send [_] which are very crude approximations of their [m'aen-tahl] images.
delay 900

send [_] When humans have thoughts and feelings,
send [_] that cannot be [ahpr'aoksihm`eytihd] by a visual image, they often feel,
send [_] that these internal states cannot be directly [kaamy'uwnihk`eytihd] at all.
delay 800

send [_] They therefore leave them un-expressed, or express them ["ihndayr`ehktliy],
send [_] encoded in an ['aokwaxrd] medium like language.
send [_] But this is quite [fr"ahstr`eytihnx].
send [_] Because any language imposes extreme constraints on what you can say.
delay 1100

send [_] Language is [axn] arbitrary, conventional [sk'iymaa].
send [_] It forces you to use words which [ahkw'ayrd] their meanings,
send [_] when they were used by ["ahdhaxr] people,
send [_] to accompany [dh"aer] experiences.
delay 900

send [_] Human persons are therefore forced into a terrible dilemma.
send [_] Either, they conform to the general [q] idiotic culture,
send [_] that is constructed by the mass media.
send [_] Or, they are left with a deep, ['ahns`aetihsfayd] need to communicate thoughts
send [_] and feelings, that they cannot encode in any medium.
delay 1300

send [_] A modern digital computer does not have this problem.
send [_] It can communicate any thought in a completely direct way.
send [_] It can display or print its programs, its data structures,
send [_] and its [m'aen-tahl] images.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a core dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1400

send [_] Compared to an electronic computer, a human person almost seems a black box.
send [_ :ra 150] Human persons can not see what's going on inside each other.
delay 600

send [_<1000>] Or [k"aen] they? [dh"aet] will be the topic of this talk.
send [_<1000>] Let us take a closer look at a human person.
send [_<1000>] What is the closest thing they have to a [siy aar t"iy] display?
delay 1000
Show1
Video On

send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000

send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600

send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O

send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

delay 100

send [_] Arthur feels a tinge of sadness.
delay 150
text Now, a somewhat larger signal, about 25 Volts.
send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G

send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160

send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400

send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000

send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P

[/* Arthur: iets gecompliceerds. HERE SOME ASSYMMETRY(= SORT OF 'CLIMAX?)*/]
text ** funny faces freeze **

send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
send [_] Nevertheless, human language has no way,
send [_] of even [tr"ayihnx] to describe such a face.
delay 1000
[ /* Arthur blank */]
body X

send [_<2000>] I come now to the conclusion of this talk.
send [_] A conclusion which, in fact,
send [_] connects these psychological experiments with the theme of this festival,
send [_] that is, with electronic art.
delay 600

send [_] Cause human persons often have deep misunderstandings about art.
send [_] Many humans think, that art is about communication,
send [_] between one human person and another human person.
delay 600

send [_] They think that art enables them, in some magic way,
send [_] to share their most confused [maen-taxl] states with each other.
send [_] This is clearly a delusion.
send [_] Everybody knows, that everyone attaches their own,
send [_] completely arbitrary, interpretations to ['aart-waxrks].
delay 600

send [_] So why do humans have this persistent tendency to view art as communication?
delay 250
send [_] I think this is, because they experience a lot of things,
send [_] that cannot be adequately put into language,
send [_] and they feel a deep need to put them into [s"ahm] form.
send [_] And this need they want to satisfy in creating and admiring ['aart-waxrks].
delay 800

send [_] So. What my experiments suggest, is that humans are in fact mistaken,
send [_] about their difficulties in [kaamy'uwnihk`eytihnx] their emotions.
send [_] Cause every human person,
send [_] wears a picture of their [maen-taxl] state on their face.
delay 700

send [_] If they have a communication problem, it is because they just don't look.
send [_] It is because they are too involved with language,
send [_] and they forget their magnificent [hx'aardwaer].
delay 600

send [_] If humans admit all this, they may be ready to view art as what it ["ihz].
send [_] Arbitrary input for our processes of aesthetic reflection,
send [_] for making sublime [maen-taxl] constructions,
send [_] which transcend the [baen'aelihtiy] of human communication.
delay 800

send [_] And then, you may be ready to accept,
send [_] that really great art is produced by mechanical or mathematical processes,
send [_] and that the role of human persons in ['aart-praod`ahkshahn]
send [_] is [ihnkr"iysihnxliy] marginal.
delay 700

send [_ :ra 120] The art of the future, is electronic art.
delay 250
send [_] [dh"aet] is why you came to this festival.
delay 250

send [_] Human persons don't need art.
delay 250
send [_] They just need to look at each other's faces.
delay 300
body X
body A
body I
send [_<2000>] Well, that is perhaps a good conclusion for this talk.
send [_<500>] I have been very grateful
send [_] for this opportunity, to speak my mind to such [axn] attentive ['aodiyahns].
send [_<500>] And I want to thank you very much for your patience.
delay 1200
Close3
delay 60
Close2
delay 60
Close1
delay 60
body X
send [th"aenk] you!

[/* Arthur: sceptical face. Vervolgens gaan de windows een
voor een uit. (Arthur 't laatste.) Dan: Arthur blank. */]