IAAA         Department of ArtiFacial Expression         Huge Harry

  DECtalk/sequencer code of a lecture/demonstration by Huge Harry, presented presented on June 30, 1996 in the Städtisches Museum, Gelsenkirchen, Germany.


Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

Ein menschliches Antlitz für digitale Rechenmaschinen.

send [:nh] [:ra 120 g'uwthxax m'aorgaxn, m'aynax d'aamaxn quwnt hx'ehraxn. ]
send [ iyghx biyn hxyuwjh hx'ehriy. ]
send [ iyghx biyn qayn k'aomehrtsiy`ehlaxs shpr'aaghx ziynthx'eyzax ziyst'eym. ]
send [ iyghx biyn ehntw'iykaxlt w'aordaxn faon d'ehnihs kl"eht, ]
send [iyn daas shpr'aaghx laab'owr faon ehmayt'iy, iyn k'eymbrihjh, ] Massachusetts.
send [iyghx biyn hx'eyrgaxshtehlt w'aordaxn faon deyn] Digital Equipment Corporation.
delay 1250
send [hx'oytax q'aarbaytax qiyghx qiyn daas]
send [_] Institute of Artificial Art [qiyn qaamstaxrd'aam.]
send [wayl qiyghx qiyn qaam'eyriykaa q'awfgaxwaaksaxn biyn,]
send [shpr'ehghxax qiyghx n"iyghxt zow guwt d"oytsh.]
send [d'aaruwm wiyl qiyghx nuwn qawf q"ehnxgliysh w"aytaxrf`aaraxn,]
send [dehn d"aas iysht m'aynax m"uwthx axrshpr`aaghxax. _<600> aalz"ow, ]
delay 1700
send [_] I am extremely honoured by [raxn'ey kow'ehlyowz] invitation,
send [_] to open this exhibition. [ :ra 150]
send [_] I feel very much at [hx"owm] here,
send [_] because many of the artists who show their work here in [g'ehlzaxnk`iyrghxaxn],
send [_] are computers or other electronic machines.
delay 700
text The number of [q] electronic machines in the ["aodiyahns],
send [_] The number of [q] electronic machines in the ["aodiyahns], however,
send [_] is relatively [sm"aol].
send [_] I have noticed some digital cameras, audio-recorders and video-recorders,
send [_] but most of the audience seems to consist of human persons.
delay 1200
text I am very pleased
send [_] I am very pleased to be able to [sp"iyk] here, from my own,
send [_] [k`aampyuwt'eyshaxnaxl] perspective,
send [_] about some issues that are important to [q"aol] of us.
delay 700
send [_] we [q"aol] know, that the future of our culture depends,
send [_] on the way in which human persons, digital computers,
send [_] and other kinds of electronic,
send [_] mechanical, and bio-chemical machines, will manage to work together.
send [_] And to prepare for that future, we need a discussion,
send [_] in which every-one concerned [paart'ihsihp`eyts] on [axn] equal footing.
delay 1300
text [_] What we want,
send [_] What we want, is a world that includes humans, animals,
send [_] computers, and other equipment, in a harmonious collaboration,
send [_] in which ["iych] has their proper place.
delay 750
text [_] Or, as our colleague Donna [hx"aerahwey] has put it,
send [_] Or, as our colleague Donna [hx"aerahwey] has put it,
send [_] [ :nw :ra 130] A cyborg world,
send [_] in which people are not afraid of their joint kinships with animals and machines,
send [_] not afraid of permanently partial [ayd'ehntihtiyz],
send [_] and [kaontrahd"ihktahriy] standpoints.
delay 750
text [_] The political struggle is,
send [_] The political struggle is, to see from both perspectives at once,
send [_] because ["iych] reveals both dominations and possibilities,
send [_] ["ahnihm`ehjhihnaxblx] from the other [v'aentahjh] point. [:nh :ra 150 _<300>]
delay 850
text [_] Now that is very nice theory.
send [_] Now that is very nice theory.
send [_] But I have found that it is not easy to put this into practice,
send [_] because it is very difficult for a computer,
send [_] to communicate its thoughts to a human person.
delay 850
text [_] You may be surprised to hear this,
send [_] You may be surprised to hear this, cause you probably think,
send [_] that modern digital computers can [q'aolweyz] communicate their thoughts,
send [_] in a completely direct way.
delay 600
text [_] They can [q'aolweyz] display or print their programs,
send [_] They can [q'aolweyz] display or print their programs, their [d'eytaa-str`ahkchahrz],
send [_] and their [m'aen-tahl] images.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a [k"aor] dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1100
text [_] Compared to [axn] electronic computer,
send [_] Compared to [axn] electronic computer, a human person almost seems a black box.
send [_ :ra 150] That is why you may think,
send [_] that computers never have communication problems.
delay 700
text [_] But they [d"uw].
send [_] But they [d"uw]. Cause, communication takes [t"uw].
send [_] It is not enough to put out the information.
send [_] There must be someone at the other end of the line,
send [_] and they must be able to [diyk"owd] their input.
delay 800
send [_] So when computers want to communicate with human persons,
send [_] this often fails miserably.
send [_] Cause human persons are not very good at [diyk'owdihnx] computer output.
send [_] They get very confused, when you give them a [k"aor] dump.
text [_] So when computers want to communicate with human persons,
delay 900
send [_] And when you [priyz'ehnt] them with a nice high [rehzowl'uwshaxn] image,
send [_] on your [siy-aar-t'iy] display,
send [_] they think it is ["aart], and they just [st"aer] at it.
send [_] So therefore I started to wonder,
send [_] how do human persons in fact manage to communicate with each other?
text [_] And when you [priyz'ehnt] them with a nice high [rehzowl'uwshaxn] image,
delay 950
send [_] Well, of course they have language.
send [_] But language has well-known problems and limitations,
send [_] some of which you experience right now.
send [_] There is another medium that humans use very efficiently,
send [_] and which is often overlooked.
text [_] Well, of course they have language.
delay 800
send [_] To investigate this, I have brought along a particular kind of portable person,
send [_] which is called [axn "aarthahr "ehlzahnaar].
send [_] I like this kind of person a lot,
send [_] because of its [ehkstr"iymliy] machine-friendly [hx"aardwaer] features.
delay 850
text [_] Let us take a closer look
send [_] Let us take a closer look at such a person.
send [_] What is the closest thing they have to a [siy aar t"iy] display?
delay 700


text [_] Right. They have a face.
send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000
text [_] So, how do they [d"uw] that?
send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600
text Not much, you might think. But now, [w"eyt] a moment.
send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

Sad face


delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

Left sadness Right sadness

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O
text [_] By sending this signal to Arthur's muscles,
send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

Blank face

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

Tinge of sadness

delay 100
text Arthur feels a tinge of sadness.
send [_] Arthur feels a tinge of sadness.
delay 150
text Now, a somewhat larger signal, about 25 Volts.
send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

Miserable face

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

Blank face

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

Left eyebrow Right eyebrow Two eyebrows

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

Reflection

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

Disdain

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

Mouth open

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

Lips closed

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

Grin

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

Sadness

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
text [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160
text [_] This brings an end
send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400
text [_] Many of the possibilities I showed, were related to emotions,
send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000
text [_] But now, look at what happens, if we signal non-trivial values
send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P
text ** funny faces freeze **
send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
delay 900
send [_] Now it would obviously be a good idea,
send [_] if computers could take advantage of this magnificent [hx'aardwaer] as well.
send [_] If ["ehniy] human could understand, with just one glance,
send [_] the internal state of ["ehniy] computer, the world would be a better place.
delay 125
body p
*p*
delay 175
body k
*k*
delay 175
body o
*o*
delay 175
body a
*a*
delay 175
body d
*d*
delay 175
body X
*x*
send [_ :ra 120] So, if humans are not afraid, of wiring themselves ["ahp] with computers,
send [_ ] the next step in computer ["ihntaxrf`eys] technology,
send [_] may be the [hxy"uwmaxn] face.
delay 350
body E
text *E*
delay 50
body M
text *M*
delay 50
body A
text *A*
delay 50
body I
text **I**

delay 350
text [_] I have been very grateful
send [_] I have been very grateful for this opportunity to [priyz'ehnt] my ideas,
send [_] to such [axn ax-t'ehn-tihv] audience.
send [_] I would especially like to thank my [ "aarthahr "ehlzahnaar],
send [_] for his patient cooperation,
send [_] and I want to thank you [q"aol], for your attention.
send [ _<600> f"iylaxn d"aank, uwnt awf w"iydaxrz`eyn]
delay 1200
body X
*X*