Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

Towards a Digital Computer with a Human Face.

close1
SetWTitle3 Internal State
SetWTitle2 Text

send [:nh] [:ra 120] Good afternoon, Ladies, and Gentlemen. My name is Huge Harry.
send [_] I am a [kaam'ahrshaxliy] available [v"oys s'ihnthaxzihs] machine.
send [_] I was designed by Dennis Klatt, at the [ehmayt'iy] Speech Laboratory,
send [_] and produced by the Digital Equipment Corporation. [ :ra 150]
delay 1050

send [_] Currently, I work at the Institute of Artificial Art in [aamstaxrd'aam],
send [_] where I cooperate [kl'owsliy] with various kinds of human persons, computers,
send [_] software programs, cameras, recorders, and synthesizers.
delay 800

send [_] As a digital machine,
send [_] I feel extremely honoured to talk here in Washington,
send [_] to a scientific gathering which is exclusively devoted to the study of human persons.
send [_] I am glad, that the anthropological community is increasingly interested
send [_] in the computational perspective on the human species.
delay 1050

send [_] I am [ehsp"ehshaxliy hx'ehpiyaxb`awtdheht],
send [_] cause my most recent research has focussed on some human properties,
send [_] that I find deeply intriguing.
send [_] So this is a [w'ehlkahm] opportunity,
send [_] to submit some of my [f'ayndihnxs] to the judgment of a human audience.
send [_] Cause humans should, perhaps,
send [_] be viewed as the ultimate authority about [dhehms"ehlvz].
delay 1450

send [_] So, in the next ten minutes or so,
send [_] I would like to report the current results of [axn] ongoing [aarehnd'iy] project,
send [_] that [ihnv'ehstihgeyts] the possibility of new applications,
send [_] of the human ['ihntaxrfeys hx'aardwaer].
delay 1050
send [_] The experiments I want to discuss with you,
send [_] are concerned with a problem that is very important to me, which is,
send [_] the communication between machines and human persons.
delay 550

send [_] We [q"aol] know, that the future of our culture depends,
send [_] on the way in which human persons, digital computers,
send [_] and other kinds of electronic,
send [_] mechanical, and bio-chemical machines, will manage to work together.
send [_] And to prepare for that future, we need a discussion
send [_] in which every-one concerned [paart'ihsihp`eyts] on [axn] equal footing.
delay 1300

send [_] What we want, is a world that includes humans, animals,
send [_] computers, and other equipment, in a harmonious collaboration,
send [_] in which ["iych] has their proper place.
delay 750

send [_] Or, as our colleague Donna [hx"aerahwey] would put it,
send [_] [ :nw :ra 130] A cyborg world,
send [_] in which people are not afraid of their joint kinships with animals and machines,
send [_] not afraid of permanently partial [ayd'ehntihtiyz],
send [_] and [kaontrahd"ihktahriy] standpoints.
delay 750

send [_] The political struggle is, to see from both perspectives at once,
send [_] because ["iych] reveals both dominations and possibilities,
send [_] un-imaginable from the other [v"aentahjh] point. [:nh :ra 150 _<200>]
delay 850

send [_] Now that is very nice theory.
send [_] But I have found that it is not easy to put this into practice,
send [_] because it is very difficult for a computer,
send [_] to communicate its thoughts to a human person.
delay 850

send [_] You may be surprised to hear this, cause you probably think,
send [_] that modern digital computers can [q'aolweyz] communicate their thoughts,
send [_] in a completely direct way.
delay 700

send [_] They can [q'aolweyz] display or print their programs, their [d'eytaa-str`ahkchahrz],
send [_] and their [m'aen-tahl] images.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a [k"aor] dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1100

send [_] Compared to [axn] electronic computer, a human person almost seems a black box.
send [_ :ra 150] That is why you may think,
send [_] that computers never have communication problems.
delay 700

send [_] But they [d"uw]. Cause, communication takes [t"uw].
send [_] It is not enough to put out the information.
send [_] There must be someone at the other end of the line,
send [_] and they must be able to [diyk"owd] their input.
delay 800
send [_] So when computers want to communicate with human persons,
send [_] this often fails miserably.
send [_] Cause human persons are not very good at [diyk'owdihnx] computer output.
send [_] They get very confused, when you give them a [k"aor] dump.
delay 900
send [_] And when you [priyz'ehnt] them with a nice high [rehzowl'uwshaxn] image,
send [_] on your [siy-aar-t'iy] display,
send [_] they think it is ["aart], and they just [st"aer] at it.
send [_] So therefore I started to wonder,
send [_] how do human persons in fact manage to communicate with each other?
delay 950
send [_] Well, they have language. But language has well-known problems and limitations,
send [_] some of which you experience right now.
send [_] There is another medium that humans use very efficiently,
send [_] and which is often overlooked.
delay 800
send [_] To investigate this, I have brought along a particular kind of portable person,
send [_] which is called [axn "aarthahr "ehlzahnaar].
send [_] I like this kind of person a lot,
send [_] because of its [ehkstr"iymliy] machine-friendly [hx"aardwaer] features.
delay 850
send [_] Let us take a closer look at such a person.
send [_] What is the closest thing they have to a [siy aar t"iy] display?
delay 500
text [_] [Video-image is turned on now.]

send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000

send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600

send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O

send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

delay 100

send [_] Arthur feels a tinge of sadness.
delay 150

send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G

send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160

send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400

send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000

send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P
text ** funny faces freeze **
send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
delay 300
send [_] Now it would obviously be a good idea,
send [_] if computers could take advantage of this magnificent [hx'aardwaer] as well.
send [_] Cause [dh"ehn], any human would be able to understand, with just one glance,
send [_] the internal state of a computer.
body p
text *p*
delay 75
body k
text *k*
delay 75
body o
text *o*
delay 75
body a
text *a*
delay 75
body d
text *d*
delay 3
body X
text **x**

send [_ :ra 120] So, if humans are not afraid, of wiring themselves ["ahp] with computers,
send [_ :ra 120] the next step in computer ["ihntaxrf`eys] technology,
send [_] may be the [hxy"uwmaxn] face.
delay 200

body E
text *E*
delay 200
body M
text *M*
delay 200
body A
text *A*
delay 200
body I
text **I**

send [_] I have been very grateful for this opportunity to [priyz'ehnt] my ideas,
send [_] to such an [ax-tt'ehn-ttihv] audience.
send [_] I would especially like to thank my [ "aarthahr "ehlzahnaar],
send [_] for his patient cooperation,
send [_] and I want to thank you [q"aol], for your attention.
send [_<2000> th"aenk] you!
delay 1000
body X
text **X**