Artistic Considerations in the use of the EyeCon Motion Tracking System

Mapping

EyeCon allows a variety of movement parameters to be used as "input" to the system:

Position of body parts in space around you (i.e. the "touchline" feature, or triggers in space)
Dynamics (total body movement) within defined fields
Position of the body on the stage area (using overhead camera)
Height of body from the floor (using horizontal camera, and the "top" feature)
Width from left-most to right-most point on the body
Degree of expansion or contraction
Size of the body image (this is not quite the same as expansion-contraction. the former is relative, the later absolute.)
Degree of symmetry in the body (how much left resembles right)
Number of dancers on stage (tracking feature)
Relative closeness of dancers to each other
Individual tracking and differentiation of dancers (based on costume color)

In terms of output, the sky's the limit. A few examples which come to mind:

Audio
musical notes (synthesizer)
on-off
volume control
pitch bending
panning
samples (*.wav files)
on-off
volume control
pitch bending
panning
real time signal processing (using, for example, MAX/msp)
Video Projection
on-off
play-freeze
film play forward-film play in reverse
real time signal processing (using, for example, NATO)
Stage Lighting
Mechanical Devices (mechanical hammers, wind, etc.)

Here then is a sample mapping:

Each choice of mapping (the arrows) may have two directions of compliance /2/. That is, "more movement" may mean "more sound" -- but it can also mean "less sound". This may sound counterintuitive, but there are cases when it feels exactly right. For example, holding a shape -- what we dancers call a suspension -- may require a lot of energy. So although not moving, this moment might anyway be well represented by mapping "less motion" to "more sound".

Must mapping be intuitive? Of course not. But in practice it won't work very well otherwise. What we have found is that straying, even a little, from what "makes sense" on a feeling level, results in an outsider quickly looses the connection. It is harder than you think to follow these mappings, any mappings, even when you've had them explained! Therefore, I would advise to stay as intuitive as possible. "Boring" you say? No way! Believe me, there is still ample room for experimentation and surprises in and around this basic framework.

Indeed, as I look at the diagram above, it occurs to me that if all those things were happening at once, no one would have any idea what was going on! I actually can't think of a single situation when we used more than two mapping at one time. In an entire piece, we rarely employ more than three or four! We have made pieces with upwards of 250 Eyecon elements in them; this is not what I am talking about. The number of mappings -- the kinds of the parameters used -- are far more limited. Not because it would be technically difficult to do so. It wouldn't. The reason is simply that one very quickly comes up against the limits of what the inexperienced viewer can follow.

The arrows in the diagram above should also not be confused with something like "tracks" in a composition. You might think, well, in an orchestra you are not "aware" of what every individual instrument is playing either, and yet the complexity can be marvelous. In the case of an orchestra, however, there is at least mutual support! If you use too many mappings they will likely fight one another, and the result will be that no single one will be evident at all. In all likelihood, no single audience member will know that anything interactive has occurred. But like I say: don't take my word for it. Try it!

Making good mapping choices is really the "art" of using Eyecon. There are no simple rules. Bizarre combinations of parameters have produced stunning results. As I have emphasized already, sufficient clarity is the most common problem. Ways to raise clarity include:

repeat crucial movements a number of times.
if you are a performer, move conscientiously. be aware of the affect you are having on your environment and let this "inform" your movement.
map to multiple outputs. for example, you may wish to link a sound element with a particular movement as well as a visual element (a stage lighting change or video projection element)
think about camera angles. choose one which helps the movements to be accurate and repeatable.
dancers tell choreographers or technicians what you need instead of the other way around.
"link" the media event to a particular gesture. the way a movement is executed may be the key to an audiences perception of the event. try to use memorable, movements -- those with character (even though technically there may be no advantage to doing it that way)
trigger or control the same events from the same stage positions or from the same body posture even though, again, this may be irrelevant to the system you are using.
look for intuitive mappings (higher body level-to-higher pitch, faster-to-louder, busier movement-to-busier sound, heavier movement-to-heavier sound, etc.)
Near the beginning of the piece, or at least at some time during the piece, use the system in a clear and transparent way. this way the system will "explain itself" to the viewer. once having done this, they will become sensitized to the "interactive experience" and will be naturally attuned to "subtler" mappings.

And finally here's one that creates a stir:

either before, or after the piece, explain to your audience how the technology works. There are as many good reasons to do this as there are not to -- but it is an option. To my thinking, it depends on the context (how formal the setting is, for example) as well as the artistic demands of the piece involved. Some pieces don't need it, some don't want it, and others simply love it. Either way, whether you do it or not, I will guarantee you one thing: people will come to you after the show and thank you from the bottom their hearts for your decision, just as the person beside him castigates you for it!

Mappings may of course also be too simple. To lower the clarity, I recommend:

if you are using Touchlines (triggers), try to avoid hand or foot pokes. Find ways to use your dancerly skills (there must be 50 ways to extend a body part).
avoid using physically easy ways to execute an event. Make it harder for yourself than it needs to be.
choose dynamic rather than passive body positions (use plié, relévé, spirals, etc. even when it is of no technical advantage).
use more than one mapping in a piece (though not necessarily simultaneously!)
think about the "direction" /2/ of the interactive relationship. positive, negative; presence, absence. You may wish to reverse it.

EyeCon Help, this file last changed on 14. Feb. 2003