jeanyoon.ch@oi ~ % cd /oi

Welcome to jeanyoon.ch/oi

Jeanyoon Choi (b.1999) is a Computational Artist researching in interconnectivity and complexity through Multi-Device Web Artworks.

You can learn more about my:

↑ ↓ select · enter open · or click / tap
artworks projects experiments research about texts contact

jeanyoon.ch@oi /oi % cat conducting-screens-a-possible-approach-to-multi-device-web-artwork

Conducting Screens: A possible approach to Multi-Device Web Artwork

2024-03-20

One mobile phone, multiple screens. A participant moves or handles the phone, and the content distributed across the surrounding screens changes.

Our everyday vocabulary of mobile interaction is dominated by fingers: scrolling, tapping, pinching. What would a more bodily, phenomenological vocabulary look like? Most phones already contain motion sensors and accelerometers. Using them shifts the interaction from the finger to the whole hand. The gesture begins to resemble conducting.

The analogy extends to the outputs. A conductor does not address one instrument in isolation; they shape relationships among instruments distributed through a space. Likewise, the phone could conduct audiovisual responses across laptops, projections or even other phones. What matters is the multiplicity of devices, not a prescribed kind of screen. Cost per device also affects what can be installed.

But a form is not yet a work. Once I have a participant conducting and screens responding, what exactly are they conducting? What context, narrative or sequence of experiences can emerge from this arrangement?

Device motion and acceleration in JavaScript suggest several directions:

A. A three-dimensional scene could rotate with the phone. The viewpoint, the objects, or both could transform.

B. Spatial sound could become louder or quieter with acceleration.

C. A more primitive response could switch a screen on and off, or between black and white. How might that intersect with projection mapping?

D. Two-dimensional graphics could deform, approaching a fluid interface.

E. Screens of text, numbers or maps could be conducted and transformed.

F. Orientation and acceleration could be combined rather than treated as interchangeable inputs. How could the outputs resonate with both?

G. A quieter, more poetic structure might let text wander between screens. Conducting could generate an audiovisual melody from which a story gradually emerges. This returns to the orchestral analogy without reducing it to a literal music controller.

There are possible variations. Why use only a phone? Conducting with a mouse could be equally poetic, with a different, slightly retro history of gestures. Existing web works often confine mouse-driven output to one screen or tab. I want to know what happens when that relationship extends across devices.

And what if there are two participants, each with a phone? Their movements might interfere, reinforce one another or create tension. Sine and cosine functions could help shape those relationships. What happens when there are not two conductors, but a hundred?

↑ ↓ select · enter open · or click / tap
Read original notes back to ../texts ../ back to oi