jeanyoon.ch@oi ~ % cd /oi

Welcome to jeanyoon.ch/oi

Jeanyoon Choi (b.1999) is a Computational Artist researching in interconnectivity and complexity through Multi-Device Web Artworks.

You can learn more about my:

↑ ↓ select · enter open · or click / tap
artworks projects experiments research about texts contact

jeanyoon.ch@oi /oi % cat two-simple-experiment-ideas-using-multi-modality

Two simple experiment ideas using multi-modality

2024-04-07

Two experiments with multimodal input. The available vocabulary includes hand, body and eye tracking; facial expression and voice; scrolling, pinching, tapping and conducting with a phone’s accelerometer; and the keyboard and mouse.

1. An expandable list controlled by gaze.

Imagine a vertical list of ten or more items on a phone. Looking at an item expands it to reveal further detail. Looking elsewhere collapses it and expands the newly attended item. This could be a UI experiment, but it could also extend into a spatial artwork. What would a gaze-responsive grid look like? What if fifteen separate screens expanded or changed according to the participant’s attention?

2. Yawning and screaming through mouth tracking.

The distance between the lips provides a simple measure of whether a mouth is open. I want to use that movement to interrupt a conversation in Das Gerede Part II.

Three screens—R, G and B—carry on an ordinary, coherent, painfully dull conversation. When the participant opens their mouth, a yawn unexpectedly enters the exchange. The conversation resumes, until another opening triggers a scream.

Gradually, semantic conversation gives way to animal-like sounds, yawns and screams: a nonsemantic symphony distributed across three screens. The bodily expression of boredom becomes the means of dismantling the boring conversation.

↑ ↓ select · enter open · or click / tap
Read original notes back to ../texts ../ back to oi