jeanyoon.ch@oi ~ % cd /oi

Welcome to jeanyoon.ch/oi

Jeanyoon Choi (b.1999) is a Computational Artist researching in interconnectivity and complexity through Multi-Device Web Artworks.

You can learn more about my:

↑ ↓ select · enter open · or click / tap
artworks projects experiments research about texts contact

jeanyoon.ch@oi /oi % cat explicit-commercial-interface-implicit-user-behaviour-tracking

Explicit Commercial Interface & Implicit User Behaviour Tracking

2025-03-06

Semantic and phenomenological interaction need not remain separate. One way to combine them is to place a familiar commercial interface over another, initially implicit layer of bodily input.

Imagine a shopping application. Selecting a product, changing its quantity and checking out provide an explicit semantic interaction: layer A. But those are not the only events occurring. The movement of a finger has position and duration. The phone has orientation and acceleration. With the relevant permission, a camera can register aspects of a face. These provide another layer of possible input.

My working categories are explicit controls—buttons, options, sliders, toggles, checkboxes and lists; implicit sensor-based inputs—gaze, body movement, head pose and similar signals; language through text and voice; and environmental information from external sources. The categories can overlap.

The implicit layer can undergo a dimensional transformation: bodily data → system parameters → changed system behaviour. Finger coordinates or orientation need not map directly to a corresponding visual movement. They can enter another conceptual domain.

The turning point occurs when the participant notices the relationship. What began as incidental behaviour becomes an intentional way to act. They discover layer B and start playing with it. An intuitive bodily movement now carries a meaningful consequence inside the system.

Ω provides one example. An ordinary MBTI questionnaire is interrupted by an accidental shake. A resistance signal spreads through the connected channels, and subsequent movement changes the scene’s orientation. SoTA offers another: a participant begins by selecting neural networks in an infinite-scrolling interface, then discovers that moving the phone also moves the surrounding scene.

These works do not yet exhaust the proposal. I have used orientation largely for scene rotation, and have not extensively explored finger trajectories, facial parameters or their transformation into deeper system dynamics. In Ω, the sudden motion that triggers resistance goes further than a simple spatial correspondence.

What small experiment could extend that? A blink, a gaze, or the geometry of a finger movement could become the input to a different domain. The mapping needs to feel discoverable, surprising and consequential.

This also offers a criticism of the narrowing of commercial interaction. Early smartphone games explored many ways of moving fingers—Temple Run, Angry Birds, rhythm games. Today the same hand often spends hours moving between feeds through scrolling and swiping. The finger remains a rich instrument, even when the interface asks so little of it.

A familiar commercial surface can therefore serve as the entry point. The participant acts as usual, notices an unexpected consequence, and begins changing how they move. The interface becomes something they can physically investigate.

The system may change in three different ways: continuously, through numerical parameters; discretely, through categories such as a personality type or neural-network model; or contextually, through language and generative systems. The third makes it possible to change the subject or organising context, rather than merely a value within a fixed one.

↑ ↓ select · enter open · or click / tap
Read original notes back to ../texts ../ back to oi