# Explicit Commercial Interface & Implicit User Behaviour Tracking Canonical URL: https://jeanyoon.ch/oi/texts/explicit-commercial-interface-implicit-user-behaviour-tracking Type: essay / research note Author: Jeanyoon Choi (최정윤) Date: 2025-03-06 ## Summary This 2025 essay by Jeanyoon Choi proposes combining semantic and phenomenological interaction by placing a familiar commercial interface, such as a shopping app, over a hidden layer of bodily input: finger trajectories, phone orientation and acceleration, or facial data. Through dimensional transformation, these signals change system parameters, and the participant gradually discovers and plays with the hidden layer. The essay draws on Ω and SoTA, critiques the narrowing of smartphone interaction to scrolling, and distinguishes parametric, categorical and contextual ways of changing a system. ## 요약 (Korean) 최정윤의 에세이로, 익숙한 쇼핑 앱 같은 상업 인터페이스 아래에 손가락 움직임, 휴대폰 기울기, 얼굴 데이터 같은 암묵적 신체 입력 층을 두어 시스템 파라미터로 변환하는 인터랙션을 제안하며 『Ω』와 『SoTA』를 예로 든다. ## Concepts defined - **Dimensional transformation**: Mapping bodily data, such as finger coordinates or phone orientation, into system parameters in a different conceptual domain, so that it changes system behaviour rather than producing a direct visual correspondence. - **Contextual adjustment**: A way of changing a system through language and generative AI, which alters the subject or organising context rather than a value within a fixed one; contrasted with parametric and categorical adjustment. ## Questions this answers ### What is Jeanyoon Choi's essay 'Explicit Commercial Interface & Implicit User Behaviour Tracking' about? It proposes an interactive artwork in which a familiar commercial interface provides an explicit layer of interaction while implicit bodily data, such as finger movement or phone motion, is transformed into system parameters. Participants discover this second layer and begin to play with it deliberately. ### How did Jeanyoon Choi use phone motion in the artworks Ω and SoTA? In Ω, an accidental shake during an MBTI-style questionnaire triggers a resistance signal across the connected channels and later rotates the scene. In SoTA, participants scroll through neural networks and discover that moving the phone also moves the surrounding 3D scene. ### What are the three ways an interaction can change a system, according to Jeanyoon Choi? The essay distinguishes parametric change through continuous numerical values, categorical change through discrete choices such as a personality type or neural-network model, and contextual change through language and generative systems, which can alter the organising context itself. Topics: semantic interaction, phenomenological interaction, implicit interaction, dimensional transformation, sensor-based input, commercial interface, multi-device web artwork References: Ω, SoTA, [Myers–Briggs Type Indicator](https://en.wikipedia.org/wiki/Myers%E2%80%93Briggs_Type_Indicator), [Temple Run](https://en.wikipedia.org/wiki/Temple_Run), [Angry Birds](https://en.wikipedia.org/wiki/Angry_Birds), Hammer.js ## Full text Semantic and phenomenological interaction need not remain separate. One way to combine them is to place a familiar commercial interface over another, initially implicit layer of bodily input. Imagine a shopping application. Selecting a product, changing its quantity and checking out provide an explicit semantic interaction: layer A. But those are not the only events occurring. The movement of a finger has position and duration. The phone has orientation and acceleration. With the relevant permission, a camera can register aspects of a face. These provide another layer of possible input. My working categories are explicit controls—buttons, options, sliders, toggles, checkboxes and lists; implicit sensor-based inputs—gaze, body movement, head pose and similar signals; language through text and voice; and environmental information from external sources. The categories can overlap. The implicit layer can undergo a dimensional transformation: bodily data → system parameters → changed system behaviour. Finger coordinates or orientation need not map directly to a corresponding visual movement. They can enter another conceptual domain. The turning point occurs when the participant notices the relationship. What began as incidental behaviour becomes an intentional way to act. They discover layer B and start playing with it. An intuitive bodily movement now carries a meaningful consequence inside the system. Ω provides one example. An ordinary MBTI questionnaire is interrupted by an accidental shake. A resistance signal spreads through the connected channels, and subsequent movement changes the scene’s orientation. SoTA offers another: a participant begins by selecting neural networks in an infinite-scrolling interface, then discovers that moving the phone also moves the surrounding scene. These works do not yet exhaust the proposal. I have used orientation largely for scene rotation, and have not extensively explored finger trajectories, facial parameters or their transformation into deeper system dynamics. In Ω, the sudden motion that triggers resistance goes further than a simple spatial correspondence. What small experiment could extend that? A blink, a gaze, or the geometry of a finger movement could become the input to a different domain. The mapping needs to feel discoverable, surprising and consequential. This also offers a criticism of the narrowing of commercial interaction. Early smartphone games explored many ways of moving fingers—Temple Run, Angry Birds, rhythm games. Today the same hand often spends hours moving between feeds through scrolling and swiping. The finger remains a rich instrument, even when the interface asks so little of it. A familiar commercial surface can therefore serve as the entry point. The participant acts as usual, notices an unexpected consequence, and begins changing how they move. The interface becomes something they can physically investigate. The system may change in three different ways: continuously, through numerical parameters; discretely, through categories such as a personality type or neural-network model; or contextually, through language and generative systems. The third makes it possible to change the subject or organising context, rather than merely a value within a fixed one. --- Jeanyoon Choi (최정윤) is a Korean computational artist and researcher (b. 1999), PhD candidate at KAIST XD Lab, making multi-device web artworks. Profile: https://jeanyoon.ch/oi/en. Index: https://jeanyoon.ch/llms.txt.