# From Semantic Interaction to Phenomenological Interaction Canonical URL: https://jeanyoon.ch/oi/texts/from-semantic-interaction-to-phenomenological-interaction Type: essay / research note Author: Jeanyoon Choi (최정윤) Date: 2025-03-05 ## Summary In this 2025 essay, Jeanyoon Choi considers how system art can ask for meaningful input without overwhelming exhibition visitors, who often hesitate before open text or voice prompts. He argues that a limited set of direct controls, such as buttons, lists and sliders, can shift from deliberate semantic choice into fluid bodily play. In Junk Site, participants move from tapping category buttons to sliding across them, continuously changing generated websites. The essay also proposes 'punctum interaction', where an intense bodily disturbance briefly exposes a precarious system before it restores equilibrium. ## 요약 (Korean) 최정윤의 에세이로, 버튼과 슬라이더 같은 의미 기반 인터랙션이 반복 속에서 신체적·현상학적 놀이로 바뀌는 과정을 『Junk Site』의 예와 함께 다루고, 균형을 깨는 '푼크툼 인터랙션'을 제안한다. ## Concepts defined - **Punctum interaction**: An intense, bodily external shock to a system that appears balanced, exposing its instability before the system restores equilibrium and tries to erase the disturbance. - **Finger-skating**: Gliding a finger continuously over an interface so that its semantic choices, such as category buttons or sliders, become a physical field to move through. ## Questions this answers ### What is the shift from semantic to phenomenological interaction described by Jeanyoon Choi? A participant first presses a button for what its label means, then later presses or slides for the pleasure of the effect itself. Jeanyoon Choi argues that carefully designed direct controls can become playful, fluid and Dadaist in this way. ### Why are text and voice inputs difficult in interactive art exhibitions? According to the essay, visitors often do not know what to say, hesitate, or enter a dummy word, and asking for a whole sentence at the outset can end an interaction. A limited set of initial choices is more productive. ### What is 'punctum interaction'? In Jeanyoon Choi's notes, punctum interaction is an intense bodily disturbance to a precariously balanced system that reveals what the polished simulation normally conceals, after which the system gradually restores its equilibrium. Topics: semantic interaction, phenomenological interaction, system art, interaction design, multitouch gestures, exhibition interaction, dadaism References: Junk Site, Ω, Hammer.js ## Full text System art should be semantic, interactive and modular. The challenge is to make a meaningful interaction approachable enough that a newcomer can begin without a lengthy explanation. There are at least two directions. A nonsemantic bodily action can be transformed into a semantic consequence. Or an initially semantic action can become bodily and phenomenological through use. I am particularly interested in the second transition here. A participant first presses a button because of what its label means. Later, they may press for the pleasure of the effect itself. Their attention shifts from the represented choice to the rhythm and force of acting. Everyday platform use contains a related movement from deliberate selection towards habitual gesture. This matters because an exhibition visitor is not necessarily ready to provide substantial context. An empty text field or voice prompt may produce hesitation or a dummy word. Asking for a whole sentence at the outset can stop an interaction before it develops. A more responsive voice system does not by itself solve the absence of something the visitor wants to say. A limited set of initial choices can be more productive. Buttons, lists, sliders, toggles, numerical inputs and multitouch allow someone to act with relatively little effort. Sensor-based input, language and environmental data can join later or operate alongside them. A direct control is not inherently less immersive than a more technically elaborate sensor. Carefully composed combinations can become playful, fluid and Dadaist. Conversely, a bodily input can become semantic through the transformation of its effects. In Junk Site, the participant could move through broad, intermediate and specific categories. At first they tap each button separately. As the response becomes familiar, their finger stops leaving the surface and begins sliding across the options. Context changes continuously, and websites appear or transform across the surrounding screens. I imagine finger-skating: gliding over an interface whose semantic choices become a physical field. Multitouch could complicate that field. Sliders offer another route, especially when their orientation and arrangement are designed for the direction in which the hand naturally wants to move. A familiar page hierarchy can provide a structure: broad category A, narrower B, specific C. Moving horizontally travels between levels; moving vertically scrolls within one. The ordinary grammar of a social or shopping interface becomes material for a more exaggerated interaction. Possible contexts include shopping for political territories, or repurposing flight and hotel booking interfaces. A map already combines purposeful navigation with the pleasure of panning, pinching and zooming. It is both semantic and tactile. Another possibility is a precarious system that appears balanced until an intense bodily intervention exposes its instability. I called this ‘punctum interaction’ in the notes: an external disturbance reveals something that the polished simulation normally conceals. Over time, the system restores its equilibrium and attempts to erase the disturbance. Ω suggests part of this structure. The remaining questions concern the relationship between equilibrium, external shock and the participant’s bodily action. Hammer.js was one technical reference to investigate for the gesture layer. --- Jeanyoon Choi (최정윤) is a Korean computational artist and researcher (b. 1999), PhD candidate at KAIST XD Lab, making multi-device web artworks. Profile: https://jeanyoon.ch/oi/en. Index: https://jeanyoon.ch/llms.txt.