# How to well-design semantical interaction - which users actually care??

Canonical URL: https://jeanyoon.ch/oi/texts/how-to-well-design-semantical-interaction-which-users-actually-care
Type: essay / research note
Author: Jeanyoon Choi (최정윤)
Date: 2025-02-06

## Summary

This 2025 note by Jeanyoon Choi addresses why semantic interaction, such as typing or speaking within a context, is harder to sustain in exhibitions than bodily gestures: visitors hesitate or enter something arbitrary. It proposes treating hesitation and passivity as part of the exchange, reconsidering onboarding through a sudden estranging event, a gradual invitation or strong narrative context, and amplifying small inputs into rich semantic responses. In the scenario considered, a visitor lies down to rest, a language model addresses them and leads the conversation, and their uncertain murmur becomes part of the work.

## 요약 (Korean)

최정윤의 글로, 전시 관객이 텍스트나 음성 입력을 주저하는 문제를 다루며 수동성 자체를 인터랙션에 포함하고 작은 음성 입력을 풍부한 응답으로 증폭하는 방식을 제안한다.

## Questions this answers

### How can interactive art get meaningful input from hesitant exhibition visitors?

Jeanyoon Choi suggests making hesitation and passivity part of the interaction, designing the introduction carefully, and transforming small, low-information actions such as a murmur into unexpectedly rich responses.

### How does Jeanyoon Choi propose using a language model in a voice-based artwork?

Instead of the user supplying context and leading the conversation, the model leads, treating a minimal or passive response as enough to continue. The scenario begins with a visitor lying on a bed, unaware that an interaction will start.

Topics: semantic interaction, voice user interface, onboarding, exhibition interaction, large language models, dadaism

References: [Rafael Lozano-Hemmer](https://en.wikipedia.org/wiki/Rafael_Lozano-Hemmer), [Verfremdungseffekt](https://en.wikipedia.org/wiki/Distancing_effect)

## Full text

How can a semantic interaction ask something meaningful of a participant without making them reluctant to engage?

Many exhibition interactions depend on bodily movement or a simple gesture. Asking someone to compose text or speak within a specific context is harder. They may hesitate, refuse, or enter something arbitrary. The work cannot assume that every visitor arrives ready to contribute an articulate response.

One approach is to make that hesitation part of the interaction. Inactivity, a face or a small involuntary response can carry the exchange forward. Another is to reconsider the introduction: an immediate estranging event, a gradual invitation that lets someone settle in, or a strong narrative context established before their first action.

A third approach transforms the scale of the input. A small, low-information action can produce an unexpectedly rich semantic response. Input A becomes output AAAAAAAA. That disproportion can be funny, Dadaist or disorienting. A murmur, for instance, could cause a voice interface to respond with extravagant conviction.

In a conventional language-model interface, the user supplies the context and leads the conversation. Here the model could lead, treating a minimal or passive response as enough to continue. I also noted the way a queue may raise a visitor’s expectations and investment, although waiting is not itself the artistic experience.

The particular scenario I am considering begins with rest. Someone lies down on a bed without expecting an interaction. The system addresses them, and their uncertain response becomes part of the exchange. The introduction, the amplification of a small vocal input and the inclusion of passivity all belong to the same design problem.

---

Jeanyoon Choi (최정윤) is a Korean computational artist and researcher (b. 1999), PhD candidate at KAIST XD Lab, making multi-device web artworks. Profile: https://jeanyoon.ch/oi/en. Index: https://jeanyoon.ch/llms.txt.