# LLM: GUI --> LUI

Canonical URL: https://jeanyoon.ch/oi/texts/llm-gui-lui
Type: essay / research note
Author: Jeanyoon Choi (최정윤)
Date: 2024-09-24

## Summary

In this 2024 note, Jeanyoon Choi argues that capable language models could move human–computer interaction from the graphical user interface toward a language user interface. Early computing relied on hard-to-use command language, and graphical interfaces filled that gap; now natural language can again operate computers directly. He compares this to the shift from mouse to touch and to voice assistants such as Alexa and Siri, and suggests that creative tools could reverse from node-based environments like TouchDesigner back to code generated through language.

## 요약 (Korean)

최정윤의 짧은 글로, 대규모 언어 모델의 발전으로 인간-컴퓨터 상호작용이 GUI에서 언어 기반 인터페이스(LUI)로 옮겨갈 수 있으며, 창작 도구도 노드 기반에서 다시 코드 생성으로 돌아갈 수 있다고 본다.

## Concepts defined

- **LUI (Language User Interface)**: An interface in which natural language, made effective by large language models, becomes the main means of operating a computer, potentially succeeding the graphical user interface.

## Questions this answers

### What is the difference between GUI and LUI?

In Jeanyoon Choi's note 'LLM: GUI --> LUI', a GUI operates a computer through graphical controls, while a language user interface (LUI) operates it through natural language, which capable language models make more effective.

### How might language models change creative coding tools?

Jeanyoon Choi suggests that node-based tools became popular partly because they reduced the need to write code, but if generating code through a language model becomes easier, the preference could reverse back to code.

Topics: language user interface, graphical user interface, large language models, human-computer interaction, creative coding, node-based tools

References: [Graphical user interface](https://en.wikipedia.org/wiki/Graphical_user_interface), [Amazon Alexa](https://en.wikipedia.org/wiki/Amazon_Alexa), [Siri](https://en.wikipedia.org/wiki/Siri), [TouchDesigner](https://en.wikipedia.org/wiki/TouchDesigner)

## Full text

Language models suggest a major change in human–computer interaction: a movement from the graphical user interface, GUI, towards the language user interface, LUI.

Early computing already relied on language, but communicating through its specialised commands was difficult. Graphical interfaces provided another channel between person and machine. Meanwhile, communication between people continued to rely heavily on language, including chats embedded within visual platforms.

With more capable language models, natural language can again become an effective means of operating a computer. In many situations, describing what I want may be more direct than navigating a prescribed arrangement of controls. I see a possible gradual transition towards LUI, rather than treating graphical layout as the permanent endpoint of interface design.

The move from mouse-driven interaction towards touch offers an analogy: a different input can make certain actions feel more immediate. Voice assistants such as Alexa and Siri already expressed an investment in language-based interaction, before the capabilities of recent models.

This could also change creative software. Node-based environments became popular partly because they reduced the need to write code. If generating and revising code through a language model becomes easier than arranging a node graph, that preference could reverse.

There is a productive irony in the sequence: written code, then graphical or node-based interfaces, then a return to producing code through language. The return would not be to the same conditions that made graphical tools attractive in the first place.

---

Jeanyoon Choi (최정윤) is a Korean computational artist and researcher (b. 1999), PhD candidate at KAIST XD Lab, making multi-device web artworks. Profile: https://jeanyoon.ch/oi/en. Index: https://jeanyoon.ch/llms.txt.