Exploring Large Language Model as a Design Partner through Verbal and Non-verbal Conversation in Architectural Design Process

This paper proposes a framework for applying Large Language Models (LLM) as a design partner in architectural design processes instead of a passive question-answering machine. The proposed design framework integrates LLM and Conversation Theory (CT) into a standard parametric design tool for architectural designers. The program establishes an ongoing conversation with the designer through verbal and non-verbal feedback by tracking brain activity and modelling commands. The program can collect conversation data for fine-tuning, thus progressively improving conversation effectiveness. The paper contributes to the knowledge area of architectural design by introducing a novel approach to integrating LLM and CT into the design process, simulated as a proof-of-concept pilot study within a commonly used design software.