Authors: Sam Earle, Samyak Parajuli, Andrzej Banburski-Fahey
Abstract: Coding assistants are increasingly leveraged in game design, both generating
code and making high-level plans. To what degree can these tools align with
developer workflows, and what new modes of human-computer interaction can
emerge from their use? We present DreamGarden, an AI system capable of
assisting with the development of diverse game environments in Unreal Engine.
At the core of our method is an LLM-driven planner, capable of breaking down a
single, high-level prompt — a dream, memory, or imagined scenario provided by
a human user — into a hierarchical action plan, which is then distributed
across specialized submodules facilitating concrete implementation. This system
is presented to the user as a garden of plans and actions, both growing
independently and responding to user intervention via seed prompts, pruning,
and feedback. Through a user study, we explore design implications of this
system, charting courses for future work in semi-autonomous assistants and
open-ended simulation design.
Source: http://arxiv.org/abs/2410.01791v1
