Authors: Jialu Li, Yuanzhen Li, Neal Wadhwa, Yael Pritch, David E. Jacobs, Michael Rubinstein, Mohit Bansal, Nataniel Ruiz
Abstract: We introduce the concept of a generative infinite game, a video game that
transcends the traditional boundaries of finite, hard-coded systems by using
generative models. Inspired by James P. Carse’s distinction between finite and
infinite games, we leverage recent advances in generative AI to create
Unbounded: a game of character life simulation that is fully encapsulated in
generative models. Specifically, Unbounded draws inspiration from sandbox life
simulations and allows you to interact with your autonomous virtual character
in a virtual world by feeding, playing with and guiding it – with open-ended
mechanics generated by an LLM, some of which can be emergent. In order to
develop Unbounded, we propose technical innovations in both the LLM and visual
generation domains. Specifically, we present: (1) a specialized, distilled
large language model (LLM) that dynamically generates game mechanics,
narratives, and character interactions in real-time, and (2) a new dynamic
regional image prompt Adapter (IP-Adapter) for vision models that ensures
consistent yet flexible visual generation of a character across multiple
environments. We evaluate our system through both qualitative and quantitative
analysis, showing significant improvements in character life simulation, user
instruction following, narrative coherence, and visual consistency for both
characters and the environments compared to traditional related approaches.
Source: http://arxiv.org/abs/2410.18975v1
