Authors: Mustafa O. Karabag, Ufuk Topcu
Abstract: Large language model-based (LLM-based) agents have become common in settings
that include non-cooperative parties. In such settings, agents’ decision-making
needs to conceal information from their adversaries, reveal information to
their cooperators, and infer information to identify the other agents’
characteristics. To investigate whether LLMs have these information control and
decision-making capabilities, we make LLM agents play the language-based
hidden-identity game, The Chameleon. In the game, a group of non-chameleon
agents who do not know each other aim to identify the chameleon agent without
revealing a secret. The game requires the aforementioned information control
capabilities both as a chameleon and a non-chameleon. The empirical results
show that while non-chameleon LLM agents identify the chameleon, they fail to
conceal the secret from the chameleon, and their winning probability is far
from the levels of even trivial strategies. To formally explain this behavior,
we give a theoretical analysis for a spectrum of strategies, from concealing to
revealing, and provide bounds on the non-chameleons’ winning probability. Based
on the empirical results and theoretical analysis of different strategies, we
deduce that LLM-based non-chameleon agents reveal excessive information to
agents of unknown identities. Our results point to a weakness of contemporary
LLMs, including GPT-4, GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet, in strategic
interactions.
Source: http://arxiv.org/abs/2501.19398v1
