Authors: Ilya Orson Sandoval, Isaac Symes Thompson, Vasilios Mavroudis, Chris Hicks
Abstract: As cyber threats grow increasingly sophisticated, reinforcement learning is
emerging as a promising technique to create intelligent, self-improving
defensive systems. However, most existing autonomous defensive agents have
overlooked the inherent graph structure of computer networks subject to cyber
attacks, potentially missing critical information. To address this gap, we
developed a custom version of the Cyber Operations Research Gym (CybORG)
environment that encodes the observable network state as a directed graph,
utilizing realistic and interpretable low-level features. %, like number of
open ports and unexpected detected connections. We leverage a Graph Attention
Network (GAT) architecture to process node, edge, and global features, and
modify its output to be compatible with policy gradient methods in
reinforcement learning. GAT policies offer several advantages over standard
approaches based on simplistic flattened state observations. They can handle
the changes in network topology that occur at runtime when dynamic connections
between hosts appear. Policies can be deployed to networks that differ in size
to the ones seen during training, enabling a degree of generalisation
inaccessible with alternative approaches. Furthermore, the graph neural network
policies outputs are explainable in terms of tangible network properties,
providing enhanced interpretability of defensive actions. We verify that our
low-level graph observations are meaningful enough to train GAT defensive
policies that are able to adapt to changing topologies. We evaluate how our
trained policies perform when deployed on networks of varying sizes with the
same subnetwork structure, comparing them against policies specifically trained
for each network configuration. Our study contributes to the development of
robust cyber defence systems that can better adapt to real-world network
security challenges.
Source: http://arxiv.org/abs/2501.14700v1
