Authors: Shayan Kiyani, George Pappas, Aaron Roth, Hamed Hassani
Abstract: A fundamental question in data-driven decision making is how to quantify the
uncertainty of predictions in ways that can usefully inform downstream action.
This interface between prediction uncertainty and decision-making is especially
important in risk-sensitive domains, such as medicine. In this paper, we
develop decision-theoretic foundations that connect uncertainty quantification
using prediction sets with risk-averse decision-making. Specifically, we answer
three fundamental questions: (1) What is the correct notion of uncertainty
quantification for risk-averse decision makers? We prove that prediction sets
are optimal for decision makers who wish to optimize their value at risk. (2)
What is the optimal policy that a risk averse decision maker should use to map
prediction sets to actions? We show that a simple max-min decision policy is
optimal for risk-averse decision makers. Finally, (3) How can we derive
prediction sets that are optimal for such decision makers? We provide an exact
characterization in the population regime and a distribution free finite-sample
construction. Answering these questions naturally leads to an algorithm,
Risk-Averse Calibration (RAC), which follows a provably optimal design for
deriving action policies from predictions. RAC is designed to be both
practical-capable of leveraging the quality of predictions in a black-box
manner to enhance downstream utility-and safe-adhering to a user-defined risk
threshold and optimizing the corresponding risk quantile of the user’s
downstream utility. Finally, we experimentally demonstrate the significant
advantages of RAC in applications such as medical diagnosis and recommendation
systems. Specifically, we show that RAC achieves a substantially improved
trade-off between safety and utility, offering higher utility compared to
existing methods while maintaining the safety guarantee.
Source: http://arxiv.org/abs/2502.02561v1
