Authors: Juliette Decugis, Alicia Y. Tsai, Max Emerling, Ashwin Ganesh, Laurent El Ghaoui
Abstract: In this paper, we investigate the extrapolation capabilities of implicit deep
learning models in handling unobserved data, where traditional deep neural
networks may falter. Implicit models, distinguished by their adaptability in
layer depth and incorporation of feedback within their computational graph, are
put to the test across various extrapolation scenarios: out-of-distribution,
geographical, and temporal shifts. Our experiments consistently demonstrate
significant performance advantage with implicit models. Unlike their
non-implicit counterparts, which often rely on meticulous architectural design
for each task, implicit models demonstrate the ability to learn complex model
structures without the need for task-specific design, highlighting their
robustness in handling unseen data.
Source: http://arxiv.org/abs/2407.14430v1
