Authors: Sarah Jabbour, Gregory Kondas, Ella Kazerooni, Michael Sjoding, David Fouhey, Jenna Wiens
Abstract: We propose a permutation-based explanation method for image classifiers.
Current image-model explanations like activation maps are limited to
instance-based explanations in the pixel space, making it difficult to
understand global model behavior. In contrast, permutation based explanations
for tabular data classifiers measure feature importance by comparing model
performance on data before and after permuting a feature. We propose an
explanation method for image-based models that permutes interpretable concepts
across dataset images. Given a dataset of images labeled with specific concepts
like captions, we permute a concept across examples in the text space and then
generate images via a text-conditioned diffusion model. Feature importance is
then reflected by the change in model performance relative to unpermuted data.
When applied to a set of concepts, the method generates a ranking of feature
importance. We show this approach recovers underlying model feature importance
on synthetic and real-world image classification tasks.
Source: http://arxiv.org/abs/2407.14509v1
