Authors: Tosin Fadahunsi, Giordano d’Aloisio, Antinisca Di Marco, Federica Sarro
Abstract: Generative models are nowadays widely used to generate graphical content used
for multiple purposes, e.g. web, art, advertisement. However, it has been shown
that the images generated by these models could reinforce societal biases
already existing in specific contexts. In this paper, we focus on understanding
if this is the case when one generates images related to various software
engineering tasks. In fact, the Software Engineering (SE) community is not
immune from gender and ethnicity disparities, which could be amplified by the
use of these models. Hence, if used without consciousness, artificially
generated images could reinforce these biases in the SE domain. Specifically,
we perform an extensive empirical evaluation of the gender and ethnicity bias
exposed by three versions of the Stable Diffusion (SD) model (a very popular
open-source text-to-image model) – SD 2, SD XL, and SD 3 – towards SE tasks. We
obtain 6,720 images by feeding each model with two sets of prompts describing
different software-related tasks: one set includes the Software Engineer
keyword, and one set does not include any specification of the person
performing the task. Next, we evaluate the gender and ethnicity disparities in
the generated images. Results show how all models are significantly biased
towards male figures when representing software engineers. On the contrary,
while SD 2 and SD XL are strongly biased towards White figures, SD 3 is
slightly more biased towards Asian figures. Nevertheless, all models
significantly under-represent Black and Arab figures, regardless of the prompt
style used. The results of our analysis highlight severe concerns about
adopting those models to generate content for SE tasks and open the field for
future research on bias mitigation in this context.
Source: http://arxiv.org/abs/2501.09014v1
