Authors: Mourad Heddaya, Qingcheng Zeng, Chenhao Tan, Rob Voigt, Alexander Zentefis
Abstract: We present a novel approach to classify causal micro-narratives from text.
These narratives are sentence-level explanations of the cause(s) and/or
effect(s) of a target subject. The approach requires only a subject-specific
ontology of causes and effects, and we demonstrate it with an application to
inflation narratives. Using a human-annotated dataset spanning historical and
contemporary US news articles for training, we evaluate several large language
models (LLMs) on this multi-label classification task. The best-performing
model–a fine-tuned Llama 3.1 8B–achieves F1 scores of 0.87 on narrative
detection and 0.71 on narrative classification. Comprehensive error analysis
reveals challenges arising from linguistic ambiguity and highlights how model
errors often mirror human annotator disagreements. This research establishes a
framework for extracting causal micro-narratives from real-world data, with
wide-ranging applications to social science research.
Source: http://arxiv.org/abs/2410.05252v1
