Causal insights from clinical information in radiology: Enhancing future multimodal AI development.

Purpose: This study investigates the causal mechanisms underlying radiology report generation by analyzing how clinical information and prior imaging examinations contribute to annotation shifts. We systematically estimate why and how biases manifest, providing insights into the data generation process that influences radiology reporting.

Methods: This retrospective study analyzes 172,380 chest X-ray reports from 45,561 distinct patients in the MIMIC-IV CXR database. The study focuses on conditional effects for the diseases pneumonia, pleurisy, heart failure, rib fracture, and COPD. Propensity score matching is employed to balance the treatment and control groups, followed by logistic regression and neural network models to estimate causal effects. Statistical analysis involves calculating risk differences and 95% confidence intervals to determine significance (p ≤ 0.05). Sensitivity analysis is deployed to estimate the robustness of the effect estimates.

Results: The inclusion of clinical questions significantly influences the reporting of key observational findings. For instance, the probability of mentioning cardiomegaly increases by 15% (p ≤ 0.05) when a clinical question is posed conditioned on rib fracture. Similar effects are observed for support devices across multiple diseases. However, the impact of clinical information varies by disease. For instance, in the presence of clinical questions, the mention of pneumonia increases significantly for one disease, while for others there is no significant effect.

Conclusion: This study demonstrates how annotation bias in radiology reports arises from clinical context and prior imaging access. Understanding these causal mechanisms is essential for mitigating biases in dataset curation, ensuring more reliable AI models, and improving the generalizability of multimodal medical imaging systems.