A simulated annealing-based Bayesian network structure optimization framework for late morbidity prediction with a large prospective dataset.

Background: Bayesian networks are seeing increased usage in healthcare, particularly for modeling complex treatment decisions under uncertainty. Bayesian networks offer significant advantages over classical machine learning and deep learning techniques due to their interpretability, with the network visualized through a directed acyclic graph outlining conditional relationships. Prior clinical knowledge can also be incorporated into these networks to enhance their clarity and facilitate integration into clinical workflows. However, out-of-box optimization techniques may produce networks that are not logically coherent or reflective of clinical understanding and may focus solely on optimizing information-based metrics without consideration for performance metrics crucial for developing predictive models. In late morbidity modeling, where the risk factors surrounding an outcome may be complex, intercorrelated, and not yet fully identified, it is important to have a customizable optimization approach to automatically produce logical, interpretable Bayesian networks that outline these complex outcomes.

Purpose: Develop a simulated annealing-based framework for developing Bayesian network structures for late morbidity prediction in cervical cancer patients, addressing limitations of traditional optimization techniques and prioritizing interpretability.

Methods: This study utilizes the multi-center EMBRACE I cervical cancer dataset (n = 1153) to develop Bayesian network structures for late moderate-to-severe (grade ≥2) cystitis (CTCAEv.3) prediction. The dataset was split into training/validation data (80%) and holdout test data (20%). A process of 10 × 5-fold cross-validation was integrated into the optimization framework. A simulated annealing-based optimization method was developed incorporating information-theoretic measures, predictive performance measures, and complexity measures. The different network structures developed by this framework were compared in terms of complexity, interpretability, and predictive performance to optimization methods available out-of-box from the PyAgrum package for Python (Greedy Hill Climbing, Tree-Augmented Naïve Bayes, and Chow-Liu Optimization). Bayesian networks were also compared to conventional machine learning classifiers in terms of feature importance and predictive performance. Differences in model predictions arising from structure differences were assessed with Cochran's Q-test (p < 0.05).

Results: The simulated annealing framework demonstrated the ability to produce Bayesian network structures with comparable or superior predictive performance compared to out-of-box models. A statistically significant performance difference was identified between the simulated annealing and out-of-box methods with Cochran's Q-test (p = 0.03). The simulated annealing approach equalled or outperformed out-of-box models on a bootstrapped holdout test set, with a balanced accuracy of 64.1%, an F1 macro score of 55.9%, and an ROC-AUC of 0.66. Simulated annealing models also featured fewer arcs and nodes, with this simplification resulting in networks that were easier to interpret without compromising on predictive performance, highlighting the effectiveness of simulated annealing in creating highly interpretable models for clinical use.

Conclusion: The proposed simulated annealing-based framework represents a novel method for automatically generating Bayesian network structures for cervical cancer late morbidity modeling. Compared to out-of-box optimization techniques, the simulated annealing Bayesian networks provide comparable or superior predictive performance while constructing a more simple, interpretable network useful for clinical implementation.