SMTLNet: Domain Prior-Inspired Tooth Segmentation Based on Self-Supervised Manifold Transfer Learning.

Accurate identification and delineation of teeth in cone-beam computed tomography (CBCT) images are crucial in the advancement of digital dentistry technology. Teeth exhibit high interclass similarity and often have fuzzy boundaries. In addition, it is difficult to obtain teeth samples due to the time-consuming annotation process. However, existing methods typically fail to incorporate this domain-specific prior information under limited labeled samples, which limits the improvement of segmentation performance. Based on the intrinsic characteristics of the tooth CBCT images, a self-supervised manifold transfer learning network (SMTLNet) is proposed to improve segmentation accuracy. Initially, an object-oriented self-supervised pretraining approach is designed to fully explore valuable image representations from unannotated images, and this helps reduce dependence on labeled samples. Furthermore, a manifold optimization strategy is employed to regularize the segmentation model to separate interclass samples while compacting intraclass neighbors. Finally, to address the issue of blurred tooth boundaries, a multiscale boundary constraint module is developed to extract multiscale boundary-aware features, and more discriminative tooth descriptions can be acquired in this way. The proposed SMTLNet method is evaluated on clinical datasets containing diverse challenging cases (e.g., impacted wisdom teeth, crowded dentition), and it achieves state-of-the-art performance with dice similarity coefficients (DSCs) of 91.8%/89.08% and Jaccard similarities (JSs) of 86.71%/82.87% under full (100%) and limited (20%) training data regimes, respectively. The method maintains anatomical precision with Hausdorff distances (HDs) of 1.41 mm (high-resource) and 2.35 mm (low-resource), demonstrating strong clinical applicability in digital dentistry workflows.