Advancing 3D Vision and Deep Learning for Shape Analysis and Synthesis: Application to Cross-Species Knee Joint Biomechanical Modeling
Abstract: The advance of Artificial Intelligence offers unprecedented possibilities for 3D image-based modeling, shape analysis, and data synthesis, with still underexplored challenges in the medical domain for tasks such as anatomical and biomechanics modeling and translational research. This thesis addresses some of these challenges by introducing methodological innovations that span the full pipeline from 3D image segmentation to shape analysis and generation, with a focus on the knee joint as a clinically relevant use case. We first address cross-modality medical image segmentation by adapting existing deep learning architectures, such as Vision Transformers (ViTs), with a novel conditioning framework. Then, transitioning from volumetric to point cloud–based surface representations, we introduce the Coupled Laplacian operator, which leverages graph Laplacian eigenmaps to match and analyze point clouds while accounting for fine local details. Moreover, building on recent advancements in transformer-based point cloud models, we introduce a technique for landmarks detection, as well as a diffusion architecture for 3D shape generation and novel metrics for evaluating their quality. All methods are extensively validated and compared against state-of-the-art techniques on benchmark datasets. Furthermore, each proposed approach is extended and adapted to the challenging task of cross-species knee modeling, using public human datasets and a limited internal dataset of canine specimens. The results demonstrate the effectiveness of the proposed methods in advancing 3D vision techniques with potential applications in translational research and clinical practice, such as automatic 3D joint modeling, Bone Side Estimation (BSE), and kinematic axes extraction. Finally, we showcase how the proposed pipeline can support Anterior Cruciate Ligament Reconstruction (ACLR) surgery planning by integrating kinematic information, providing a solid groundwork for future clinical studies aimed at improving surgical outcomes.
