TIGP (BIO)—From Kidney Ultrasound Segmentation to Renal Failure Risk Prediction: Deep Learning and Multimodal Approaches

Abstract

Chronic kidney disease (CKD) is a major healthcare burden, and identifying patients at risk of progressive renal function decline remains an important clinical challenge. Kidney ultrasound is routinely available, non-invasive, and relatively inexpensive; however, extracting quantitative prognostic information from ultrasound images is challenging because of variations in image quality, acquisition conditions, disease stage, and imaging equipment.

In this seminar, I will present our development of a deep learning framework for extracting clinically relevant information from kidney ultrasound images and predicting the risk of renal function deterioration.

We first developed a semi-supervised semantic segmentation approach that leverages both labeled and unlabeled data. Patch-level guided contrastive learning was incorporated to improve the representation of kidney structures, particularly when expert annotations were limited. The resulting image encoder was subsequently transferred to a survival analysis framework for predicting renal function decline. 

We further investigated how ultrasound imaging can be integrated with clinical and laboratory measurements through multimodal deep learning. Combining image and clinical representations improved survival-risk prediction compared with models using either modality alone.

Finally, we explored a clinically guided image representation strategy, in which an image encoder learns latent features associated with clinical measurements. A dual-encoder fusion architecture was then developed to improve risk prediction using ultrasound images alone. 

Through this case study, the seminar will illustrate several general challenges in biomedical AI, including limited expert annotations, representation learning, transfer learning, survival analysis, multimodal data integration, and the translation of routinely collected medical images into clinically meaningful risk estimates.

► 線上視訊請點選【連結】加入。

附件下載