Towards Practical Ultrasound AI Across Real-World Patient Diversity

Abstract

Needle-puncture procedures are often used to treat patients with traumatic and life-threatening injuries. However, properly locating the safest needle insertion location, such as the femoral region, in such high-tempo situations is difficult and can lead to severe complications. The aim of this thesis is to address this difficulty by developing an automatic robot-guided needle insertion system. To close the loop, it requires an imaging modality so the ultrasound modality was used due to its lack of radiation, low costs, and portability.

A major cause for the majority of complications related to needle-puncture procedures is human judgment. In an attempt to minimize the amount of human input for such procedures, which may be clouded by emotions, this thesis aims to fully automate the procedure. As deep neural networks are capable of learning more complex, non-linear functions to approximate the data, and have the potential to generalize well given sufficient data, this thesis leverages the power of deep neural networks and computer vision. Because localization of the proper anatomical landmarks is critical for percutaneous (needle-puncture) procedures, this work focuses on the task of semantic segmentation - which aims to classify each pixel of the images. However, ultrasound images present their own set of challenges: (1) extremely noisy images, which often necessitates trained medical professionals for interpretation, resulting in training data being expensive to collect, and (2) immense variations across ultrasound scanners, imaging settings, body types, and injury scenarios. I aim to address such challenges in the four works included in this thesis.

In the first part of this thesis, I present a deeper introduction of the ultrasound imaging challenges we face as well as a short background of the imaging modality. I then continue with work on studying how semantic segmentation networks can generalize across different populations of ultrasound images using a technique known as transfer learning. The second work then more directly addresses the high-costs of training data for ultrasound images. The proposed method introduces novel temporal data augmentation strategies to increase the size of training data, specifically for dealing with various ultrasound scanning patterns. I evaluate our methods on multiple types of scanning patterns and notice improvements with our simple stochastic augmentation methods. The following work focuses more on addressing the variations across body and injury types when imaging them. This thesis introduces a novel spatial non-uniform data augmentation method which is able to deform various sections of the ultrasound images to mimic long-tailed scenarios.

The final portion of this thesis introduces an initial prototype for a robotic system to automatically insert a needle into the femoral region of a patient. This prototype only represents the first step in achieving our long-term goal; the system introduced aims to determine the safest insertion point for the needle. I believe there is a significant amount more which can be built on top of all these works described and plan to pursue such further in the future.