In the case of high-tempo, traumatic scenarios on the battlefield, real-time ultrasound (US) imaging serves as an enabler for countless possible robotic interventions. Having the ability to automatically segment anatomical landmarks in the body, such as arteries, veins, ligaments, and veins, for percutaneous procedures remains to be a difficult task, even in “well-controlled” settings such as hospitals where physicians and other caregivers, with considerable training, and therefore such procedures may experience complications. Performing percutaneous procedures outside of the hospital, in the field, presents additional challenges, when considering all the situations in which medical care is being delivered. These situations can be in the aftermath of a natural disaster or in a combat or hostage situation. Our goal is to create AI tools that assist the medical caregiver, either in centers of medical excellence, in rural hospitals, or in the field, deliver medical care in the form of percutaneous interventions.

We have chosen ultrasound imaging to assist, really close the loop, for needle insertion in the field because ultrasound is low-cost, portable, and radiation-free. The problem with ultrasound is that it is often difficult to interpret, therefore making percutaneous interventions even more difficult, both for human caregivers and the medical AI that supports them. Therefore, the medical AI tools that we create using ultrasound to close the loop therefore must consider the countless domains across body types, potential traumatic injury scenarios, and imaging artifacts.

For medical AI, we harness the power of deep neural networks because they can learn a more complex, non-linear function and, therefore, given sufficient training data, deep neural networks more easily discriminate among different classes. In other words, they are better function approximators. The drawback of deep neural networks is that they are data hungry, and therefore require significant amounts of diverse datasets. This can be time consuming and expensive, especially when you need a trained profession, like a physician, to label data.

This work proposes a method for enhancing deep learning models’ capabilities by generating synthetic, or augmented, data which is transformed in a manner designed to account for various body types, injury scenarios, and imaging features. Our goal is to research a learning-based data augmentation method which can adaptively generate augmented images for learning invariances across various anatomical shapes and imaging artifacts. We further explore additional perspectives with which to apply similar ideas, such as transfer learning.

Relevant Publications:

Wanwen Chen, Kathan Nilesh Mehta, Bhumi Dinesh Bhanushali and John Galeotti, “Ultrasound-Based Tracking Of Partially In-Plane, Curved Needles,” 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), 2021

Alex Ling-Yu Hung, Edward Chen, and John Galeotti, “Weakly-and semi-supervised probabilistic segmentation and quantification of ultrasound needle-reverberation artifacts to allow better ai understanding of tissue beneath needles,” 2021

Edward Chen, Tejas Sudharshan Mathai, Vinit Sarode, Howie Choset, and John Galeotti, “A Study of Domain Generalization on Ultrasound-based Multi-Class Segmentation of Arteries, Veins, Ligaments, and Nerves Using Transfer Learning,” NeurIPS Machine Learning for Health Workshop (ML4H), 2020

Edward Chen, Howie Choset, and John Galeotti, “Uncertainty-based Adaptive Data Augmentation for Ultrasound Anatomical Variations,” 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), 2021 (Accepted as oral presentation)

Edward Chen, Howie Choset, and John Galeotti, “Ultrasound AI for RoboTRAC in Far-Forward Environments: Live-Pig Identification of Femoral Arteries, Veins, Ligaments, and Nerves,” Military Health System Research Symposium (MHSRS), 2020

Edward Chen, Abhimanyu, Vinit Sarode, Howie Choset, and John Galeotti, “Multi-Class Bayesian Segmentation of Robotically Acquired Ultrasound Enabling 3D Site Selection along Femoral Vessels for Planning Safer Needle Insertion,” 2021

Edward Chen, Tejas Sudharshan Mathai, Howie Choset, and John Galeotti, “Stochastic Temporal Data Augmentation for Adaption to Out-of-Distribution Temporal Features,” 2021