Ultrasound Research at ISBI 2018

The International Symposium on Biomedical Imaging (ISBI) 2018 was held April 4-7 in Washington, DC.  In 2017, ISBI hosted 15 papers on ultrasound research.  At this year’s ISBI, ultrasound research was featured in a tutorial, 7 invited presentations, and 25 poster and podium presentations!  Ultrasound is infiltrating nearly every aspect of patient care: providing new diagnostic tests, generating new insights, reducing costs, and improving outcomes.

The following is a summary of the papers and presentations on ultrasound research at ISBI 2018. I created these notes for my own use, and I am sharing them in the hope of drawing others into this exciting field of research and clinical applications.

If you want to learn more about the state of ultrasound research, see live demonstrations, or present your own ultrasound research, please consider attending or submitting a paper to our upcoming MICCAI 2018 Workshop on Point-Of-Care Ultrasound Algorithms, Hardware, and Applications (call for papers). To learn more about ultrasound research at MICCAI, see my previous post.

Tutorial on Elastography and Photoacoustic Imaging

Tutorial Chairs: Abbas Samani, Western University; Hassan Rivaz, Concordia University; Mohammad Mehrmohammadi, Wayne State University

This half-day tutorial presented the principles and methods of elastography and photoacoustic imaging.  It also presented an overview of clinical applications that are supported by these modalities. For those who want an introduction to these modalities, this tutorial is an excellent starting point.  Hopefully their tutorial slides will soon be available online from http://biomedicalimaging.org/2018/tutorials/

Special Session on Ultrasonic Imaging-Based Tissue Characterization

Session Chair: Lavarello, Roberto (Pontificia Universidad Catolica Del Peru)

Multi-Parametric Ultrasound-Based Elastographic Characterization of Renal Transplants

  • Urban, Matthew (Mayo Clinic), et al.
  • Uses shear wave elastography to monitor kidney transplants.  Various elastic parameters (e.g., shear wave modulus and velocities) were shown to be well correlated with various biopsy-based measures.

Next-Generation Ultrasound Imaging for Assessing the Microvascular Fingerprint of Cancer

  • Dayton, Paul (UNC Chapel Hill), et al.  (Note that I am a co-author on this paper)
  • A dual-frequency (transmit low, receive high) probe is used with a micro-bubble contrast agent to create angiographic images.  The tortuosity and other morphological characteristics of the microvasculature of tumors imaged using this method are shown to distinguish benign from malignant tumors and to be early indicators of tumor response to therapy in humans and in pre-clinical trials.

The H-Scan Sensitivity to Scatterer Diameter

  • Parker, Kevin (Univ. of Rochester)
  • Gaussian Weighted Hermite functions are used to characterize the scattering and reflecting types in an image.  The theory suggest that the system will be able to distinguish cellular and vascular changes, with potential application to studying inflammation and injury.  Phantom and ex vivo tissue studies support this theory.

Recent Developments in Spectral-Based Ultrasonic Tissue Characterization

  • Lavarello, Roberto (Pontificia Univ. Catolica Del Peru), et al.
  • Presents spectral-based ultrasound characterization methods, emphasizing recent advances in in vivo imaging, image penetration, simultaneous estimation of backscatter coefficients, shear wave speed maps, and attenuation slope maps.

Ultrasound Tomography of the Human Breast

  • Wiskin, James (QT Ultrasound, LLC), et al.
  • Presents theory and application of tomographic whole-breast reconstruction, with emphasis on estimating speed of sound variations associated with different breast tissue types.  Employing speed of sound, attenuation, and reflectivity measures enabled machine learning for tissue classification.

Special Session on Imaging for Developing Countries

Session Chairs: Miguel A. Gonzalez-Ballester (Universidad Pompeu Fabra, Spain); Erin Girard (Siemens, USA)

Use of Advanced Technology in Lower-Middle Income Countries to Improve Outcomes

  • Chowdhury, Devyani (Cardiology Care for Children)
  • Highlighted the clinical considerations and potential of using machine learning to identify at-risk patients and optimize the allocation of limited resources.  Particularly emphasized machine learning with echocardiography to detect at-risk neonates.

Computer-Augmented Point-Of-Care Ultrasound: Technologies and Applications

  • Aylward, Stephen (Kitware Inc), et al.  (Note that I am the lead author on this paper)
  • Focused on the application of ultrasound to in-field trauma patient triage and scoliosis monitoring. The proposed computer-augmented POCUS systems utilize custom hardware and machine learning so that the system can be operated without extensive clinical training.

Poster and Podium Presentations

Deep Learning with Ultrasound Physics for Fetal Skull Segmentation

  • Cerrolaza, Juan J. (Imperial College London), et al.
  • Uses a 3D UNet architecture to identify skulls in fetal ultrasound images.  It contains extra heuristics (e.g., distance map and shadow map computations) in order to fill-in the skull segments that are tangential to the ultrasound signal lines and thereby don’t appear as hyperechogenic voxels.

Universal Multi-Modal Deep Network for Classification and Segmentation of Medical Images

  • Harouni, Ahmed* (IBM Almaden Research Center), et. al.
  • Explores training a single network (a modified UNet) for segmentation and classification of objects in images from different modalities, including ultrasound.  The modification is that the middle layer of the UNet (the last of the compression layers) is not only fed to the standard decompression subnet that performs segmentation but it is also fed to a subnet that performs classification.

Echocardiography Segmentation based on a Shape-Guided Deformable Model Driven by a Fully Convolutional Network Prior

  • Veni, Gopalkrishna (IBM Almaden Research Center), et al.
  • Uses a convolutional neural network (UNet) to produce an initial segmentation estimation for the objects in an image.  That initial segmentation estimate serves as a spatial prior for a final level set segmentation of the objects in the image.  Furthermore, the level set evolution uses a modified energy term based on expected intensity values of the different components (e.g., ventricles should be dark and the muscle around them should be bright).

Ultrasound Imaging for Identifying Dynamics of Soft Tissue

  • Yamada, Daisuke (Harvard University and Toyota Research Institute North America), et al.
  • Induces whole-body vibration (by vibrating a chair) and uses tracked ultrasound to measure the resultant movement of tissues, by tracking a single point within an ultrasound sequence. Ultimate goal is predicting vehicular ride comfort.

Restoration of Ultrasonic Images using Non-Linear System Identification and Deconvolution

  • Hourani, Mohamad (Université de Toulouse), et al.
  • Provides a method for estimating tissue reflectivity function using a joint linear and non-linear representation of the probes output.  Estimation is accomplished without the need of explicitly defining the input/excitation function or the probe’s point spread function, by using a deconvolution approach.  Results on phantom data are presented.

Automatic Kidney Segmentation in 3D Pediatric Ultrasound Images using Deep Neural Networks and Weighted Fuzzy Active Shape Model

  • Tabrizi, Pooneh R. (Children’s National Health System),, et al.
  • Deep learning (3D convolution followed by fully connected layers) produces a bounding box (location, orientation, and scale measures) containing the kidney, and fuzzy active shape modeling can then segment the boundary of the kidney.  Applied to hydronephrosis diagnosis (kidney dilation estimation) in pediatrics – only considered non-severe cases.

Simulating Patho-Realistic Ultrasound Images using Deep Generative Networks with Adversarial Learning

  • Tom, Francis (Indian Institute of Technology Kharagpur), et al.
  • Simulates intravascular ultrasound images.  Physical model initializes the process. First GAN refines the simulator output to create a low-res image that resembles the input and that fools the discriminator.  Second GAN converts low-res to high-res image.

Multi-Rate Higher Order Singular Value Decomposition for Enhanced Non-Contrast Ultrasound Doppler Imaging of Slow Flow

  • Bayat, Mahdi (Mayo Clinic), et al.
  • Goal is to separate blood flow from tissue motion.  Uses echo data at multiple temporal resolutions simultaneously (instead of at one or two used in conventional Doppler), in combination with SVD, to generate tensor representations that capture Doppler events at a multiple flow rates – enabling blood flow rate analyses.  Simulations and in vivo breast imagings were performed to demonstrate utility.

Direct Strain Estimation in Ultrasound Elastography using a Novel Dynamic Programming Approach

  • Khodadadi, Hossein (Concordia Univ.), et al.
  • Instead of optimizing a discrete displacement field to maximize the match between a sequence of RF fields and to minimize the displacement field curvature, it optimizes the gradient of the displacement field (actually axial strain and lateral displacement) in order to maximize RF field match.  Compared with another method on intra-operative liver data.

Regularization Methods for Solving Third-Order Volterra Filter with Improved Convergence Speed: in-Vivo Application

  • Cunningham, James (The Pennsylvania State Univ.), et al.
  • Certain ultrasound applications achieve improved anatomic feature conspicuity using harmonic imaging using a 3rd-order Volterra filter.  The authors use Truncated-SVD for regularization to more quickly solve for Volterra filter coefficients than least mean square methods, potentially enabling real-time use. Evaluations are performed using transcranial ultrasound imaging of hypoechogenic regions.

An Intuitive Muscle-Computer Interface using Ultrasound Sensing and Markovian State Transitions

  • Dhawan, Ananya S. (George Mason University), et al.
  • Traditional ultrasound and other methods for controlling prostheses require returning to a neutral/rest state before initiating a new motion.  The presented system combines hidden markov models with ultrasound sensors to implement state transitions that support initiating moves (all subsequent moves having equal probability) during transition periods.  Prediction performance experiments are conducted.

Quantification of Cardiac Bull’s-Eye Map based on Principal Strain Analysis for Myocardial Wall Motion Assessment in Stress Echocardiography

  • Omar, Hasmila A. (Univ. of Oxford), et al.
  • Compares hand-crafted cardiac strain image features (e.g., principal strain encodings) with convolutional neural networks on strain images for distinguishing healthy from diseased (abnormal wall motion) in 3D Dobutamine Stress Echo videos.  Considered random forest, SVM, and CNN methods. Results were similar.

VibroEEG: Improved EEG Source Reconstruction by Combined Acoustic-Electric Imaging

  • Tsizin, E. (Tel Aviv University), et al.
  • Improves spatial localization of EEG sources by using ultrasound to induce localized resonant vibrations that selectively displace local sources and then measuring the cyclic effect on the signals recorded on the scalp.

Automatic Needle Localization in 3D Ultrasound Images for Brachytherapy

  • Younes, Hatem (University Grenoble Alpes), et al.
  • Goal is accurate localization of the curved path of needles (brachytherapy strands) in 3D ultrasound images.  Uses planned path to define ROI, gaussian mixture modelling to label needle voxels, and optimizes a parametric curve to define the needle path through labeled needle voxels.

Enhancement and Automated Segmentation of Ultrasound Knee Cartilage for Early Diagnosis of Knee Osteoarthritis

  • Desai, Prajna Ramesh (Rutgers, The State University of New Jersey), et al.
  • Target application is joint cartilage thickness estimation. Uses oriented Log-Gabor filters, noise estimates, and phase-based images features weighted by their distance from transducer to enhance the bone boundaries in an image.  Bone shadow enhancement is then applied, and dynamic programming is use to segment the two bone surfaces and the region between them. Cartilage segmentation follows, using the log-gabor enhanced images and heuristically specified seed points for region growing.  Evaluation involved 100 scans with mean / std dev of dice scores reported.

SonoEyeNet: Standardized Fetal Ultrasound Plane Detection Informed by Eye Tracking

  • Cai, Y. (University of Oxford), et al.
  • Combined eye movement data (visual fixation information) with ultrasound image features maps to improve the accuracy of detecting AC plane images in fetal ultrasound video clips.   Deep learning was used for the machine learning method. Evaluation involved 323 ultrasound videos.

Auto-Context Fully Convolutional Network for Levator Hiatus Segmentation in Ultrasound Images

  • Wang, Na (Shenzhen Univ.), et al.
  • Goal is segmentation of the levator hiatus in pelvic ultrasound. Integrates a fully convolutional network with an auto-context model (i.e., multiple layers that have the input image and the output of previous auto-context layer as inputs) for iteratively refining a prediction map.  Active shape model is then applied to that prediction maps to produce a boundary/segmentation (can address gaps in the image boundary, etc). Evaluated using 372 images from 284 patients. Compared with multiple other deep learning networks.

A Method of Ultrasound Simulation from Patient-Specific CT Image Data: A Preliminary Simulation Study

  • Satheesh B, Anila (Indian Institute of Tech. Madras), et al.
  • Most current simulation methods begin with simulating a scatter image (not view dependent) using the Field II package, and then augmenting that image with echo reflection and intensity transmission simulations.  The authors propose to replace the Field II package with a convolution model that integrates a point-spread function with a tissue scatter function – which has been previously published.

Transfer Learning for Diagnosis of Congenital Abnormalities of the Kidney and Urinary Tract in Children based on Ultrasound Imaging Data

  • Zheng, Qiang (University of Pennsylvania, Yantai University and Chinese Academy of Sciences), et al.
  • Demonstrated that the combination of transfer learning features, traditional (HOG) features, and geometric features enable an SVM classifier to identify morphological patterns of kidney images better than any subset of features alone.  Transfer learning achieved by converting an image, its gradient magnitude image, and its distances transform image into RGB channels as needed by the transfer learning network (ImageNet).

Ultrasound Elastography of Breast Cancer-Related Lymphedema

  • Hashemi, H.S. (Univ. of British Columbia), et al.
  • Goal is staging breast cancer-related lymphedema using ultrasound. Investigated a novel strain imaging method using quasi-static ultrasound elastography.  Extends work on GLUE (GLobal Ultrasound Elastography) – adds second-order model of displacement into the cost function and presents a new optimization method.  Phantom results and results from two patients are given.

A Time Domain Method to Monitor Temperature in Microwave Hyperthermia using Ultrasound Attenuation

  • Kothawala, AliArshad (Indian Institute of Tech. Madras)
  • Measures attenuation of pulse-echo ultrasound and correlates its change with temperature change resulting from microwave ablation/heating/hyperthermia.  Uses published model (high-order polynomial) that maps attenuation to temperature.

Scoliosis Screening and Monitoring using Self Contained Ultrasound and Neural Networks

  • Greer, Hastings (Kitware Inc.), et al. (Note that I am an author on this paper)
  • Presents a self-contained ultrasound system for estimating scoliosis in children.  An IMU attached to the probe estimates the angle between the probe and vertical/gravity.  A time-series of convolutional neural networks analyzes images from the probe to estimate the angle between a vertebral body and the probe.  Combining both angle estimates gives the angle between a vertebral body and vertical/gravity. The greatest angle between two vertebral bodies is the Cobb angle.  Studies involved a physical spine phantom.

Detecting Anomalies from Echocardiography using Multi-View Regression of Clinical Measurements

  • Lu, Allen* (Yale University); et al.
  • Uses multiple ultrasound views (various windows in m-mode, b-mode, and doppler) to directly estimate measures typically made on those views.  Measures are converted to pixel units to provide resolution insensitivity. Currently using 3 views, b-mode – and a CNN – to estimate LV diameter, area, and volume.  Various network architectures, training set sizes, and problem encodings are evaluated.

End-to-End Detection-Segmentation Network with ROI Convolution

  • Zhang, Zichen* (Univ. of Alberta), et al.
  • Improves segmentation output of a fully convolutional network by including a localization unit – network performs localization followed by segmentation.  Localization unit is the previously published region proposal network, which classifies foreground and background regions. Novel is that ROI info is only used to mask the convolution layer’s output, prior to feeding to the rest of the network.  That is, convolution outputs outside the ROI are set to 0 and do not contribute to subsequent convolution computations. Applied to diagnosis of developmental dysplasia of the hip (DDH) in children from (49) 3D ultrasound images. Dice score is used to compare this method with other networks of varying sizes and formulations.

Automated Ovarian Volume Quantification in Transvaginal Ultrasound

  • Narra, Ravi Teja (Samsung R&D Institute), et al.
  • Goal is automating the quantification of ovarian volume on 3D transvaginal ultrasound.  Deep learning provides an energy map for level set segmentation. UNet network generates energy map after being trained to produce binary segmentations.  Evaluated using 50 volumes (1500 radial slices). Dice and volume measurements were compared with experts.

Placental Maturity Grading via Hybrid Descriptors based on Fisher Vector

  • Jiang, Feng (Shenzhen University), et al.
  • Features from b-mode and doppler images are extracted by SIFT filters and by (transfer learning) CNNs, those features are summarized using Fisher vector encodings, and those encodings are then processed by a Support Vector Machine (SVM) to generate a grade of placental maturity.  Various networks, feature combinations, and Fisher kernels are compared. 212 image pairs are split 80/20 for training/testing.

Questions or comments are always welcome!