Video-based AI for Beat-to-Beat Assessment of Cardiac Function

Authors and Affiliations

• David Ouyang, Stanford University
• Bryan He, Stanford University
• Amirata Ghorbani, Stanford University
• Neal Yuan, Cedars-Sinai Medical Center
• Joseph Ebinger, Cedars-Sinai Medical Center
• Curtis P. Langlotz, Stanford University
• Paul A. Heidenreich, Stanford University
• Robert A. Harrington, Stanford University
• David H. Liang, Stanford University
• Euan A. Ashley, Stanford University
• James Y. Zou, Stanford University

Background and Importance

• Accurate cardiac function assessment is crucial for cardiovascular disease diagnosis and management.
• Traditional human assessment involves limited cardiac cycle sampling and is prone to inter-observer variability.
• EchoNet-Dynamic is a deep learning algorithm that surpasses human performance in segmenting the left ventricle, estimating ejection fraction (EF), and assessing cardiomyopathy.

EchoNet-Dynamic Overview

• Model Capabilities:
  • Segments the left ventricle with a Dice Similarity Coefficient of 0.92.
  • Predicts ejection fraction with a mean absolute error of 4.1%.
  • Classifies heart failure with reduced ejection fraction with an AUC of 0.97.
• Performance:
  • External dataset prediction: mean absolute error of 6.0%, AUC of 0.96.
  • Comparable or less variance than human experts in prospective evaluation.

Methodology

Video-based Deep Learning Model Components

1. Semantic Segmentation:

   • Utilizes atrous convolutions for frame-level segmentation of the left ventricle.
   • Automatic segmentation mimics clinical workflow by identifying ventricular contractions.

2. Ejection Fraction Prediction:

   • Employs a CNN with residual connections and spatiotemporal convolutions.
   • Combines spatial and temporal data for accurate EF prediction.

3. Beat-to-Beat Evaluation:

   • Generates video-level predictions and averages EF estimates across multiple beats.
   • Developed on 10,030 echocardiograms from Stanford Medicine.

Evaluation and Results

• Internal Stanford Test Dataset:

  • EchoNet-Dynamic achieved a mean absolute error of 4.1% and R2 of 0.81.
  • Outperformed other deep learning architectures.

• External Dataset from Cedars-Sinai:

  • Robust EF prediction with a mean absolute error of 6.0% and AUC of 0.96.

• Comparison with Human Variation:

  • Lower variance compared to traditional methods in prospective study of 55 patients.

Left Ventricle Segmentation Analysis

• Automatic segmentation enables clinicians to understand predictions.
• Provides a point for human oversight in the clinical workflow.

Discussion and Implications

• EchoNet-Dynamic introduces a new standard for cardiac function assessment.
• Predicts EF rapidly and accurately, potentially aiding in more precise clinical assessments.
• Largest annotated medical video dataset released to stimulate future research.

Future Directions

• Model robust to video acquisition variation but needs further validation in diverse environments.
• Potential to augment current methods with greater precision and early detection capabilities.

Key References

• Multiple studies cited supporting the significance of accurate cardiac function assessment and the efficacy of deep learning models in this domain.