Machine Learning and Artificial Intelligence

1355791 - High Frame Rate Cardiac Cine using Transformer-Based Phase Interpolator: A Multi-Center Multi-Vendor Study

Thursday, January 26, 2023

11:00 AM – 11:10 AM

Location: Silver Pearl 2

Presenting Author(s)

MM

Manuel A. Morales, PhD

Post-doc Research Fellow
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States

Primary Author(s)

MM

Manuel A. Morales, PhD

Post-doc Research Fellow
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States

Co-Author(s)

AA

Amine Amyar, PhD

Postdoctoral Research Fellow
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States
SY

Siyeop Yoon, PhD

Post-doc Research Fellow
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States
SA

Salah Assana, MSc

Research Assistant
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States
JR

Jennifer Rodriguez

Clinical Trials Specialist
Beth Israel Deaconess Medical Center
Boston, Massachusetts, United States
MM

Martin S. S. Maron, MD

Assistant Professor
Tufts Medical Center, United States
ER

Ethan J. J. Rowin, MD

Assistant Professor
Tufts Medical Center
Boston, Massachusetts, United States
SN

Shiro Nakamori, MD

Senior Lecturer
Mie University Hospital
Tsu, Mie, Japan
Jiwon Kim, MD

Associate Professor of Medicine
Weill Cornell Medicine
New York, New York, United States
Robert M. Judd, PhD

Professor Emeritus
Duke University
Durham, North Carolina, United States
Jonathan W. Weinsaft, MD

Professor of Medicine
Weill Cornell Medical College
New York, New York, United States
Warren J. Manning, MD

Professor of Medicine and Radiology
Harvard Medical School
Boston, Massachusetts, United States
RN

Reza Nezafat, PhD

Professor
Harvard Medical School
Boston, Massachusetts, United States

Disclosure(s):

Jiwon Kim, MD: No financial relationships to disclose

Robert M. Judd, PhD: No relevant disclosure to display

Jonathan W. Weinsaft, MD: No relevant disclosure to display

Warren J. Manning, MD: No relevant disclosure to display

Reza Nezafat, PhD: No financial relationships to disclose

Background: Cardiac cine imaging allows accurate and reproducible measure of cardiac function and volume. In ECG-segmented or real-time cine imaging, there is a trade-off between breath-hold duration and spatiotemporal resolution. Insufficient temporal resolution results in temporal blurring and limits the ability of CMR to accurately measure diastolic parameters. Numerous techniques have been proposed over the past decades to improve spatial or temporal resolution by reducing scan time. Convolutional neural networks have been recently applied in reducing scan time by under-sampling the k-space, but they are limited in the time domain. Thus, we sought to develop a high-frame-rate cine imaging technique using cine phase interpolator transformer (CPIT).

Methods: We propose a CPIT model (Fig. 1) whose input is T = 4 sequential cine images of size W × H collected with temporal resolution Δt at times t - Δt, t, t + Δt and t + 2Δt. The output is an interpolated image with equivalent acquisition time t + 0.5Δt, which was compared to ground-truth using L₁ loss during training. CPIT has 6 pairs of transformer layers. Each layer splits inputs into 2D spatial (W•H•T/M² × M²) and temporal (W•H × T) windows (M = 8). Self-attention is used to learn the spatiotemporal correspondence within each window. Decoding convolutional layers capture rigid and non-rigid motion components for each phase, and these components are used to deform and combine the 4 images onto t + 0.5Δt (Fig. 1A). During training and testing, cine slices with T phases were sampled with frame rate / 2 and / 4 patterns to obtain 4 input phases with temporal resolutions 2Δt and 4Δt (Fig. 1B).

ECG-segmented cine images collected in short-axis and 2-, 3- and 4-chamber views from 5788 subjects, undergoing clinical CMR, were used in a retrospective multivendor (vendors = 3) multisite (sites = 9) study. Initially, a subset from 6 sites (n = 551) was set aside for external testing. Subsequently, data from the remaining internal sites was split into train (80%, n = 4190), validation (10%, n = 524), and test (10%, n = 523) sets. The split was balanced to ensure data from each site was properly represented (Fig. 1C). CPIT was trained for 100 epochs.

Results: CPIT doubled the frame rate of cines with temporal resolutions of 76-150 ms, as quantified by discarding phases from existing data (Fig. 2). The average peak signal-to-noise ratio (PSNR) between discarded and interpolated phases with frame rate / 2 was 45 and 41 for internal and external testing, accordingly (Fig. 3A). For frame rate / 4 the PSNR was 41 and 39 (Fig. 3B). CPIT was further capable of interpolating beyond the original frame rates to realize 2- and 4-fold gains in temporal resolution, achieving a temporal resolution of up to 10 ms (Fig. 3C).

Conclusion: We demonstrated a high frame rate cardiac cine using a transformer-based interpolator and evaluated its performance in a multivendor multisite study.