1334818 - Reducing the Need for Annotated Training Data Using Style Transfer for Mapping CMR

Machine learning solutions for automatic segmentation are still limited by the amount of annotated data available for training. We propose to synthetically generate T1 maps from CINE MRI with deep learning style transfer for pre-training. We show that by pre-training on synthetic data we can improve segmentation results on unseen maps, with or without finetuning on annotated data from a new center.



Methods:

9101 subjects were selected from the UK Biobank resource¹ (access application 30769), where acquisition included a short axis CINE stack and one short axis ShMOLLI T1 map². CINE data was automatically segmented using a previously trained model³ and images with wrong segmentations were manually removed leaving 8977 subjects (CINE data). For 90 subjects reserved for testing, the T1 maps were manually segmented (Mapping1 data), while for the rest no manual annotations were performed. Instead, a CycleGan network⁴ was trained on 1755 paired images (256 for validation) to learn a style transfer between CINE and mapping images (Figure 1). The model was used to generate 8807 synthetic maps (synthetic) with matching segmentations from CINE.

Dataset Mapping2 contains 144 separately acquired clinical subjects (52 normal, 49 myocarditis, 20 sarcoidosis, 23 systemic disease), where T1 MOLLI maps were acquired pre and post contrast and T2 mapping was performed. The maps were manually segmented. Data was randomly divided for training, validation, and testing (100/15/29). All data was acquired at 1.5T (MAGNETOM Aera, Siemens Healthcare, Erlangen, Germany). 

A dense Unet⁵ (Figure 1) was trained for myocardium segmentation from T1 maps using 100%, 50%, 25%, and 10% of the data in Mapping2. The models were trained either from scratch or after pre-training with CINE or synthetic data, and were tested on Mapping1 and Mapping2.



Results:

When the goal is to segment T1 maps from a new center, i.e. Mapping1, without requiring manual annotations, the model pre-trained on synthetic and finetuned on the separately acquired Mapping2 significantly outperforms the model trained on Mapping2 only (Figure 2). Pre-training on CINE directly, without the synthetic image generation step, does not lead to an improvement, while training on CINE and synthetic only leads to poorer results.

When finetuning on data from the same distribution as the testing data, pre-training on synthetic before finetuning on Mapping2 also brings a significant improvement, when evaluation on the test subset of Mapping2. When reducing the amount of training/finetuning data from Mapping2, the impact of pre-training is significantly larger (Figure 3).



Conclusion:

Using synthetically generated T1 maps with deep learning style transfer helps improve segmentation results, without the need for additional manual annotations for the data from a new center, but also when finetuning on scarce data.

The concepts presented are based on research results that are not commercially available.