1059: Vision Transformer Assisting Rheumatologists in Screening for Capillaroscopy Changes in Systemic Sclerosis: An Artificial Intelligence Solution

Alexandru Garaiman¹, Farhad Nooralahzadeh², Carina Mihai¹, Nikitas Gkikopoulos¹, Nicolas Perez Gonzalez², Mike Becker¹, Oliver Distler¹, Michael Krauthammer³ and Britta Maurer⁴, ¹Department of Rheumatology, University Hospital Zurich, University of Zurich, Zürich, Switzerland, ²Department of Biomedical Informatics, University of Zurich, Zürich, Switzerland, ³Department of Quantitative Biomedicine, University of Zurich, Zürich, Switzerland, ⁴Department of Rheumatology and Immunology, University Hospital Bern, University of Bern, Bern, Switzerland

Background/Purpose: An accurate assessment of nailfold capillaroscopy (NFC) images has great importance in the diagnosis and prognosis of systemic sclerosis (SSc). To overcome some of the inherent problems with NFC image analysis, there is an interest to automate and standardize NFC image assessment using computer vision algorithms, such as the Vision Transformer (ViT), which are based on recent advances in deep learning. Therefore, our aims were to implement and assess the performance and the reliability of ViT in detecting changes on NFC images, and to compare the performance of ViT to that of practicing rheumatologists.

Methods: We used NFC images of patients with SSc enrolled in our European Scleroderma Trials and Research group (EUSTAR) and Very Early Diagnosis of Systemic Sclerosis (VEDOSS) local registries. Concretely, we included routine NFC images (all NFC images available - digit II to V of both hands -, irrespective of any image artefacts) of patients aged ≥ 18 years with visits between 2012 and 2021, who fulfilled either the 2013 ACR/EULAR classification criteria for SSc (established disease) or the preliminary criteria for VEDOSS.

ViT was trained to identify the following NFC signs of microangiopathy: enlarged capillaries (apex diameter > 20 μm, and < 50μm), giant capillaries ( > 50 μm), loss of capillaries (< 7 capillaries/mm), and microhemorrhages. If on a NFC image extremely lowered capillary density (≤ 3 capillaries/mm) OR presence of giant capillaries were detected, that NFC image was labeled as "presence of scleroderma pattern" positive.

VIT's performance was evaluated in a cross-fold (k = 5 folds) validation setting, and the gold standard was the treating physician. Area under the ROC curve (AUC) was used as indicator of the performance.

From the first cross-fold, a randomly sampled reliability set was created which was used to compare the NFC assessment performance of four rheumatologists (three invited and trained annotators and the treating physician) with that of ViT.

Results: We analyzed 17,126 NFC images of 234 EUSTAR patients (14.6% males, 67.8% limited cutaneous SSc, median age 57 years, median disease duration 9 years) and 55 VEDOSS patients (92.7% females, median age 44 years, median time elapsed since first Raynaud's phenomenon 5 years).

ViT had good mean performance in identifying the NFC changes in all five folds. The AUC ranged from 81.8% to 84.5% for identifying the different microangiopathic changes and 83.2% for detecting the presence of scleroderma pattern.

In the reliability set (464 images), see figure 1, we observed highest performance for diagnosing giant capillaries (AUC = 92.6%), followed by identification of enlarged capillaries (AUC = 90.2%). Good AUCs were seen in depicting capillary loss (AUC = 86.7%) and microhemorrhages (AUC = 85.0%). In detecting the presence of scleroderma pattern, ViT achieved also a good performance (AUC = 88.5%). The rheumatologists had generally higher performance in assessing NFC images. However, ViT outperformed one rheumatologist in classifying the capillary loss.

Conclusion: ViT is a well performing and readily available AI model to assess signs of microangiopathy on NFC images acquired during routine practice.
Figure 1. Sensitivity/specificity trade-off across the images in the reliability set.
The continuous lines show the ROC curves corresponding to the ViT predictions, with ViT’s AUC, "optimal" sensitivity and specificity shown in the box. Also shown are the point performances of each of the four human annotators, with performances below the continuous line indicating lower performance than ViT.
Abbreviations: A, annotator A; B, annotator B; C, annotator C; AUC, area under the ROC curve; P, attending physician; SE, sensitivity; SP, specificity.

Disclosures: A. Garaiman, None; F. Nooralahzadeh, None; C. Mihai, Boehringer-Ingelheim, Mepha, MEDTalks, Roche, Janssen; N. Gkikopoulos, None; N. Perez Gonzalez, None; M. Becker, Amgen, Bayer, Mepha, Novartis, Vifor, Novartis Foundation; O. Distler, AbbVie/Abbott, Amgen, GlaxoSmithKlein(GSK), Novartis, Roche, UCB, Kymera, Mitsubishi Tanabe, Boehringer Ingelheim, 4P-Pharma, Acceleron, Alcimed, Altavant Sciences, AnaMar, Arxx, AstraZeneca, Blade Therapeutics, Bayer, Corbus Pharmaceuticals, CSL Behring, Galapagos, Glenmark, Horizon, Inventiva, Lupin, Miltenyi Biotec, Merck/MSD, Prometheus Biosciences, Redx Pharma, Roivant, Sanofi, Topadur, Pfizer, Janssen, Medscape, Patent issued “mir-29 for the treatment of systemic sclerosis” (US8247389, EP2331143), FOREUM Foundation, ERS/EULAR Guidelines, EUSTAR, SCQM (Swiss Clinical Quality Management in Rheumatic Diseases), Swiss Academy of Medical Sciences (SAMW), Hartmann Müller Foundation; M. Krauthammer, Oncobit; B. Maurer, Boehringer-Ingelheim, Medtalk, Pfizer, Roche, Actelion, Mepha, Merck/MSD, Novartis, Boehringer-Ingelheim, Janssen, AbbVie/Abbott, Protagen, Novartis.