1022: Validation of the SPARCC_RETIC E-Tool for Ensuring Scoring Proficiency of MRI Lesions in Axial Spondyloarthritis

Walter P Maksymowych¹, Anna Hadsbjerg², Mikkel Østergaard³, Raphael Micheroli⁴, Susanne J. Pedersen⁵, Adrian Ciurea⁶, Nora Vladimirova², Michael Nissen⁷, Kristyna Bubova⁸, Stephanie Wichuk⁹, Manouk de Hooge¹⁰, Ashish J Mathew¹¹, Karlo Pintaric¹², Monika gregova¹³, Ziga Snoj¹², Marie Wetterslev¹⁴, Karel Gorican¹⁵, Joel Paschke¹⁶, Iris Eshed¹⁷ and Robert G Lambert⁹, ¹Department of Medicine, University of Alberta, Edmonton, AB, Canada, ²Copenhagen Center for Arthritis Research, Copenhagen, Denmark, ³Rigshospitalet, University of Copenhagen, Glostrup, Denmark, ⁴University Hospital Zurich, Department of Rheumatology, Zürich, Switzerland, ⁵Rigshospitalet, University of Copenhagen, Copenhagen, Denmark, ⁶University Hospital Zurich, Zürich, Switzerland, ⁷Hopitaux Universitaires de Genève, Geneva, Switzerland, ⁸First Faculty of Medicine, Charles University,, Prague, Czech Republic, ⁹University of Alberta, Edmonton, AB, Canada, ¹⁰Ghent University Hospital, Luxembourg, Luxembourg, ¹¹Rigshospitalet, Copenhagen, Denmark, ¹²Radiology, University Medical Centre Ljubljana, Ljubljana, Slovenia, ¹³Institue of Rheumatology, Prague, ¹⁴Rigshospitalet, Copenhagen, Glostrup, Denmark, ¹⁵MSK radiology unit, Radiology section – Diagnostic department, Geneva University Hospitals, Geneva, Switzerland, ¹⁶CARE Arthritis LTD, Edmonton, AB, Canada, ¹⁷Sheba Medical Center, Tel Aviv, Israel

Background/Purpose: Quantification of inflammatory and structural lesions in the sacroiliac joints (SIJ) on MRI scans from patients with axial spondyloarthritis (axSpA) has a growing list of applications, especially for clinical trials. It is therefore essential that knowledge transfer tools are made available that can ensure understanding of scoring methodologies and enable remote training of readers across the globe so that they can reliably perform assessments. The web-based Spondyloarthritis Research Consortium of Canada (SPARCC) real-time iterative calibration (RETIC) modules for scoring inflammatory and structural MRI lesions in axSpA have been created by SPARCC developers to enable remote training of readers to appropriately use the SPARCC MRI inflammation and structural damage instruments and to attain adequate scoring proficiency. We aimed to test the performance of these modules in enhancing scoring proficiency in comparison to SPARCC developer gold standard.

Methods: The SPARCC_RETIC SIJ inflammation and structural damage modules are each comprised of 50 DICOM axSpA cases with baseline and follow up scans and an online scoring interface based on SIJ quadrants. Continuous visual real-time feedback regarding concordance/discordance of scoring per SIJ quadrant with expert readers is provided by a color-coding scheme. Reliability is assessed in real-time by intra-class correlation coefficient (ICC), ICC data being provided every 10 cases, which are scored until proficiency targets for ICC are attained. Participants (n=15) from the EuroSpA Imaging project were randomized, stratified by reader expertise in scoring with SPARCC, to one of two reader training strategies (A and B) that each comprised 3 stages: A. 1. Review of original SPARCC manuscript describing scoring method. 2. Review of powerpoint summary of SPARCC method plus completion of SPARCC_RETIC module. 3. Re-review of powerpoint summary. B. Same 3-step strategy as A except SPARCC_RETIC module completed at stage 3. Participants scored scans from 25 axSpA cases blinded to time point after each stage of training and reliability of scoring was compared to SPARCC developer scores by ICC. Each of the 75 cases that were scored by the 15 readers had 2 time points varying from 3 months to 2 years and each batch of 25 cases had comparable scores for BME and structural lesions when scored by SPARCC developers.

Results: Very good scoring proficiency for status and change scores was evident for SPARCC BME even at stage 1 and even by non-experienced readers with similar levels of reliability irrespective of reader prior expertise with the method (Table). The beneficial impact of the SPARCC_RETIC module on scoring proficiency was most consistently evident for the scoring of structural lesions and for Strategy B, where the impact was evident for all structural lesions, level of reader expertise, and status as well as change scores (Figures 1 and 2).

Conclusion: Attaining scoring proficiency for MRI structural lesions in axSpA is difficult but can be consistently improved by using the SPARCC_RETIC module, even for experienced readers.
Table. Inter-rater reliability (Status/Change ICC) compared to radiologist SPARCC developer

Figure 1

Figure 2
Disclosures: W. Maksymowych, AbbVie, Boehringer-Ingelheim, Celgene, Eli Lilly, Galapagos, Janssen, Novartis, Pfizer, UCB, CARE Arthritis Limited; A. Hadsbjerg, Novartis; M. Østergaard, AbbVie/Abbott, Amgen, Bristol-Myers Squibb(BMS), Celgene, Eli Lilly, Janssen, Gilead, Novartis, Pfizer, UCB; R. Micheroli, None; S. Pedersen, Novartis, AbbVie/Abbott, UCB, Pfizer, Merck/MSD; A. Ciurea, AbbVie, Novartis, Merck/MSD; N. Vladimirova, Novartis; M. Nissen, AbbVie/Abbott, Pfizer, Amgen, Novartis, Janssen; K. Bubova, None; S. Wichuk, None; M. de Hooge, None; A. Mathew, None; K. Pintaric, None; M. gregova, None; Z. Snoj, None; M. Wetterslev, None; K. Gorican, None; J. Paschke, None; I. Eshed, None; R. Lambert, Calyx, CARE Arthritis, Image Analysis Group.