---

About

Model Facts

Summary

This algorithm automatically measures and detects signs of carpal instability in conventional radiographs of the hand and wrist. It was designed for processing a radiographic study with an arbitrary number of series. It was developed in 2022-2023 at the Radboud University Medical Center and Jeroen Bosch Hospital in the Netherlands.

Mechanism

Algorithm description: The spatial and geometric properties of the relevant carpal bones are first determined by segmentation with convolutional neural networks (CNNs) and are then used to identify the articular facet joint surfaces with multi-scale active appearance models (AAMs). Based on the obtained bone surfaces and angles, the following carpal instability measurements and detections can be conducted: (a) scapholunate (SL) joint distance in mm, (b) SL angle in degrees, (c) capitolunate (CL) angle in degrees, (d) markers of potential disruptions of the carpal arcs with an overall disruption score. Interruptions of the carpal arcs are determined by comparing the observed and reconstructed hypothetical shape of the carpal arc polylines if non-interrupted (obtained from a point distribution model [PDM]). A heat map using vectors and color-coding visualizes the degree (z-score) and location of carpal arc interruptions in the original image.

Training data location and time-period: Picture archiving and communication systems of the Radboud University Medical Center and Jeroen Bosch Hospital in the Netherlands, radiographs acquired between January 2018 and April 2019.

Input:

• Data type: frontal view (i.e., [neutral, ulnar-deviated, clenched fist] anterior-posterior [AP], posterior-anterior [PA]), and lateral view of the hand or wrist.
• File format: DICOM or MHA file containing at least the Pixel Spacing Attribute (tag: 0028,0030) or Imager Pixel Spacing Attribute (tag: 0018,1164). The Photometric Interpretation Attribute (tag: 0028,0004) should be "MONOCHROME2" (minimum value is intended to be displayed as black).
• Target population: all patients with sufficiently developed carpal bones (at least seven years old, preferably older than 18 years).

Output:

• PDF report (report.pdf): PDF report with a study-level analysis, providing a case summary on the first page and screenshots of the measurements on the subsequent pages. Patient and study metadata are anonymized by default on Grand Challenge and are not available in the current set-up. This means that the normal value range for the scapholunate distance measurement cannot be matched to the patient age and sex and will be automatically set to <3 millimeter (normal range for adults).
• Results JSON file (results.json): JSON file containing the raw outputs of the algorithm, which could be used for linking this algorithm to other applications or for debugging purposes.

IMPORTANT: Currently, Grand Challenge does not support the use of image inputs of arbitrary length and therefore the input has been limited to two images. Furthermore, it is important to note that the processing per study can take a significant time in the AWS cloud (due to model initialization, node availability, and node hardware specifications). On the local machine that was used for our experiments (Nvidia RTX Titan GPU and Intel i9-9900K CPU), the average processing time per study was approximately half a minute.

Validation and Performance

Carpal bone segmentation:

Bone	n	Radiographic view	Mean DSC	Mean HD (mm)
Scaphoid	1103	Frontal	0.95 ± 0.09	1.5 ± 1.7
Lunate	1095	Frontal	0.94 ± 0.12	1.3 ± 1.3
Triquetrum	1089	Frontal	0.90 ± 0.14	1.7 ± 1.1
Capitate	1107	Frontal	0.95 ± 0.04	2.0 ± 1.3
Hamate	1098	Frontal	0.90 ± 0.12	2.0 ± 1.5
Scaphoid	1060	Lateral	0.93 ± 0.06	2.4 ± 1.8
Lunate	1060	Lateral	0.93 ± 0.09	1.8 ± 1.7
Capitate	1061	Lateral	0.94 ± 0.04	2.2 ± 1.3

Note. — The carpal bones were segmented in 2178 radiographs from 1993 patients (see dataset 1 in paper). The metrics are reported with their standard deviation. Each bone is not always depicted, and the number of masks (n) is reported. Frontal view included neutral, ulnar-deviated, clenched fist anterior-posterior (AP) or posterior-anterior (PA) view and oblique view. DSC = Dice similarity coefficient, HD = symmetric Hausdorff distance, mm = millimeter.

Anatomical landmark localization:

Bone	Radiographic view	Mean fitting error	Mean DSC	Mean HD (mm)
Scaphoid	Frontal	0.044 ± 0.022	0.86 ± 0.07	3.2 ± 1.6
Lunate	Frontal	0.081 ± 0.096	0.85 ± 0.08	2.8 ± 1.4
Triquetrum	Frontal	0.069 ± 0.039	0.81 ± 0.09	3.3 ± 1.6
Capitate	Frontal	0.047 ± 0.026	0.89 ± 0.05	3.8 ± 2.4
Hamate	Frontal	0.054 ± 0.034	0.73 ± 0.09	3.3 ± 1.4
Lunate	Lateral	0.055 ± 0.023	0.86 ± 0.06	3.6 ± 2.1
Capitate	Lateral	0.046 ± 0.019	0.89 ± 0.05	3.3 ± 1.4

Note. — The articular facet joint surfaces were labelled in 400 radiographs from 400 patients (200 frontal view, 200 lateral view, equally distributed between hospitals, see dataset 1 in paper). All bones were (fully) depicted in the radiographs. The metrics are reported with their standard deviation. Frontal view included neutral, ulnar-deviated, clenched fist anterior-posterior (AP) or posterior-anterior (PA) view and oblique view. DSC = Dice similarity coefficient, HD = symmetric Hausdorff distance, mm = millimeter.

Carpal instability measurement and detection results:

	SL distance measurement error (mm)	SL angle measurement error (degrees)	CL angle measurement error (degrees)	Carpal arcs measurement error (mm)
MAE	0.65 (0.59, 0.72)	7.9 (7.0, 8.9)	5.9 (5.2, 6.6)
MFD				1.34 (1.20, 1.49) (arc 1)
				1.15 (1.02, 1.29) (arc 2)
				1.25 (1.11, 1.41) (arc 3)

	SL distance abnormality detection	SL angle abnormality detection	CL angle abnormality detection	Carpal arc interruption detection
Sensitivity (%)
Value	64 (51, 70)	84 (76, 91)	70 (50, 90)	83 (74, 91)
Proportion	49/77	82/98	14/20	58/70
Specificity (%)
Value	98 (93, 98)	74 (65, 83)	95 (92, 98)	64 (56, 71)
Proportion	178/181	69/93	163/171	94/147
AUC				0.80 (0.73, 0.87)

Note. — A total of 258 SL distances, 189 SL angles, 191 CL angles, and 217 sets of labelled carpal arcs were labelled in 217 studies from 217 patients by two experienced musculoskeletal (MSK) radiologists (see dataset 2 in paper). 95% confidence intervals are reported in parentheses. Carpals arcs 1, 2, and 3 are respectively the proximal, middle, and distal arc. AUC = area under the receiver operating characteristic curve, CL = capitolunate, MAE = mean absolute error, MDF = mean Fréchet distance, mm = millimeter, SL = scapholunate.

Uses and Directions

Benefits: The automated measurements and detections of signs of carpal instability may help to raise awareness of carpal instability and could potentially improve detections of carpal arc interruptions.

Target population and use case: Patients clinically suspected of having wrist trauma often undergo conventional radiography. The algorithm preprocesses incoming hand and wrist radiographs in the picture archiving and communication system, and may be able to assist clinicians by acting either as a concurrent or second reader, or as a triage tool that helps prioritizing worklists.

General use: This algorithm is intended to be used by clinicians for measuring and detecting signs of carpal instability in conventional radiographs of the hand or wrist. It is not a diagnostic for carpal instability and is not meant to guide or drive clinical care. It should only be used to complement other pieces of patient information related to carpal instability as well as a physical evaluation to determine the need for carpal instability treatment.

Appropriate decision support: The algorithm generates a PDF report of automated measurements and subsequent detections of signs of carpal instability. The clinician examines the automated measurements and detections along with other clinical information to determine whether signs of carpal instability are present and whether follow-up examination should be conducted.

Before using this algorithm : Test the algorithm retrospectively on a diagnostic cohort that reflects the target population that the algorithm will be used upon to confirm validity of the algorithm within a local setting.

Safety and efficacy evaluation: Our study shows that AI driven measurements and detections of radiological signs of carpal instability are feasible at a clinically acceptable level. The mean absolute errors (MAEs) in measuring SL distances, SL angles, and CL angles were 0.65 mm, 7.9 degrees, and 5.9 degrees, respectively. The sensitivity and specificity for detecting arc interruptions were 83% and 64%, respectively. In an observer study, the performance of the AI system was compared to that of five clinicians with different specialties: a musculoskeletal (MSK) radiologist, non-MSK radiologist, emergency doctor, hand surgeon, and junior doctor on general surgery. The observer study showed that the AI system had a comparable accuracy to most clinicians in measuring SL distances and SL angles (equal or higher [p < .05] than respectively four and five clinicians). It had a lower accuracy in measuring the CL angle than most clinicians (p < .05 for three clinicians), but the difference was slight (MAE, 6.0 vs. 4.0 degrees [clinician average]). The AI system had a higher sensitivity than three clinicians at equal specificity in detecting carpal arc interruptions (sensitivity/specificity, 73%/91% vs. 45%/95% [clinician average], p < .05 [sensitivity] and p ≥ .05 [specificity]). Future research should validate the AI system in an observer study with more clinicians per profession and investigate its potential impact on patient outcomes in a concurrent reading setting.

Warnings

Risks: Even if used appropriately, clinicians using this algorithm can overlook signs of carpal instability. Delays in diagnosis and treatment of carpal instability can lead to progressive limitation of movement, and eventually to degenerative arthritis, chronic pain, and functional loss.

Inappropriate settings: This algorithm is not suited for analysing radiographs in which the (relevant) carpal bones are incompletely depicted, obstructed by casts or implants, excessively damaged or malformed, (partially) resected, non or partially ossified and developed (children). Do not use the algorithm when any of these circumstances apply. Furthermore, only use radiographs in which only one hand is depicted and the hand is not depicted upside down (i.e. fingers should be pointing upwards).

Clinical rationale: The algorithm automatically measures and detects signs of carpal instability. These outputs are only interpretable as long as the input radiograph is similar to the radiographs used for developing the algorithm. Clinical end users are expected to place the algorithm output in context with other clinical information to determine whether follow-up examination should be conducted (with wrist arthroscopy or alternatively with MRI and CT arthrography).

Inappropriate decision support: This algorithm may not be accurate outside of the target population (primarily adult patients, children must be older than six years and have sufficiently ossified and developed wrist bones). This algorithm is not a diagnostic and is not designed to guide clinical diagnosis and treatment for carpal instability.

Generalizability: This algorithm was evaluated within the local setting of the Radboud University Medical Center, Jeroen Bosch Hospital, and Hospital Gelderse Vallei. Do not use this algorithm in an external setting without further evaluation.

Discontinue use if: Clinical staff raise concerns about the utility of the algorithm for the indicated use case or large, systematic changes occur at the data level that necessitates re-training of the algorithm.

Common Error Messages

"Pixel spacing of (1, 1) encountered, pixel spacing information could be wrong or missing, please make sure that your image meets the input requirements (see algorithm information page).": The uploaded image most likely does not contain valid pixel spacing information. Accurate pixel spacing information is required to apply scale normalization before segmentation.

"Uneven pixel spacing encountered, image scaling might not be accurate": The uploaded image has a different pixel spacing in the x and y direction. The algorithm uses the pixel spacing information to rescale the image in order to normalize the bone sizes before segmentation. Provided that the pixel spacing information is inaccurate, the scaling operation may result into an unnatural looking image.

"The [bone] could not be segmented.": The automated segmentation of the given bone failed and no mask was returned by the CNN.

"The mask of [bone] is relatively small or short, which possibly indicates (1) segmentation failure, (2) abnormal bone dimensions, (3) inaccurate pixel spacing.": The height (major axis) or width (minor axis) of the given bone are smaller than observed in the development data of the algorithm (outside 1-99th percentile). Consequently, the results of the algorithm may not be accurate.

"The mask of the [bone] is relatively wide or long, which possibly indicates (1) segmentation failure, (2) abnormal bone dimensions, (3) inaccurate pixel spacing. The default region-of-interest patch size will be temporarily increased to fully enclose the mask if necessary, but the algorithm may not produce optimal results.": The height (major axis) or width (minor axis) of the given bone are larger than observed in the development data of the algorithm (outside 1-99th percentile). As already indicated by the message, this means that the results of the algorithm may not be accurate.

"The scaphoid mask is missing and therefore the laterality could not be determined.": By default, the algorithm automatically determines the laterality of hand (left or right hand) by analysing the scaphoid in the image instead of using the metadata of the DICOM or MHA file, as this information is not always available in the metadata. If a left hand is depicted, then the image is temporarily horizontally flipped to standardize the input for the analysis. If the scaphoid cannot be automatically segmented and localized, then this preprocessing step won't be performed. This means that the results of algorithm may not be accurate.

"The [bone] mask is missing and the [measurement] could not be conducted.": The algorithm failed to segment and localize the bones required for performing the measurement.

"Not all masks of the bones constituting the carpal arcs are available and the carpal arcs could not be generated.": The algorithm failed to segment and localize one or more of the bones constituting the carpal arcs and therefore it cannot draw the carpal arcs.

"No generated carpal arcs are available and the interruption detection could not be conducted.": This error message is triggered when the carpal arcs have not been drawn (see previous error message) and therefore cannot be analysed.

"The [measurement] could not be conducted. Please check whether the [bones] are properly depicted.": This error message is triggered when the measurement could not be performed due to an unforeseen error. The software developer should further investigate this in order to pinpoint the problem.

Information on this algorithm has been provided by the Algorithm Editors, following the Model Facts labels guidelines from Sendak, M.P., Gao, M., Brajer, N. et al. Presenting machine learning model information to clinical end users with model facts labels. npj Digit. Med. 3, 41 (2020). 10.1038/s41746-020-0253-3