Our study included a medical student (reader 1) trained in the mvdHSharp lecture and the eye-tracking lecture who read radiographs via eye tracking and an experienced radiologist who read only the mvdHSharp (reader 2). Fig 1 shows the screen shown by readers using the eye-tracking software.

The materials used in this study consisted of 440 joints (a set of 20 radiographs, including both hands and feet) of 10 patients with rheumatoid arthritis (ACR/EULAR 2010) chosen by a rheumatologist expert (AS) to represent the range of disease severity with early inflammatory rheumatism from the local cohort (from normal to severe damage). Using the mvdH, the score of erosions per hand was between 0 and 43, the number of erosions per foot was between 0 and 23, the score of Joint Space Narrowing (JSN) per hand was between 0 and 27, and the number of JSN per foot was between 0 and 19.

One reading was performed by Reader 2 according to the mvdH Sharp score for both erosions and JSN. This scoring lecture was considered the gold standard for our study to establish a reliable and accurate reference point for comparison. In total, each image was linked with a JSN Sharp score and erosion Sharp score. The joints involved in the scores of erosions and JSN are shown in Fig 2.

The eye tracking reader (reader 1) trained to read radiographs with a metronome to learn to respect 2 seconds for one point of the mvdH Sharp score. Reader 1 was seated in front of a computer screen to interpret each radiograph using the eye-tracking score. Radiographs were read 4 times by an eye-tracking reader (2 times for the JSN eye-tracking score and 2 times for the erosion eye-tracking score).

For reader 1’s reading under metronome, we obtained an eye tracking score: a 1-point Sharp score was assigned every 2 seconds of fixations duration by the reader. Fixation duration is the time when the reader focuses on a anomaly without looking elsewhere. By dividing the fixation values by a pair of milliseconds by 1000 and then by two (with the reader fixed at 2 seconds for each erosion) and then subtracting the value from 0.5 (to eliminate the background noise of the duration of reading a normal joint), we obtained a reading in which 1 point was read by eye-tracking = 1 point for the mvdH Sharp score: 1ptsmvdH=1ptsEyetrackingmvdH=fixationvaluesms10002−0.5

An eye tracker consists of cameras, projectors and algorithms. The projectors create a pattern of near infra-red light on the eyes. The camera take images of the user’s eyes. Mathematical algorithms calculate the eyes’ positions and gaze point. Tobii Pro Lab Eye-tracking software, a validated, high-precision software package, was used to record gaze data at a sampling rate of 120 Hz and analyse the eye movements of the different readers. With this eye-tracking software, data values can be represented as colours in a map, diagram or image, also known as a heatmap. This software gives a good overview of what area of interest the reader is paying attention to by dividing the image into a grid within each pixel and showing the intensity of values in the recordings.

440 joints were analysed (16 joints on each hand for erosions, 15 joints for JSN on the hands, and 6 joints on each foot). Eye tracking readings incorrectly captured by the eye tracker were considered no analysable. The data were analysed descriptively by calculating the fixation duration in milliseconds on each joint based on the diameter of the Gaze Plot’s circles created by the software. Gaze plots also show the location and order of the fixation duration at each point. The fixation time data are reported in a table that compares the recorded data with the initial Sharp score, which was established as the gold standard. The number of joints with at least one instance of erosion and JSN was analysed, and the local fixation time was converted to seconds. All the statistical analyses were performed using SPSS software version 25.0 (IBM, Armonk, NY) and were based on average values. We used the intraclass correlation coefficient (ICC) to determine intra-observer eye-tracking reliability (from 0.00 to 10.00). An ICC > 0.8 was considered excellent. For inter observer reliability, we performed an analysis by class (yes versus no and value from 0 to 10) using Cohen’s kappa.

All 20 radiographs were read by both readers. A total of 440 joints were selected for interpretation. Finally, 416 and 396 joints could be interpreted when read by eye tracking for erosive involvement and JSN, respectively, because eye-tracking technology cannot measure the time spent when two pathological joints are too close together.

On the 20 X-rays of rheumatoid arthritis patients analysed by eye tracking, the mean sharp score for erosions per joint was 0.52 +- 1.13, that for JSN per joint was 0.81 +- 1.36, and that for JSN and erosion was 2.27 +- 2.62 seconds and 1.78 +- 2.30 seconds, respectively. An analysis of the scores obtained by the eye-tracking method was performed jointly. The erosion sharp score was more important for the feet, notably for metatarsophalangeal (MTP) 3, MTP4 and MTP5, for which the mean length of reading was between 0.80 and 3.15 seconds. JSN was described in a diffuse way and was particularly high for metacarpophalangeal (MCP) 2, MCP3, MTP3, and MTP4, with a mean time of reading between 1.15 and 4.06 seconds. All corresponding results are presented in Table 1.

Table 2 shows the agreement between eye tracking mvdH Sharp score and classical mvdH Sharp score for both erosion (upper) and JSN (lower). The intraclass agreement of the eye tracking method was excellent at 0.90 (95% CI 0.88–0.92) for both erosions and JSN. The agreement between eye tracking mvdH Sharp score and classical mvdH Sharp score by class yes (at least one erosion or JSN) versus no (no erosion or no JSN) was excellent for both erosions (kappa 0.82; 95% CI: 0.79–0.0.85) and JSN (kappa: 0.68; 95% CI: 0.65–0.0.71). The agreement between eye tracking mvdH Sharp score and classical mvdH Sharp score by class (0 to 10) remained excellent for erosions (kappa 0.82; 95% CI: 0.79–0.0.85) and good for JSN (kappa: 0.68; 95% CI: 0.65–0.71).

Heatmaps were generated to reveal which subregions tended to contribute to severe lesions. Both heatmaps and glaze plot figures of each reading show clear segmentation and fixation duration. Joints with a “hot” colour had the highest Sharp score and fixation duration, and conversely, “cold” (green) colours represented a short fixation time and a low severity of joint damage. According to the statistical analysis, the colour of the joint was strongly correlated with the severity of the sharp spot, which was preliminarily established via the gold standard. Green pixels for Sharp lower than 1, yellow for Sharp equal to 1, and orange to red for Sharp above or equal to 2. Fig 3 shows an example of a foot and hand erosion of one patient with a modified van der Heijde Sharp score and eye-tracking score (E-T) in milliseconds.

The aim of this study was to develop a technique using eye-tracking software to analyse radiographs of rheumatoid arthritis patients, segment the joints and establish a score of lesions with the fixation time score and pixels at the same time. We found that his new method of quantifying the mvdH score based on readings via eye-tracking technology is highly correlated with the classical mvdH Sharp score determined by an experienced radiologist. This method allowed us to segment and evaluate the joint lesions one time because of the fixation duration.

Our findings suggest that eye tracking can provide valuable insights into the processes of radiographic interpretation. These findings highlight the importance of sharp score training to best evaluate joints, particularly for the accurate diagnosis and treatment of rheumatologic disorders. This approach allowed us to identify erosions and narrow more comprehensively with a score translation with colours on heatmaps. It would be a more intelligible and playful technique to illustrate the severity of lesions to students, professionals or patients. This method is also less time-consuming; instead of a two-phase lecture with score analysis and segmentation, this technique makes it possible to merge the 2 phases into one, save time and reduce the number of professionals involved.

In the literature, deep learning and convolutional neural networks are the most commonly used artificial intelligence methods for analysing radiographic images. Few studies using IAs have been described in the literature; these studies used the Sharp/van der Heijde SvdH score, erosions score, JSN, or both, with variable sensitivity and specificity [24,25,26]. Recently, Wang et al. described an artificial intelligence study of X-ray images scored by the Sharp/van der Heijde score index trained with You Only Look Once version 4 (YOLOv4), an object detection model trained to look at an image and search for a subset of object classes [24]. By understanding the similarities between the variations in the sharp score and the heatmap created by the fixation duration, AI could directly create a heatmap score for an easier and more comprehensive interpretation of radiographic findings. Our study also revealed interesting insights regarding the impact of reading time on radiographic interpretation, shedding light on its relationship with diagnostic accuracy. The experimental data were subsequently analysed with artificial intelligence software before training to read the rheumatoid arthritis radiographs of several cohorts. The correlation between Sharp and the colour score of heatmaps was determined to analyse future radio files with a colour code more explicit than the van der Heijde modified Sharp score. This approach will be very intuitive for students or professionals who are less accustomed to the radiology discipline. One limitation of our study is the definition of the gold standard, which was based on the means of reading by one experienced reader.

This approach could introduce bias, as the gold standard may not necessarily reflect the true or objective interpretation of the radiographs; rather, the differences in reading, intraobservers and interobserver agreements between eye-tracking and sharpeners are similar to the differences that could occur between 2 radiologists interpreting images.

Additionally, the sample size was relatively small, which could limit the generalizability of the findings to larger populations. Our study focused only on a simplified radiographic score, which may not fully capture the complexity of radiographic interpretation in clinical practice. Furthermore, the steps of reasoning by an AI must be clearly described to clinicians, but often, these algorithms include "black boxes", complex information processing steps that make very little sense to a human. However, eye tracking technology is still relatively new, and limitations exist regarding its use. For example, eye tracking can be affected by factors such as lighting, screen resolution, participant comfort, and glasses worn, which may affect the quality of the data collected. With respect to the efficiency of the software, there may be limitations in wrist fixation problems. Since the joint bonds are very close together, the fixing zones are regularly confused. Therefore, while eye tracking is a promising tool for studying radiographic interpretation, it should be used in conjunction with other methods to provide a more comprehensive understanding of the cognitive processes involved.

Another limitation of our study is that we conducted it in only one center, with one eye tracking reader and one experimented radiologist as gold standard. Nevertheless, the radiologist (FG) is highly experimented, with a very high agreement with other specialist of mvdH Sharp score, and the eye tracking reader was trained with previous reading.

In conclusion, our study demonstrated the usefulness of eye-tracking technology for evaluating severity through radiographic interpretation strategies, segmenting joints and establishing a score for lesions based on both the fixation time score and the number of pixels. It would be a more intelligible and playful technique to illustrate the severity of lesions to students, professionals or patients. Our findings may have important implications for improving interdisciplinary communication and collaboration in the diagnosis and treatment of rheumatological disorders. In the future, this technique could also be developed for other disciplines using scores in their diagnostics.