URS-001 The user receives quantifiable data on the intensity of clinical signs

Source

• Tania Menendez, Digital Manager at Ribera Salud
• Dr. Marta Ruano, dermatologist at Hospital de Torrejón
• Dr. Ramon Grimalt, dermatologist at Grimalt Dermatología

Causes failure modes

• The AI models misinterpret the clinical signs in the images or miscalculate the intensity of clinical signs, leading to inaccurate data being presented.
• Poor quality or improperly taken images might lead to incorrect analysis and quantification.
• Issues with integrating scoring systems like SCORAD, PASI, or SALT could lead to incorrect severity quantification.
• Delays or timeouts in processing and delivering the clinical data could affect timely access to information.

Related risks

3. Misrepresentation of magnitude returned by the device
4. Misinterpretation of data returned by the device
5. Incorrect clinical information: the care provider receives into their system data that is erroneous
6. Incorrect diagnosis or follow up: the medical device outputs a wrong result to the HCP
7. Incorrect results shown to patient
8. Sensitivity to image variability: analysis of the same lesion with images taken with deviations in lightning or orientation generates significantly different results
9. Inaccurate training data: image datasets used in the development of the device are not properly labeled
10. Biased or Incomplete Training Data: image datasets used in the development of the device are not properly selected
11. Lack of efficacy or clinical utility
12. Stagnation of model performance: the AI/ML models of the device do not benefit from the potential improvement in performance that comes from re-training
13. Degradation of model performance: automatic re-training of models decreases the performance of the device

Description

Skin conditions are distinguished by a spectrum of observable manifestations and associated symptoms. These visual indicators, including but not limited to erythema, are pervasive across a diverse range of dermatological pathologies; however, their presence is not uniformly evident in all instances of a given pathology. These visual manifestations often coincide with accompanying symptoms such as fever, pruritus, and more.

The quantification of the severity of these visual indicators constitutes a crucial endeavor in comprehending the extent of a particular pathology. Typically, this quantification process employs a rating scale, often ranging from 0 to 3 or 0 to 4, for ease of human evaluation. While scales extending from 0 to 10 are infrequently employed for visual indicators, they may find utility in assessing symptoms such as pruritus.

It is imperative to acknowledge that gauging the intensity of these visual indicators is inherently subjective, contingent upon the observer's ability to discern how the visual signs manifest on the skin. Consequently, greater expertise on the part of the observer typically yields more accurate results, albeit retaining a subjective element. This subjectivity gives rise to inter-observer variability, leading to variances of approximately 10-20% in the quantification of the intensity of visual signs like erythema, induration, swelling, and others when assessed by different specialists. This inter-observer variability introduces a degree of unreliability in the severity assessment process, necessitating the observation of a notable improvement or deterioration to ensure the validity of the quantification.

Addressing this imperative, there arises the need for the development of a suite of algorithms designed to automate the quantification of the following visual indicators:

• Erythema
• Induration
• Desquamation
• Edema
• Oozing
• Excoriation
• Lichenification
• Dryness
• Pustulation
• Exudation
• Edges
• Affected tissues
• Facial palsy

The visual signs mentioned above are among the most frequently encountered indicators used in the assessment of dermatosis and facial nerve injury.

Automating the quantification of visual signs entails a structured approach, typically divided into two principal phases, mirroring the common workflow of data science projects: data annotation and algorithm development.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001