REQ_013 The user receives the pixel coordinates of possible ICD categories

Category

Minor

Source

• Dr. Miguel Sánchez Viera, head dermatologist at IDEI
• Alfonso Medela JD-005

INTERNAL SYSTEM INPUTS AND OUTPUTSDESIGNINTERFACES USER

Activities generated

• MDS-461
• Following our GP-001 Documents and records control we validate the documentation created as a result of this requirement implementation and compile the evidence within the R-TF-001-006 IFU and label validation_2023_001

Related risks

• 9. Image artefacts/resolution: the medical device receives an input that does not have sufficient quality in a way that affects it performance
• 30. The user is unable to provide adequate lighting conditions
• 37. Inaccurate training data: image datasets used in the development of the device are not properly labeled.
• 38. Biased or incomplete training data: image datasets used in the development of the device are not properly selected
• 54. The device inputs images that do not represent skin structure
• 61. Stagnation of model performance: the AI/ML models of the device do not benefit from the potential improvement in performance that comes from re-training
• 62. Degradation of model performance: automatic re-training of models decreases the performance of the device

User Requirement, Software Requirement Specification and Design Requirement

In order to assess the correct lesions the first step consists on identifiying and selecting those lesions. The requirements deriving from this need are:

• User Requirement 1.1: Users should be able to clearly see the location of the lesions in the submitted image.
• Software Requirement 1.2: Deploy ICD category detector to find the coordinates of the lesions.
• Design Requirement 1.3: Structure device responses such that the lesion coordinates are clearly communicated through key-value pairs in compliance with medical data standards.

Description

A photo taken with a smartphone, also referred to as a clinical image, often captures multiple skin conditions within the same image. For instance, individuals with fair skin typically have between 10 to 40 common moles (ICD category 2F20 - Benign cutaneous melanocytic neoplasms). When photographing a specific area of the body, it's common to unintentionally capture more than the intended category. This can occur due to the presence of additional lesions adjacent to the primary one, resulting in pixels that represent multiple categories. For example, consider actinic keratosis on a bald scalp, where the keratosis occupies the area affected by alopecia, representing two distinct categories.

Localized conditions, particularly pigmented lesions like benign cutaneous melanocytic neoplasms, are prevalent. Healthcare professionals often prioritize examining these lesions, and providing a visual guide can greatly assist dermatologists in identifying and highlighting them.

To facilitate this, we will develop a detector capable of identifying various categories of localized conditions, as outlined in the following sections, using bounding boxes (spatial coordinates) as a visual aid.

Success metrics

In light of the task's nature, our assessment will be grounded in object detection metrics, notably mean Average Precision (mAP). According to Rehman, H-u et al. [1], mAP values ranging from 0.69 to 0.96 have been reported for the ISIC 2017, 2018, and PH2 datasets, comprising dermatoscopic images of pigmented lesions. This is pertinent as dermatoscopic imaging typically features a solitary pigmented lesion centrally, rendering the task comparatively less complex than clinical imaging, which often presents multiple lesions of varying sizes. In contrast, alternative research [2] has demonstrated mAP results spanning from 0.37 to 0.77, utilizing the ISIC dataset with custom adaptations and labels, complicating direct comparisons between studies. Furthermore, a temporal gap of 3 years since the publication of this study renders it somewhat outdated in the context of evolving deep learning models. Conversely, a related study conducted on the ISIC 2018 dataset achieved an mAP of 0.96.

Given these findings, primarily derived from dermatoscopic imaging, and recognizing the added complexity of clinical imaging, we propose setting a minimum mAP threshold of 0.6. This benchmark also resonates with prior results obtained from tests such as TEST_002, wherein quantifiable data on the count of clinical signs was provided. Notably, for tasks like hive detection, which can be assumed to be more intricate, mAP values exceeding 0.6 were attained.

Goal	Metric	Target
An algorithm identifies pixel coordinates of localised ICD categories	mAP	>.6

References

1. Rehman, H.u., Nida, N., Shah, S.A. et al. Automatic melanoma detection and segmentation in dermoscopy images using deep RetinaNet and conditional random fields. Multimed Tools Appl 81, 25765–25785 (2022). https://doi.org/10.1007/s11042-022-12460-8
2. Nie, Y., Sommella, P., O'Nils, M., Liguori, C., & Lundgren, J. (2019). Automatic Detection of Melanoma with Yolo Deep Convolutional Neural Networks. 2019 E-Health and Bioengineering Conference (EHB). Presented at the IEEE International Conference on e-Health and Bioengineering 2019, Romania, 21-23 November 2019. https://doi.org/10.1109/EHB47216.2019.8970033
3. Taqi, A.M., Al-Azzo, F., Awad, A., & Milanova, M.G. (2019). Skin Lesion Detection by Android Camera based on SSD- Mo-bilenet and TensorFlow Object Detection API.

Design change management

This section only applies if this requirement is a change of the product.

According to MDR 120, this change is considered .

Rationale to determine impact according to MDR 120, as defined in GP-023

• Does this change modify the intended use?
  • If yes, this is considered Significant according to MDR 120
  • If not, does this change come from CAPA?
    • If yes, this is considered a Non-significant change
    • If not, does this change modify the design or performance specifications?
      • If yes, this is considered Significant according to MDR 120
      • If not, does this change modify the software?
        • If yes, this is considered Significant according to MDR 120
        • If not, this is considered a Non-significant change

Assessment of impact of this change

This section only applies if this requirement is a change of the product.

When reviewing the changes, we must evaluate the impact of the changes over the product, over inputs and outputs of risk management and over the processes of product delivery.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Tester: JD-017, JD-009, JD-005, JD-004
• Approver: JD-003