R-TF-007-005 Post-Market Clinical Follow-up (PMCF) report

Details

• Corresponding PMCF plan number and date: 001 dated 2024-11-27 (versions and date are available in the Git repository)
• PMCF report number: 001
• PMCF report date and version: registered in the Git repository

Manufacturer contact details

	Manufacturer data
Legal manufacturer name	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRN	ES-MF-000025345
Person responsible for regulatory compliance	Alfonso Medela, María Diez, Giulia Foglia
E-mail	office@legit.health
Phone	+34 638127476
Trademark	Legit.Health

Medical device characterization

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.0.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product (True/False)	FALSE
Novel related clinical procedure (True/False)	FALSE
SRN	ES-MF-000025345

Intended use or purpose

Intended use

The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing:

• quantification of intensity, count, extent of visible clinical signs
• interpretative distribution representation of possible International Classification of Diseases (ICD) categories.

Quantification of intensity, count and extent of visible clinical signs

The device provides quantifiable data on the intensity, count and extent of clinical signs such as erythema, desquamation, and induration, among others; including, but not limited to:

• erythema,
• desquamation,
• induration,
• crusting,
• dryness,
• oedema,
• oozing,
• excoriation,
• swelling,
• lichenification,
• exudation,
• depth,
• edges,
• undermining,
• pustulation,
• hair loss,
• type of necrotic tissue,
• amount of necrotic tissue,
• type of exudate,
• peripheral tissue edema,
• peripheral tissue induration,
• granulation tissue,
• epithelialization,
• nodule count,
• papule count,
• pustule count,
• cyst count,
• comedone count,
• abscess count,
• draining tunnel count,
• lesion count

Image-based recognition of visible ICD categories

The device is intended to provide an interpretative distribution representation of possible International Classification of Diseases (ICD) categories that might be represented in the pixels content of the image.

Device description

The device is computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended medical indication

The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails) and associated mucous membranes (conjunctival, oral and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).

Intended patient population

The device is intended for use on images of skin from patients presenting visible skin structure abnormalities, across all age groups, skin types, and demographics.

Intended user

The medical device is intended for use by healthcare providers to aid in the assessment of skin structures.

User qualification and competencies

In this section we specificy the specific qualifications and competencies needed for users of the device, to properly use the device, provided that they already belong to their professional category. In other words, when describing the qualifications of HCPs, it is assumed that healthcare professionals (HCPs) already have the qualifications and competencies native to their profession.

Healthcare professionals

No official qualifications are needes, but it is advisable if HCPs have some competencies:

• Knowledge on how to take images with smartphones.

IT professionals

IT professionals are responsible for the integration of the medical device into the healthcare organisation's system.

No specific official qualifications are needed, but it is advisable that IT professionals using the device have the following competencies:

• Basic knowledge of FHIR
• Understanding of the output of the device.

Use environment

The device is intended to be used in the setting of healthcare organisations and their IT departments, which commonly are situated inside hospitals or other clinical facilities.

The device is intended to be integrated into the healthcare organisation's system by IT professionals.

Operating principle

The device is computational medical tool leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures.

Body structures

The device is intended to use on the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails) and associated mucous membranes (conjunctival, oral and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).

In fact, the device is intended to use on visible skin structures. As such, it can only quantify clinical signs that are visible, and distribute the probabilities across ICD categories that are visible.

Variants and models

The device does not have any variants.

Expected lifetime

The expected operational lifetime of the device is established at 5 years, which is subject to regular software updates and the lifecycle of the integrated components and platforms. The lifetime will be increase in equivalent spans as the design and development continues and maintenance and re-design activities are carried out.

This timeline accounts for the expected evolution of the underlying operating systems and tools, the progression of medical device technology, and the necessary update cycles to maintain security and operability.

List of any accessories covered by this report

The device does not have any accessories.

The device's functionality can be leveraged through its API, which is compatible with any device that has an internet connection, from personal computers and servers to mobile phones, regardless of operating systems.

Explanation of any novel features

The device represents a significant enhancement of existing technology, developed to advance the current state-of-the-art in analyzing photographs of skin structures. It employs sophisticated artificial intelligence algorithms to process these images and extract data with clinical significance.

We detail the novel features of the device within the Description and specifications document.

Post-Market Clinical Follow-up (PMCF) activities

Activity 1: Clinical literature review

This activity compiles and evaluates state-of-the-art publications on image diagnostic and severity measure methods. The overall objective of this strategy is to identify, select and collect the relevant clinical literature to determine if the device is safe for its intended use and if there is any emergent risk that we must consider.

The clinical literature review was performed according to the R-TF-015-001 Clinical evaluation plan and the results were collected and discussed in the R-TF-015-003 Clinical evaluation report.

Identification of relevant bibliographic data

A comprehensive search of selected scientific literature databases (PubMed (MEDLINE)) was carried out on 2024-10-29.

The literature searches were performed according to the criteria defined in the Clinical Evaluation Plan. The algorithms used were the following:

• PubMed: ("skin cancer" OR "epidermis" OR "chronic skin conditions" OR "skin conditions" OR "inflammatory skin diseases" OR "malignant skin lesions" OR "melanoma" OR "acne" OR "psoriasis" OR "dermatofibroma" OR "dermatosis") AND ("Legit.Health" OR "software" OR "digital imag*") AND ("SkinVision" OR "artificial intelligence" OR "machine learning" OR "computer vision" OR "smartphone") AND ("performanc*" OR "safe*" OR "clinical")

The searches were performed for a period of 10 years (2014-10-29 / 2024-10-29).

The search process is summarised below:

• Results of the search in PubMed: 61 articles.

A bibliographic screening of articles that are not related to the medical condition and the device under evaluation (software devices for the assessment of skin structures) was performed and 49 articles were discarded. Articles for which the full content is not accessible, which are not in English and which are not related to the product under evaluation or the medical condition were discarded.

The remaining 12 articles were appraised according to the appraisal criteria defined in R-TF-015-001 Clinical evaluation plan and 6 articles, whose score was equal or lower than 10 points were discarded.

The literature review is summarized below:

• Title and abstract review: 61 articles
• Screening: 49 articles
• Appraised articles: 12 articles
• Exclusion based on appraisal criteria defined in R-TF-015-001 Clinical evaluation plan: 6 articles
• Total studies included in the review: 6 articles.

Evaluation of clinical data

To determine the value of the data identified in the literature, the results obtained from the search in PubMed were screened on the basis of the title and the information contained in the abstract.

The assessment of the relevant data has been carried out according to the assessment plan described in the CEP (R-TF-015-001 Clinical Evaluation Plan).

The 49 excluded articles were discarded mainly because their content is not related to the device under evaluation or the medical conditions according to the intended use of Legit.Health Plus. The remaining 12 articles were then evaluated, and 6 articles were finally selected for analysis.

Product Specific Data - Summary and Evaluation of Clinical Data

The 12 articles selected for their appraisal were evaluated and scored according to the appraisal criteria defined in R-TF-015-001 Clinical evaluation plan. Finally, 6 articles were selected and analyzed in the following section.

Analysis of clinical data

The following is a summary of the 6 articles selected for evaluation.

Article ID 1

• Title / Year: Implementation of artificial intelligence for the detection of cutaneous melanoma within a primary care setting: prevalence and types of skin cancer in outdoor enthusiasts (2023)
• Author: Miller, IJ
• DOI / PMID: 10.7717/peerj.15737
• Brief Summary of the article: The article discusses the use of artificial intelligence (AI) and machine learning, particularly convolutional neural networks (CNN), as tools in skin cancer screening. AI technology, including teledermatology and high-definition dermatoscopy, is increasingly applied in primary care for non-invasive, early skin cancer detection. AI analysis of individual skin lesions, using datasets of reference images, has shown high sensitivity in distinguishing melanomas from benign lesions, with some algorithms demonstrating sensitivity similar to that of trained dermatologists. For instance, the Fotofinder Moleanalyzer achieved sensitivity and specificity rates comparable to dermatologists, proving effective in identifying melanomas. Studies on CNN-based AI systems reveal that they can enhance dermatologists' diagnostic accuracy, with one example showing an increase from 60% to 75% in sensitivity and from 65% to 73% in specificity when AI-assisted. Another AI algorithm reached 100% sensitivity and a high receiver operating characteristic (91.8%) in detecting suspicious skin lesions from a large image set, outperforming dermatologists in some cases. However, the article notes a gap in real-world data, as most studies rely on datasets rather than clinical settings. The study's findings underscore the potential of AI-based imaging to aid clinical decision-making in identifying high-risk lesions, though limitations in real-world implementation remain.
• Risks Interpretation:
  • Data Limitations and Bias: Many AI models are trained on limited datasets that may not represent the full diversity of skin types, lesion types, or conditions encountered in clinical settings. This can lead to biases in AI assessments, potentially causing inaccuracies, particularly with underrepresented skin types.
  • False Positives and False Negatives: Although AI can be highly sensitive, it may yield false-positive results (incorrectly identifying benign lesions as suspicious) or false negatives (failing to detect true melanomas). These misclassifications could lead to unnecessary anxiety, unwarranted biopsies, or delayed diagnosis.
  • Dependency on Image Quality: The accuracy of AI assessments is significantly affected by the quality and consistency of images. Poor lighting, resolution, and angle may compromise results, especially in teledermatology settings, leading to potential misdiagnoses.
  • Over-Reliance on AI by Non-Specialists: While AI tools are increasingly accessible to primary care providers, there is a risk that less experienced clinicians may over-rely on AI results. This could reduce diagnostic vigilance, with clinicians relying on the tool without sufficient expertise to verify its findings.
  • Challenges in Real-World Implementation: Many AI models show promising results in controlled settings but may struggle with performance and reliability when applied in diverse real-world environments. The lack of clinical validation data for AI models complicates their integration into healthcare workflows, which could introduce risks if not thoroughly vetted.
  • Liability and Accountability: The article also notes the unclear regulatory and legal frameworks surrounding AI, raising questions about accountability in cases of misdiagnosis. If an AI tool contributes to an incorrect diagnosis or treatment decision, responsibility is often ambiguous, which complicates both clinical practice and patient safety.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing: quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus, and nails) and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.

Article ID 31

• Title / Year: AI outperformed every dermatologist in dermoscopic melanoma diagnosis, using an optimized deep-CNN architecture with custom mini-batch logic and loss function (2021)
• Author: Pham, TC
• DOI / PMID: 10.1038/s41598-021-96707-8
• Brief Summary of the article: This study addresses the use of AI, specifically deep convolutional neural networks (CNNs), to detect melanoma from skin lesion images. With melanoma being a highly deadly skin cancer, early detection is crucial, and recent advancements in computer vision within AI have shown potential for automating its diagnosis through image analysis. The study identifies challenges, such as data limitations and class imbalances, with melanoma images being much fewer than non-melanoma ones in datasets. This imbalance impacts the model's sensitivity to melanoma (minority class) versus nevus (majority class). To address this, the researchers propose a customized AI model with a custom loss function (CLF) and custom mini-batch logic that specifically adjusts training to better detect melanoma in imbalanced datasets. The model uses real-time image augmentation and specialized fully connected layers optimized for binary melanoma classification, achieving a high AUC, sensitivity (SEN), and specificity (SPE). Among tested CNN architectures, DenseNet169 performs best due to its dense connections, achieving superior sensitivity and specificity compared to dermatologists on benchmark datasets. The study’s proposed techniques could extend to other medical image classifications, demonstrating the potential for customized AI to enhance diagnostic accuracy in clinical applications.
• Risks Interpretation:
  • Data Imbalance Risks: With fewer melanoma images compared to non-melanoma in available datasets, there is a significant risk of the model underperforming in detecting melanoma cases. This imbalance may lead to reduced sensitivity (SEN) toward melanoma, meaning the model could miss critical cases, potentially resulting in delayed diagnosis or misdiagnosis of patients.
  • Overfitting and Generalization Issues: The limited number of annotated melanoma images raises concerns about overfitting, where the model performs well on training data but poorly on new data. This could result in unreliable predictions when the model encounters real-world, varied patient images, limiting its clinical utility.
  • Misclassification Costs: The cost of misclassifying melanoma as a non-melanoma condition is much higher than vice versa, as failing to detect melanoma early can be life-threatening. However, the article notes that it’s challenging to quantify these costs in training the AI model, increasing the risk of inappropriate weighting and potential diagnostic errors.
  • Model Dependence on Dataset Quality: The performance of AI models heavily relies on high-quality, representative datasets. If the datasets lack diversity or contain inaccuracies, the model’s predictions may be unreliable, posing risks to patient safety.
  • Technical Complexity of Solutions: Custom loss functions and mini-batch logic are complex to implement and may be less adaptable across other medical image classification tasks. This could limit scalability and complicate model validation, increasing the risk that the solution may not be broadly applicable or easy to update.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails), and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature, and the subcutaneous tissue (subcutis).
  • Intended User: The software as a medical device is specifically intended for use by IT professionals (ITPs) working within healthcare organizations who are responsible for integrating the software into the healthcare system infrastructure.
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.
  • Use Environment: The device is intended to be used in the setting of healthcare organizations and their IT departments, which commonly are situated inside hospitals or other clinical facilities.
  • Operating Principle: The device is a computational medical tool leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures.
  • Part of the Body: The device is intended to analyze images of the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails), and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature, and the subcutaneous tissue (subcutis). Due to the nature of the device (stand-alone software), it doesn't come into contact with tissue or bodily fluids.

Article ID 34

• Title / Year: Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities (2020)
• Author: Goyal, M.
• DOI / PMID: 10.1016/j.compbiomed.2020.104065
• Brief Summary of the article: The use of artificial intelligence (AI), particularly convolutional neural networks (CNNs), is advancing in dermatology for assessing skin conditions, primarily focused on skin cancer detection. Traditional diagnostic practices rely on visual examination by dermatologists, often enhanced by dermoscopy and biopsy, but AI is poised to transform this process by enabling automated, computer-aided diagnostics (CAD) through medical imaging. Algorithms have shown promising results, frequently matching or surpassing clinicians in classifying lesions as malignant or benign based on large datasets of clinical, dermoscopic, and histopathology images. Esteva et al.'s deep learning model, for example, achieved performance on par with dermatologists for differentiating between melanoma and benign lesions. Multiple public datasets, such as the ISIC Archive, Interactive Atlas of Dermoscopy, and HAM10000, support these AI-driven advancements by providing annotated images for algorithm training. AI solutions apply to various imaging modalities like dermoscopy, clinical photos, and whole-slide pathology scanning, adapting to high-resolution images captured through devices ranging from DSLR cameras to mobile phones. Studies by Codella, Haenssle, and Brinker illustrate that ensemble and deep learning methods can perform comparably or even better than dermatologists in specific cases. Challenges persist, however, as AI algorithms depend on balanced, high-quality datasets for accuracy and face limitations in real-world settings due to patient-specific factors like skin type and lifestyle, which these models currently overlook. Further collaboration between AI and clinical fields is essential to refine these tools for consistent, accessible, and cost-effective skin cancer diagnostics.
• Risks Interpretation:
  • Data Quality and Diversity: AI algorithms require large, diverse, and balanced datasets to achieve high diagnostic accuracy. Limited or unbalanced data can result in overfitting, bias, or reduced performance on less-represented skin types, ethnicities, or lesion types, leading to potential misdiagnoses.
  • Intra- and Inter-Class Variability: Skin lesions can show significant intra-class similarity (e.g., similar benign lesions may appear different under varying conditions) and inter-class dissimilarity (e.g., two benign and malignant lesions may look alike), complicating accurate classification by AI models. This variability increases the risk of diagnostic errors if the algorithm fails to generalize across different cases.
  • Over-Reliance on Imaging Data Alone: Current AI models primarily rely on imaging data, excluding critical clinical information like patient history, ethnicity, lifestyle factors, and past treatments. This omission could lead to inaccuracies in real-world settings where a holistic view of the patient is necessary for accurate diagnosis.
  • Lack of Real-World Testing: Many AI models have shown high accuracy in controlled settings but have not been rigorously tested in actual clinical environments. Applying these algorithms without adequate validation in real-world settings poses a risk of misdiagnosis.
  • Dataset Limitations: Models trained on existing datasets may fail when exposed to skin conditions or lesion types not represented in the training data, potentially leading to errors when assessing atypical cases.
  • Misdiagnosis Risk: The technology's limitations in handling unseen cases and the risk of overfitting to specific datasets increase the likelihood of false positives or negatives, especially if the AI is applied to skin lesions beyond its trained dataset scope.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing: quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus, and nails) and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.
  • Operating Principle: The device is a computational medical tool leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures.

Article ID 35

• Title / Year: Diagnostic capacity of skin tumor artificial intelligence-assisted decision-making software in real-world clinical settings (2020)
• Author: Li, CX
• DOI / PMID: 10.1097/CM9.0000000000001002
• Brief Summary of the article: This study focuses on Youzhi AI, a software developed by Shanghai Maise Information Technology using deep learning algorithms to assist dermatologists in diagnosing skin tumors through image analysis. Trained on the extensive Chinese Skin Image Database, Youzhi AI employs a convolutional neural network architecture, incorporating a classification and segmentation system to identify skin tumor types. The software achieves a diagnostic accuracy of 91.2% for distinguishing between benign and malignant tumors, and 81.4% for identifying specific disease types, aligning with international standards in laboratory settings. In clinical trials, Youzhi AI’s diagnostic accuracy was compared to that of dermatologists on clinical and dermoscopic images of skin tumors. While it showed no significant advantage over dermatologists in detecting malignancies (BMA), Youzhi AI surpassed dermatologists in disease type classification (DTA) for dermoscopic images. However, real-world performance variances are noted, as AI algorithms can yield lower diagnostic accuracy outside controlled environments. This analysis highlights Youzhi AI's potential to improve diagnostic consistency and support dermatologists, particularly in regions where diagnostic capabilities may be limited.
• Risks Interpretation:
  • Diagnostic Variability: The article emphasizes that the diagnostic accuracy of AI systems like Youzhi AI may differ when tested in real-world conditions versus lab-controlled datasets. Performance metrics validated in research settings often drop when the AI encounters unfiltered clinical data, making diagnostic reliability a potential risk in diverse healthcare environments.
  • Image Quality Sensitivity: AI systems may underperform if input images are of low quality, poorly focused, or obscured by factors like exogenous pigments. This sensitivity to image quality could lead to diagnostic inaccuracies if the images do not meet certain technical requirements.
  • Interoperability with Different Databases: The software is primarily trained on the Chinese Skin Image Database (CSID), and its efficacy may be limited if applied to populations or conditions not adequately represented in the training dataset. Variability in performance across different populations and imaging technologies poses a risk of inconsistent diagnostic accuracy.
  • Model Generalizability: The AI’s dependency on a specific convolutional neural network (CNN) model (GoogLeNet Inception v4) trained under controlled conditions limits its adaptability to novel, untrained cases. Any updates or changes in clinical presentation may not be well-recognized by the software.
  • Human-Software Interaction: The article highlights the need for dermatologists to be trained in specific image processing techniques (e.g., cropping) to optimize AI analysis. This dependency on correct human input introduces a risk of human error impacting diagnostic outcomes, especially if instructions are not followed precisely.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing: quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus, and nails) and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.

Article ID 38

• Title / Year: Automated detection of nonmelanoma skin cancer using digital images: a systematic review (2019)
• Author: Marka, A.
• DOI / PMID: 10.1186/s12880-019-0307-7
• Brief Summary of the article: The article explores the use of machine learning (ML) and artificial intelligence (AI) for diagnosing nonmelanoma skin cancer (NMSC) through image-based analysis, primarily using digital photography and dermatoscopy. The study addresses challenges in traditional visual diagnosis, where benign lesions often mimic malignancies, leading to invasive biopsies. Digital image-based ML models offer an alternative for early and accurate detection of NMSC, especially basal cell carcinoma (BCC) and cutaneous squamous cell carcinoma (CSCC), which are visually identifiable by skilled dermatologists. Various ML techniques are reviewed, including artificial neural networks (ANNs), decision trees, random forests, logistic regression, and support vector machines (SVMs). In some studies, models focus on identifying specific dermoscopic features of NMSC, such as telangiectasia, pink blush, and vascular characteristics, while others use global color and texture analysis across the entire lesion. The article highlights that most studies utilize separate training and test datasets or cross-validation to ensure accuracy, though some were criticized for overlapping data or using biased image sets. Accuracy metrics, such as the area under the receiver operating characteristic (AUROC) curve, were common, with AUROC values ranging from 0.832 to perfect classification. While some models reached 100% accuracy in specific studies, the results varied based on sample sizes and image sets. Overall, the review demonstrates the potential of AI and ML in dermatologic practice, though the quality of data sources and standardization of test methods remain areas for improvement.
• Risks Interpretation:
  • Misclassification of Lesions: There is a risk of falsely classifying benign lesions as malignant, leading to unnecessary biopsies, treatments, and associated morbidity. This issue arises because benign lesions can mimic the appearance of NMSC, which can result in overtreatment.
  • Generalization Errors: ML models may struggle to generalize effectively to novel cases not included in their training datasets. This could lead to inaccuracies in diagnosing skin conditions that differ from those represented in the training data.
  • Insufficient Training Data: Some studies used small sample sizes or non-consecutive sampling methods, which may affect the robustness of the models. For instance, reliance on images from clinics or public datasets without ensuring diverse representation can limit the model's ability to perform well across varied populations.
  • Quality of Input Data: The accuracy of the ML algorithms heavily relies on the quality of the input images. If images are poorly taken or not properly annotated, this can negatively impact the model's performance.
  • Overfitting: Models that do not employ proper validation techniques may become too tailored to their training data (overfitting), resulting in poor performance on unseen data.
  • Variability in Results: The article notes variability in reported metrics (e.g., sensitivity, specificity) across studies, which can lead to inconsistent outcomes and uncertainty about the reliability of the diagnostic tools.
  • Ethical Considerations: The deployment of AI in clinical settings raises ethical concerns regarding accountability in case of diagnostic errors, as well as patient consent and data privacy issues.
  • Dependence on Technology: Over-reliance on automated systems may diminish the role of experienced dermatologists, potentially leading to a decrease in the quality of clinical evaluations.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing: quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus, and nails) and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.
  • Operating Principle: The device is a computational medical tool leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures.
  • Use Environment: The device is intended to be used in the setting of healthcare organizations and their IT departments, which commonly are situated inside hospitals or other clinical facilities.

Article ID 58

• Title / Year: Melanoma and other skin lesion detection using smart handheld devices (2015)
• Author: Zouridakis, G.
• DOI / PMID: 10.1007/978-1-4939-2172-0_30
• Brief Summary of the article: Recent advancements in noninvasive diagnostic procedures for skin cancer have significantly transformed the landscape of dermatology, with dermoscopy emerging as the most widely adopted technique. This method enhances melanoma detection by more than 50% compared to traditional naked eye inspection, utilizing specialized equipment such as macro lenses for magnification and various light sources to visualize the skin's epidermal layers. Digital dermoscopy systems, equipped with automated image analysis capabilities, demonstrate impressive sensitivity and specificity rates for melanoma detection, often surpassing those of general practitioners and nearing the accuracy of dermatologists and dermoscopy experts. The development of portable dermoscopy devices has further expanded the accessibility of skin cancer screening. Devices like MelaFind and DermLite II utilize multispectral imaging techniques to capture multiple images of pigmented lesions and analyze their morphological characteristics. MelaFind, for example, can classify lesions based on their 3D disorganization in under a minute. The rise of smartphones has also played a pivotal role in delivering image-based diagnostic services, allowing users to capture and analyze skin lesions conveniently. Modern smartphones, with their advanced processors and high-resolution cameras, facilitate complex image analysis, enabling healthcare providers to deliver effective diagnostic services even in resource-limited settings. Smartphone applications for skin cancer screening have proliferated, providing tools for self-examination and lesion monitoring. Some applications incorporate algorithms to analyze images and predict the likelihood of malignancy. These apps often integrate with attachments designed for dermoscopic imaging, enabling users to obtain high-quality images for further analysis. The growing reliance on smartphones and digital technologies in dermatology reflects a broader trend toward personalized, accessible healthcare solutions. Automated classification systems have evolved significantly, employing objective dermoscopic criteria to evaluate pigmented lesions and aiding in early melanoma detection. Through image segmentation, feature extraction, and robust classification algorithms, these systems improve diagnostic accuracy, making them invaluable in contemporary dermatological practice.
• Risks Interpretation:
  • Diagnostic Accuracy: While automated systems and smartphone applications can enhance diagnostic capabilities, there is a risk of misclassification or false negatives, potentially leading to missed melanoma cases or unnecessary biopsies.
  • User Dependence: The effectiveness of smartphone applications relies heavily on the user's ability to capture high-quality images. Poor image quality or incorrect use of the technology may compromise diagnostic accuracy.
  • Limited Training and Experience: General practitioners using these technologies may lack the necessary training or experience, which could affect their diagnostic skills compared to specialists. This can lead to variability in the interpretation of results.
  • Data Privacy and Security: The use of smartphone applications raises concerns about data privacy and security, as personal health information may be at risk of exposure or misuse.
  • Over-Reliance on Technology: There is a risk that clinicians might become overly reliant on automated systems and smartphone applications, potentially neglecting the importance of clinical examination and thorough patient history in the diagnostic process.
  • Accessibility Issues: While portable devices and smartphone applications enhance accessibility, disparities in technology access and internet connectivity could create inequalities in skin cancer detection and treatment.
• IFUs claims interpretation:
  • Intended Purpose: The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing: quantification of intensity, count, extent of visible clinical signs, and interpretative distribution representation of possible International Classification of Diseases (ICD) classes.
  • Indications: The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus, and nails) and associated mucous membranes (conjunctival, oral, and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).
  • Contraindications: We advise not to use the device in: Skin structures located at a distance greater than 1 cm from the eye, beyond the optimal range for examination. Skin areas that are obscured from view, situated within skin folds or concealed in other manners, making them inaccessible for camera examination. Regions of the skin showcasing scars or fibrosis, indicative of past injuries or trauma. Skin structures exhibiting extensive damage, characterized by severe ulcerations or active bleeding. Skin structures contaminated with foreign substances, including but not limited to tattoos and creams. Skin structures situated at anatomically special sites, such as underneath the nails, requiring special attention. Portions of skin that are densely covered with hair, potentially obstructing the view and hindering examination.

Results from systematic review of literature

The clinical performance of artificial intelligence (AI) and software applications for assessing skin images, particularly in the context of skin cancer detection, has shown promising results across various studies. AI, specifically deep convolutional neural networks (CNNs), has been successfully implemented in primary care settings to enhance the sensitivity and specificity of melanoma detection. For instance, algorithms like Fotofinder Moleanalyzer have demonstrated sensitivity and specificity rates comparable to those of trained dermatologists. Additionally, a customized AI model developed to address data limitations achieved a high diagnostic accuracy and sensitivity in identifying melanoma, outperforming dermatologists in some instances.

Furthermore, studies have illustrated the potential of AI-driven algorithms to match or even surpass clinician performance in differentiating malignant from benign lesions, with large annotated datasets playing a critical role in training these models. The Youzhi AI software, trained on extensive databases, has shown diagnostic accuracies of 91.2% for distinguishing benign and malignant tumors, providing significant support in clinical settings. However, real-world performance can vary, and the reliance on high-quality datasets remains a challenge, particularly concerning class imbalances between melanoma and non-melanoma cases.

Recent advancements in smartphone applications and portable dermoscopy devices have also improved access to skin cancer screening, enabling patients to engage in self-examinations and facilitating remote diagnostic services. Although these technologies have made significant strides in accuracy and accessibility, challenges persist in ensuring consistent real-world implementation due to variations in user skill, patient demographics, and environmental factors. Overall, while AI and software applications present substantial opportunities for enhancing skin cancer diagnosis, further research and clinical integration are essential to maximize their efficacy and address existing limitations.

Activity 2: PMCF studies

Introduction

This activity compiles the clinical validations performed to assess the safety and performance of the medical device in a real-world environment. Currently, 5 clinical investigations have been finished, 1 clinical investigations is currently ongoing and recruiting patients and 1 investigation the first part has concluded and the second will start in 2025. The overall objective of this strategy is to support the safety and effectiveness of the device in specific patient populations and under different clinical conditions.

These studies were performed according to the R-TF-015-001 Clinical evaluation plan and the R-TF-007-002 Post-Market Clinical Follow-up (PMCF) Plan and the protocols collected in the Clinical Investigation Plans. The results were collected at the R-TF-015-002 Preclinical and clinical evaluation record_2023_001 and also in the corresponding Clinical Investigation Report of each study.

Ongoing studies

Pilot study for the clinical validation of an artifical intelligence algorithm to optimize the appropriateness of dermatology referrals.

• Main objective: To validate that Legit.Health artificial intelligence algorithms are a valid tool for optimizing the appropriateness of dermatology referrals.
• Secondary objectives:
  • To validate that the device reduces costs in secondary care.
  • To validate that the device reduces dermatology waiting lists.
  • To validate that the device optimizes clinical flow in Osakidetza.
• Design: Prospective observational and analytical study of a longitudinal clinical case series.
  • Sample size: This study will include 400 patients. Currently, 79 patients have been recruited.
  • Duration: 4 months. An extension of the study has been requested to continue patient recruitment.
• Current status: Recruiting patients to achieve the expected sample size.
• Results:
  • In the detection of malignancy, the device showed a 100% sensitivity and 76% of specifity. On the other hand, the primary care practicioners do not properly identify malignant cases, showing a sensibility of 25% and they have a high specificity of 96%.
  • Dermatologists opt for in-person consultations for 71% of the patients, while they address and resolve 29% of teledermatology cases directly.
  • General practitioners believed that 86% of cases should be referred to a dermatologist, though dermatologists did not schedule in-person visits for 29% of teleconsultations. An estimated 15% of referrals could be avoided using tools like Legit.Health, with 30% of referred cases ultimately not requiring in-person visits. The algorithm flagged 57% of cases marked for referral as low malignancy, correctly identifying 43% of non-attended cases as low-risk. For cases without referral, all had low malignancy values, confirming no malignancy, except for one case later diagnosed as psoriasis.
  • Regarding cost reduction, around 30% of the consultations initially deemed necessary for referral did not result in an in-person dermatologist consultation. Approximately 56% of the consultations that appeared to require an in-person visit turned out to be benign or cases that could have been managed in primary care.
  • The average waiting time for dermatologist consultations was 10 days.
• Conclusions: It is soon to draw conclusions since this study is still recruiting patients. But these initial results seem to show that the use of the medical device could optimize costs, reduce waiting times, and expedite urgent cases, assuming appointments were delayed due to waiting lists.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT.HEALTH_DAO_Derivación_O_2022.

Optimization of clinical flow in patients with dermatological conditions using Artificial Intelligence.

• Main objective: To validate that the device optimizes the clinical flow and patient care process, decreasing the time and cost of care per patient, through greater accuracy in medical diagnosis and determination of the degree of malignancy or severity.
• Secondary objectives:
  • To demonstrate that the device improves the ability of healthcare professionals in detecting malignant or suspected malignant pigmented lesions.
  • Demonstrate that the device improves the ability and accuracy of healthcare professionals in measuring the degree of involvement of patients with female androgenic alopecia.
  • To demonstrate that the device improves the ability and accuracy of healthcare professionals in measuring the degree of involvement of patients with acne.
  • Automate the initial triage/assessment process in patients consulting for pigmented lesions.
  • To evaluate the reduction in the use of healthcare resources by the center by reducing the number of triage consultations and direct referral of the patient to the appropriate consultation (aesthetic or dermatological).
  • Evaluate the degree of usability of the device by the patient.
  • Demonstrate that the device increases specialist satisfaction.
  • Evaluate the reduction in the use of healthcare resources by reducing the number of triage consultations and directing the patient directly to the appropriate consultation, whether in the aesthetic or dermatological field.
• Design: This is a prospective observational study with both longitudinal and retrospective case series.
  • Sample size: Prospectively, a minimum of 60 cases will be included: 30 with pigmented lesions, 15 with androgenic alopecia and 15 with inflammatory acne. Retrospectively, 60 patients with pigmented lesions, 15 with androgenic alopecia and 15 with inflammatory acne will be included.
  • Duration: The first part of the study took 5 months to be completed.
• Current status: The first part of the study, which focused on the analysis of pigmented lesions and androgenic alopecia, finished in August 2024. The second part of the study will start on 2025.
• Results:
  • For retrospective images, the medical deviced exhibited an AUC of 0.76 in detecting lesion malignancy, while the dermatologists achieved an AUC of 0.79.
  • The medical device achieved a top-5 accuracy of 0.47 regarding the diagnosis assessment, while the dermatologists achieved a 0.45 top-3 accuracy. When not accounting for the specific kind of nevus in the diagnosis, the medical device achieves a superior top-5 accuracy of 0.78 and the dermatologists achieve a top-3 accuracy of 0.70.
  • In the analysis of prospective images and in relation with the performance of the dermatologist with the medical device, they get an AUC of 0.94. In terms of diagnosis performance, the dermatologists aided by the legacy medical device achieved a top-1 accuracy of 0.30.
  • For androgenetic alopecia, 49 retrospective images in addition to 13 previously obtained were collected. The overall accuracy of the model was 47%, while the accuracy of the latest model optimized for FAA was 53%, based on the investigator's scores.
• Conclusions: It is soon to draw conclusions, since the second part of the study has yet to begin. But in this first part of the study the device's diagnostic capability in distinguishing malignancy is on par with expert dermatologists, not only in teledermatology but also in in-person consultations. This confirms its reliability as a screening tool for malignant ICD-11 categories, helping to prioritize patients based on urgency and direct them to the appropriate specialist or consultation. Additionally, we observed a strong correlation in Ludwig scores, despite a decline in the prospective trial, which may be attributed to inconsistencies in criteria alignment.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan Legit.Health_IDEI_2023.

Clinical validation study of a Computer-Aided Diagnosis (CADx) system with artificial intelligence algorithms for early non-invasive detection of in vivo cutaneous melanoma.

• Main objective: To validate that the artificial intelligence algorithm developed by AI LABS GROUP S.L. for the identification of cutaneous melanoma in images of lesions taken with a dermoscopic camera achieves the following values:
  • AUC greater than 0.8.
  • Sensitivity of 80% or higher.
  • Specificity of 70% or higher.
• Secondary objectives: Validate the usefulness and feasibility of the artificial intelligence algorithm developed by the manufacturer in adverse environments with severe technical limitations, such as lack of instrumentation or lack of internet connection.
• Design: Analytical observational and cross-sectional case series study for the performance of a diagnostic test study.
  • Sample size: The proposed number for this study was 200. By the end of the study, 105 patients were recruited. Despite being a smaller sample than the goal (200 subjects), we managed to increase the ratio of cutaneous melanoma cases originally planned (from 20% to 34%).
  • Duration: 5 years.
• Results:
  • In relation with the melanoma detection, the medical device achieved a top-1 precision of 0.80, a top-3 sensitivity of 0.90 and a top-1 specifity of 0.80.
  • For melanoma detection, the medical device showed and AUC of 0.84, which is considered excellent.
  • In the analysis of skin recognition, the medical device achieved a top-1 accuracy of 0.54, a top-3 accuracy of 0.75 and a top-5 accuracy of 0.84.
  • Finally, the medical device showed an AUC of 0.88 in the detection of malignancy.
• Conclusions: The device demonstrates great malignancy prediction and compelling image recognition capacity for melanoma and other pigmented skin lesions such as carcinoma, keratoses or nevus, with results similar to internal validation tests. Regarding the detection of melanoma, the data collected in this study limits the power of the analysis due to class imbalance, difficult diagnoses, and inconsistent image quality, but the results obtained are compelling even under such challenging conditions.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT_MC_EVCDAO_2019.

Clinical Validation of a Computer-Aided Diagnosis (CAD) System Utilizing Artificial Intelligence Algorithms for Continuous and Remote Monitoring of Patient Condition Severity in an Objective and Stable Manner.

• Main objective: The primary aim of this study is to ascertain the validity of the device, leveraging artificial intelligence and developed by AI LABS GROUP S.L., in objectively and reliably tracking the progression of chronic dermatological conditions. This validation is deemed successful if the tool achieves a score of 8 or higher in the Clinical Utility Questionnaire (CUS).
• Secondary objectives:
  • Confirming that the utilization of the device elicits a high level of patient satisfaction, particularly in its remote application.
  • Demonstrating that the implementation of the device leads to a reduction in face-to-face consultations, thereby optimizing healthcare resources and patient convenience.
  • Validating the device's ability to consistently generate reliable condition monitoring, thereby establishing its trustworthiness as a monitoring system.
• Design: Prospective observational analytical study of a longitudinal clinical case series.
  • Sample size: 160 patients.
  • Duration: 19 months.
• Results:
  • A total of 400 patients were initially considered for inclusion in this study. However, after screening based on the predefined study criteria, 240 individuals were excluded. Consequently, the final cohort comprised 160 patients who met the specified eligibility criteria.
  • The analysis of the Clinical Utility Questionnaire, the overall score, calculated by averaging scores across specialists and questions normalized from 0 to 100, stands at 71.39. For questions 2, 6, and 10, a score of 100 indicates "yes" and a score of 0 indicates "no." For question 5, responses "I have not reduced time" score 0, otherwise they score 100.
  • Regarding the Data Utility Questionnaire, the aggregate score, obtained by averaging responses across specialists and questions, stands at 87 +- 16.
  • An evaluation of the System Usability Questionnaire showed by averaging responses across specialists and questions, stands at 87.00.
  • In relation with the Patient Satisfaction Questionnaire, the overall score, calculated by averaging scores across patients and questions, stands at 70.77.
• Conclusions: The device proves highly effective, safe, and user-friendly for managing chronic dermatologic conditions. Positive feedback from specialists and patients underscores its potential as a valuable clinical tool. The device exhibits significant clinical relevance in dermatology, offering objective follow-up in skin evaluation. The device streamlines the diagnostic process, reducing clinician workload. While providing substantial benefits, it is emphasized that the tool should complement, not replace, clinical judgment. Overall, the device holds promise as a valuable clinical decision support tool for dermatologists.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT_COVIDX_EVCDAO_2022.

Enhancing Dermatology E-Consultations in Primary Care Centres using Artificial Intelligence

• Main objective: To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.
• Secondary objectives:
  • Reduce and correct the referral of patients with skin pathologies from primary care to dermatology.
  • Individualize and improve the ongoing training of primary care physicians in the area of dermatology.
  • Offer healthcare adapted to technological innovations.
  • Measure the satisfaction of primary care physicians with the Legit.Health platform.
  • Measure the satisfaction of dermatologists with the Legit.Health platform.
• Design: Prospective, observational and analytical study.
  • Sample size: At least 15 researchers have made at least one diagnostic report using Legit.Health. In total, 180 diagnostic reports were recorded for 131 patients. The target of 100 has been exceeded.
  • Duration: 18 months.
• Results:
  • 15 primary care physicians were included in this study. In total, 180 diagnostic reports were recorded for 131 patients.
  • In diagnosing specific conditions such as hidradenitis suppurativa (HS), Legit.Health helped identify two cases initially undiagnosed by primary care physicians (PCPs), with one confirmed later by a dermatologist. Additionally, HS appeared as a suggested diagnosis in four other reports.
  • In skin cancer cases, specifically melanoma, five cases were confirmed through pathology, with dermatologists suspecting melanoma in 10 cases.
  • Using Legit.Health, specialists achieved a sensitivity of 60% and specificity of 91% for melanoma detection. Overall, the tool's AUC for malignancy detection (including melanoma and carcinomas) was 0.84.
  • Clinicians completed a survey on the clinical utility of Legit.Health, with responses from 8 primary care physicians and 2 dermatologists.
• Conclusions: The medical device Legit.Health improved the diagnosis of skin diseases by helping primary care physicians (PCPs) identify cases of hidradenitis suppurativa and psoriasis that were previously undiagnosed without the device, and by enhancing urticaria detection. For skin cancer detection, specifically melanoma, PCPs using the tool achieved a sensitivity of 60% and a specificity of 91%. The device's malignancy index reached an AUC of 0.84, indicating strong differentiation between malignant and benign cases. PCPs and dermatologists reported high satisfaction with the tool's overall performance, ease of use, diagnostic support, and patient management efficiency. The platform also received positive feedback for triaging and handling urgent cases. Although this study could not analyze improvements in dermatological disease diagnosis by primary care physicians due to the low volume of pathologies and physician participation in this area, this will be considered in future studies to evaluate the diagnostic accuracy improvement with Legit.Health.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT.HEALTH_DAO_Derivación_PH_2022.

Non-Invasive Prospective Pilot in a Live Environment to improve diagnosis in primary care.

• Main objective: To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.
• Secondary objectives:
  • To validate what percentage of cases should be referred according the HCPs with the information provided by the device.
  • To validate what percentage of cases could be handled remotely with the information provided by the device.
• Design: Prospective observational analytical study of a longitudinal clinical case series.
  • Sample size: In this study the population will consist of primary care physicians and dermatologists. A minimum of 15 physicians will be selected. Each participant will be presented with 30 images to review.
  • Duration: 3 months.
• Results:
  • Primary care doctors demonstrated an accuracy of 72.96%, which notably increased to 82.22% with the integration of Legit.Health.
  • In assessing the impact of Legit.Health on referrals, our findings revealed that 48.89% of cases did not necessitate a referral.
  • Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results show that 60.74% of the cases can be handled remotely.
  • A strong association exists between referrals and remote consultations. 36.67% of the cases do not require referral and can have follow-up remotely, 12.22% of the cases do not require referral but require an in-person appointment, 24.07% of the cases require referral and remote consultation and a 27.04% of the cases require a referral in addition to an in-person appointment.
• Conclusions: The medical device Legit.Health showed an improvement in the diagnostic accuracy in primary care, specially in conditions such as hidradenitis suppurativa or actinic keratosis. The implementation of these technologies can help improve remote patient management and reduce healthcare pressure.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT.HEALTH_PH_2024_NIPPLE.

Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of Generalized Pustular Psoriasis.

• Main objective: To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of generalized pustular psoriasis (GPP).
• Secondary objectives: To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of other dermatological skin conditions, such as hidradenitis suppurativa.
• Design: Prospective cross-sectional study.
  • Sample size: 15 healthcare professionals. Each participant was presented with a total of 100 cases or images to review.
  • Duration: 4 months.
• Results:
  • On average, diagnostic accuracy increased from 47.91% to 62.81%, a relative increase of 31%. For primary care doctors, the improvement was even more pronounced, with a 40% relative increase in correct diagnoses.
  • Regarding General Pustular Psoriasis (GPP), there was an increase of 24.44% with the use of the medical device. This effect was even bigger in primary care, with an increase of 120% in GPP diagnoses.
  • For other conditions like hidradenitis suppurativa and palmoplantar pustulosis, Primary care doctors correctly diagnosed 12.43% more cases, while dermatologists showed similar improvements with the device. For palmoplantar pustulosis, primary care doctors demonstrated an outstanding 146% increase, with dermatologists maintaining their performance.
• Conclusions: In conclusion, while the dermatologist's data were not statistically significant at the pathology level due to the small sample size (only four dermatologists compared to eleven in primary care), this was not a design flaw, as the focus of the study was on primary care. The high level of expertise among the dermatologists, particularly in hidradenitis suppurativa (HS), combined with the average complexity of HS cases, may explain why the tool was less impactful for them. However, this does not diminish the tool's potential usefulness for other dermatologists. Overall, Legit.Health had a substantial impact, particularly in primary care and for rare conditions like generalized pustular psoriasis (GPP), significantly improving diagnostic accuracy. This improvement may enhance referral appropriateness and reduce healthcare pressure by managing more cases effectively at the primary care level.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT.HEALTH_BI_2024.

Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of skin pathologies in primary care and dermatology.

• Main objective: To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.
• Secondary objectives:
  • To validate what percentage of cases should be refered according the HCP with the information provided by the device.
  • To validate what percentage of cases could be handled remotely with the information provided by the device.
• Design: Prospective cross-sectional study.
  • Sample size: In this study, a total of 16 healthcare professionals (HCPs) participated, comprising 10 primary care doctors and 6 dermatologists. Among them, 12 completed the entire process, while the remaining 4 reviewed a partial number of images, specifically 28, 15, 9, and 1 respectively.
  • Duration: 4 months.
• Results:
  • The use of the medical device improved diagnostic accuracy from 68.08% to 88.78%, with primary care physicians seeing an increase from 62.90% to 89.92% and dermatologists from 76.47% to 86.93%. Significant improvements were observed in conditions like tinea, granuloma annulare, and seborrheic keratosis.
  • In assessing the impact of Legit.Health on referrals, our findings revealed that 58.1% of cases did not necessitate a referral. However, this percentage varied slightly to 60.89% for primary care doctors and 53.59% for dermatologists.
  • Experts agreed on remote management for acne, herpes, and tinea, while melanoma and nevus required in-person care.
  • Additionally, 87% of healthcare professionals found the tool efficient, reducing consultation time to under 10 minutes.
• Conclusions: In conclusion, Legit.Health proved to be a valuable tool in enhancing diagnostic accuracy for both primary care physicians and dermatologists. The system was particularly effective in improving the management of conditions such as tinea, granuloma annulare, and seborrheic keratosis, and facilitated more accurate diagnoses across a wide range of skin conditions. Its use reduced the need for referrals, allowing a significant portion of cases to be managed remotely, which helped alleviate pressure on specialist care. Feedback from healthcare professionals highlighted its utility and efficiency, especially in supporting remote consultations and streamlining patient management. These findings suggest that Legit.Health can play a crucial role in improving diagnostic workflows and optimizing healthcare resources in dermatology.
• More information: For more project information, see the study protocol at R-TF-015-006 Clinical investigation plan LEGIT.HEALTH_SAN_2024.

Activity 3: Image recognition processor success metrics

Introduction

This activity is an analysis of the performance of the previous generation of the device, which is identical in terms of software components. All the reports generated by the device during its time in the market were reviewed and analysed.

The goal of this analysis is to understand the performance of the device and its evolution throughout different iterations of the device, as well as to detect and inspect any possible case of unsuccessful performance, confirm the safety and performance, identify possible product misuse and monitor emergent risks.

Methodology

Source data

To conduct this analysis, a total of 4,857 reports were available to review, out of which 3,708 came from images taken by patients and 1,149 by health care practitioners.

Each report contains a variety of data, but we focused on the following items:

• Input image
• List of predicted classes
• Confirmed diagnosis (the class selected by the healthcare practitioner)
• Malignancy and premalignancy scores
• Name of the healthcare organisation
• Name of the health care practitioner
• Body site depicted in the image

Metrics

The performance of the device was evaluated using the following metrics:

• Top-1, Top-3, and Top-5 accuracy: Top-1, Top-3, and Top-5 accuracy metrics collectively provide a nuanced understanding of the device's performance, revealing not just its precision in making the correct prediction outright, but also its capability to list the correct outcome among its most confident suggestions. This ensures a comprehensive assessment of the device's reliability and effectiveness in aiding medical practitioners in the diagnostic process.
• Top-5 error rate: to measure how often the top-5 outputs of the device do not include the expected class.
• Taxonomic Discrepancy Rate (TDR): describes the rate at which there are differences between the device's output and the practitioner's diagnosis due to taxonomic discrepancies.
• Area Under the Curve (AUC): to measure the malignancy suspicion performance. The AUC is derived from the Receiver Operating Characteristic (ROC) curve, which is a graphical representation of a diagnostic test's true positive rate (sensitivity) against its false positive rate (1 - specificity) across various threshold setting. An AUC value ranges from 0 to 1, where an AUC of 0.5 indicates no discriminative ability (akin to random guessing), and an AUC of 1.0 indicates perfect discriminative ability.

These metrics can be easily computed from the data available in the reports based on the classes predicted by the image recognition processor used at the time of the report and the corresponding confirmed class. We refer to these metrics as the accumulated metrics, since they don't come from a single processor but from the continuous iterations of the image recognition processor.

In order to compare the accumulated performance to that of the latest image recognition processor, all the images were processed to obtain the latest ICD class probability distributions and the malignancy and premalignancy probabilities. We refer to these as the latest metrics.

Definitions

• Accumulated Metrics: Derived from past reports, these metrics aggregate the results of continuous iterations of the image recognition processor over time, using predicted and confirmed classes.
• Latest Metrics: These metrics result from reprocessing all images with the most current image recognition processor to obtain up-to-date probability distributions and assess its performance.

Known limitations

Before delving into the results, it's important to understand four limitations of this analysis:

1. We only used the reports that had confirmation

The percentage of reports with a confirmation from an HCP was 81.20% (3,944). We discarded the remaining 913 reports to conduct this analysis.

2. The 'confirmation' cannot be considered a gold standard

The data we are using as 'confirmed' has been determined by a wide range of practitioners in real-world settings. These HCPs range from primary care physicians to nurses, who are not experts in the disease - as well as dermatologists. Furthermore, we have no way of knowing how thorough their assessment was. This means that, when the device does not match the confirmation, there is a significative chance that the practitioner is wrong and the device is correct.

3. The Taxonomic Discrepancy Rate (TDR) is 42%

From the 3,944 reports that had confirmation from an HCP, it is of utmost importance to mention that there was a TDR of 0.42, which is very high.

What is Taxonomic Discrepancy Rate (TDR)

TDR describes the rate at which there are differences between the device's output and the practitioner's diagnosis due to taxonomic discrepancies. For example, this can happen if the device outputs Acne, but the practitioner confirms the class Acne vulgaris or Steroid acne, which are not actually part of the device.

After inspecting the reports where the confirmed class was not in the list of predicted classes, we discovered that most of them were mismatches due to the taxonomy of the ICD categories of the image recognition processor at the time of the report.

We found reports of acne, rosacea, atopic dermatitis and psoriasis that present this behavior: these may all contribute to the low top-K metrics.

4. Image quality and image content-related issues

After inspecting the images of the incorrect reports we observed that a high percentage of images were not properly taken. For example, in many images the object of interest was too far away and not properly selected, thus reducing the usability of the image to generate a valid preliminary report. However, the overall visual quality of the images of the incorrect report was acceptable. To assess visual quality, we used the device's integrated Dermatology Image Quality Assessment (DIQA) algorithm.

Why are there fewer images in this chart?

DIQA was developed in 2022, which means that all reports before that year could not be used to create the chart as they did not contain a DIQA score.

Unsurprisingly, there are no images with a DIQA score below 50%. This is because all the reports included in this analysis are the ones sent through a user interface that rejects images which low quality and prompts the user to re-take the image.

However, as further explained later on, there are still issues with images. Even if their quality is good or acceptable, the region of interest may be too small or occluded in the image, potentially reducing the accuracy of the device.

These images are examples of pigmented lesions that are too small compared to the total image size. When the images are resized to the analyzed by the image recognition processor, the relevant semantic content of the images becomes almost residual.

To correct this, we cropped the pigmented lesions from the images. This resulted in a total of 467 images to be analyzed. The analysis was done using the prediction of the latest version of the image recognition processor.

Summary of results

After making efforts to correct for taxonomic discrepancy and issues of quality and incorrect images, the analysis of the performance of the device yelds the following results:

Metric name	Value
Top-1 accuracy	0.5161
Top-3 accuracy	0.6959
Top-5 accuracy	0.7730
Top-5 error rate	0.2270

These results are somewhat lower than the results garnered during the initial clinical evaluation of the device, but are still high enough to fulfill its intended purpose effectively.

Malignancy suspicion

Even considering the 42% Taxonomic Discrepancy Rate, the AUC metric for the malignancy suspicion is outstanding, as the following table shows.

Metric name	Value
Malignancy AUC	0.9501
Taxonomic Discrepancy Rate	0.1895

This is possibly due to the fact that the malignancy suspicion is an index, or an aggregated metric that sums the likelihood of ICD categories that are considered malignant, which makes it less susceptible to the impact of the high TDR.

Evolution of malignancy suspicion

In order to assess the evolution of the image recognition processor, we grouped the reports by year and computed the metrics in each year frame:

Year	2021	2022	2023
Taxonomic Discrepancy Rate	0.47	0.56	0.19
Malignancy AUC	0.47	0.58	0.95

Data shows that there is an inverse correlation between the Taxonomic Discrepancy Rate and the AUC of the malignancy suspicion index, which is to be expected. Still, there is an impressive jump in accuracy in 2023.

Why has malignancy suspicion improved so much in 2023?

It was not really so bad before 2023. It's just that prior to 2023, the malignancy probability included both malignant and premalignant. Since 2023, malignancy scores only account for malignant ICD categories.

Classification of pigmented lesions

We explored the accumulated performance of the image recognition processor on pigmented lesions by computing the metrics exclusively with the reports of pigmented lesions. The results suggest that the current image recognition processor is particularly suitable and reliable for specific ICD categories.

This can be attributed to the current heterogeneity of the image dataset used to train the image recognition processor, which includes a higher percentage of images of pigmented lesions than other ICD categories.

Metric name	Value (pigmented lesions)
Top-1 accuracy	0.5846
Top-3 accuracy	0.7752
Top-5 accuracy	0.8094
Top-5 error rate	0.1906
Taxonomic Discrepancy Rate	0.4161

Metrics group by managing organisation

It is interesting to analyse performance metrics by grouping reports by the managing organisation they belong to.

To this effect, part of the metrics were also computed per managing organisations and then averaged. We present both the average and weighted average (using the number of reports of each managing organisation as weights) of each metric and the corresponding standard deviation, if applicable.

Method	Top-1 accuracy	Top-3 accuracy	Top-5 accuracy
Average	0.4488 ± 0.3274	0.6283 ± 0.3574	0.6941 ± 0.3436
Weighted average	0.2018	0.3534	0.4133

Metrics group by ICD class

For each ICD class, we reframed the multi-class recognition as a binary classification and computed the same classification metrics. The high variability (high standard deviation) suggests that the performance of the processor heavily depends on the ICD class captured in the input image.

Note that sensitivity is omitted because recall and sensitivity refer to the same metric.

Metric name	Value (mean and standard deviation)
Top-1 precision	0.2117 ± 0.3152
Top-1 recall	0.2452 ± 0.3446
Top-1 specificity	0.9948 ± 0.0327
Top-3 precision	0.1129 ± 0.1877
Top-3 recall	0.3889 ± 0.1877
Top-3 specificity	0.9865 ± 0.0452
Top-5 precision	0.0884 ± 0.1422
Top-5 recall	0.4996 ± 0.4305
Top-5 specificity	0.9789 ± 0.0528

Activity 4: Similar devices data

A search for safety incidents and alerts was conducted for the similar products to Legit.Health Plus, which have been described in the clinical evaluation report (R-TF-015-003) and in the post-market clinical follow up plan (R-TF-007-002): SkinVision and MoleScope.

The information on vigilance was obtained from the following vigilance databases:

-FDA (Food and Drug Administration): provides Total Product Lifecycle data, including enforcement reports, warning letters, MAUDE database reports, CDRH inspections database, FDA recall database, and TPLC database.

• Swissmedic: Swiss agency for the authorization and supervision of therapeutic products, offering a recall list of medical devices within the scope of market surveillance.
• MHRA (Medicines and Healthcare Products Regulatory Agency): an executive agency of the Department of Health in Great Britain, responsible for ensuring the effectiveness and safety of medicines and medical devices.
• AEMPS Vigilancia de productos sanitarios: a state agency in Spain attached to the Ministry of Health, responsible for guaranteeing the quality, safety, efficacy, and accurate information of medicines and health products.

The results of the search are summarised in the following table.

Source of information	Link	Search Team	Alerts	Relevant
FDA website Enforcement Report Searchable database	http://www.fda.gov/Safety/Recalls/EnforcementReports/default.htm	MoleScope SkinVision	0	N/A
FDA website Warning letters Searchable database	http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/default.htm#recent	MoleScope SkinVision	0	N/A
FDA website MAUDE – manufacturer and User Facility Device Experience Searchable database	http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/search.CFM	MoleScope SkinVision	0	N/A
FDA website Medical Device Recalls	http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRes/textsearch.cfm	MoleScope SkinVision	0	N/A
FDA website TPLC – Total Product Life Cycle database	http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfTPLC/tplc.cfm	MoleScope SkinVision	0	N/A
Swissmedic Swiss Competent Authority	https://www.swissmedic.ch/swissmedic/en/home/medical-devices/fsca.html	MoleScope SkinVision	0	N/A
AEPMS Vigilancia de productos sanitarios	https://alertasps.aemps.es/alertasps/alertas	MoleScope SkinVision	0	N/A
MHRA Adverse events reporting	https://www.gov.uk/drug-device-alerts	MoleScope SkinVision	0	N/A

No relevant safety incidents, alerts, or adverse events were found across all databases for MoleScope and SkinVision within the search period. This indicates that, to date, similar products have not raised notable safety concerns in international regulatory databases, which supports the anticipated vigilance profile for Legit.Health Plus.

Evaluation of clinical data relating to similar devices

The table above presents data related to the similar devices identified for Legit.Health Plus in terms of safety. As previously discussed, no safety incidents, alerts or adverse events were found across the databases, thus confirming teh safety profile of Legit.Health Plus.

Further information about the similar devices, especially in terms of clinical characteristics, can be found in the R-TF-015-003 Clinical evaluation report, section Conclusion on the comparison with similar devices.

Impact of the results on the technical documentation

Clinical Evaluation Report

Upon cross-referencing the user feedback with the data and conclusions in the clinical evaluation report, we have determined that there is no need for updates or modifications to our technical documentation in this regard. The current state of the clinical evaluation report adequately reflects the real-world performance and utility of our application.

Risk Management File

Our examination has confirmed that the risks associated with the complaints received were already identified and mitigated as per the existing risk management protocols. The implemented CAPAs were in alignment with the risk mitigation strategies outlined in the risk management file, affirming the robustness and effectiveness of our risk management practices.

Clinical Evaluation Report

Regarding the last version of the Clinical Evaluation Report published at the time of publishing this document:

• No relevant information from the clinical evaluation report to be considered in this plan
• Relevant information analyzed and monitored

Analysis of the outcome is to be reported in the updated clinical evaluation report.

Risk Management File

Regarding the last version of the Risk Management File published at the time of publishing this document:

• No relevant information from the clinical evaluation report to be considered in this plan
• Relevant information analyzed and monitored

Analysis of the outcome is to be reported in the updated clinical evaluation report.

Reference to any applicable common specifications, harmonized standards or applicable guidance documents

When new standards, requirements or guides were applied we will update the corresponing section within the product Technical File.

In this section we will point out whether the collected clinical data related the device in question still confirm adherence to applied common specifications and/or applied harmonized standards, and/or guidances listed in the PMCF plan.

Common specifitations

The following common specifications have been considered during the device life cycle and documentation preparation:

• Commission Implementing Regulation (EU) 2021/2226 of 14 December 2021 laying down rules for the application of Regulation (EU) 2017/745 of the European Parliament and of the Council as regards electronic instructions for use of medical devices.
• Commission Implementing Regulation (EU) 2021/2078 of 26 November 2021 laying down rules for the application of Regulation (EU) 2017/745 of the European Parliament and of the Council as regards the European Database on Medical Devices (Eudamed).
• Commission Implementing Decision (EU) 2019/939 of 6 June 2019 designating issuing entities designated to operate a system for the assignment of Unique Device Identifiers (UDIs) in the field of medical devices.

Harmonised standards

The following harmonised standards have been considered during the device life cycle and documentation preparation:

• UNE-EN ISO 13485:2016/A11:2021 Medical devices. Quality management systems. Requirements for regulatory purposes.
• ISO 14971:2020/A11:2021 Medical devices.Application of risk management to medical devices
• ISO 15223-1:2021 Medical devices - Symbols to be used with information to be supplied by the manufacturer - Part 1: General requirements
• UNE-EN 62304:2007/A1:2016 Medical device software - Software life-cycle processes
• UNE-EN 62366-1:2015/A1:2020 Medical devices - Part 1: Application of usability engineering to medical devices
• UNE-EN ISO 14155:2021 Clinical investigation of medical devices for human subjects - Good clinical practice (ISO 14155:2020)

Guidance on PMCF

• MDCG 2020-5 Clinical evaluation - Equivance (April 2020)
• MDCG 2020-7: Post-market clinical follow-up (PMCF) Plan Template. A guide for manufacturers and notified bodies (April 2020).
• MDCG 2020-8: Post-market clinical follow-up (PMCF) Evaluation Report Template. A guide for manufacturers and notified bodies (April 2020).

Conclusions

Based on the extensive Post-Market Clinical Follow-up (PMCF) activities conducted, we have gathered valuable insights into the safety, effectiveness, and user satisfaction of the device. These findings play a crucial role in guiding our future strategies, ensuring the continuous improvement of the device, and maintaining compliance with regulatory requirements.

Relationship with R-TF-007-002 PMCF Plan

Activity 1: Clinical Literature Review

The clinical literature review has proven instrumental in validating the safety of the device for its intended use. By meticulously analyzing state-of-the-art publications on image diagnostic and severity measure methods, we have ensured that the device aligns with the current scientific and clinical knowledge.

Activity 2: PMCF studies

The PMCF studies carried out has provided results in validating the safety, performance, and clinical benefits of the use of the medical device in a real-world environment. The rigurous design of the studies helped to identify any new risk or adverse event that may not have been observed during pre-market clinical trials. Furthermore, these studies provide additional clinical evidence to support the device's ongoing use. This includes confirming that the device's intended clinical benefits are maintained in the post-market phase. Last, but not least, these studies has provided feedback on how the device is used in everyday clinical settings, helping to identify areas for device improvement or refinement.

Activity 3: Image Recognition Processor Success Metrics

The analysis of the device's performance through image recognition processor success metrics has provided valuable insights into its operational efficacy. The substantial number of reports reviewed, originating from both patients and healthcare practitioners, has allowed for a comprehensive evaluation, ensuring that the device performs as intended in diverse settings.

Activity 4: Similar Devices data

By comparing the device to similar products in the market and consulting relevant sanitary alert databases, we have gained a broader perspective on its position in the market and its safety profile. This proactive approach has ensured that we stay ahead of potential risks and maintain a competitive edge.

Integration into Clinical Evaluation and Risk Management

The findings from the PMCF activities will be integrated into the ongoing clinical evaluation of the device, ensuring that our risk management processes are informed by the most up-to-date and comprehensive data. This integration is vital for maintaining the safety and efficacy of the device throughout its lifecycle.

Identification of Preventive or Corrective Measures

While the PMCF activities have generally affirmed the safety and efficacy of the device, the identification and resolution of non-conformities through CAPAs highlight our commitment to continuous improvement. These actions have not only addressed specific issues but have also contributed to the overall enhancement of the device.

Input for the next PMCF Plan

The insights gained from this PMCF evaluation are invaluable for shaping future post-market surveillance strategies. They provide a solid foundation for the next PMCF plan, ensuring that future evaluations are even more targeted, efficient, and effective.

In conclusion, the PMCF activities have validated the safety and effectiveness of the device, highlighted areas for improvement, and underscored the positive impact of ongoing development and enhancements. These findings ensure that the device continues to meet user needs and expectations, maintains compliance with regulatory standards, and contributes positively to patient care.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-005