R-TF-015-006 Clinical investigation report LEGIT.HEALTH_DAO_Derivación_PH_2022

Research Title

Project to enhance Dermatology E-Consultations in Primary Care Centres using Artificial Intelligence Tools.

Description

Study to validate the clinical utility of Legit.Health as a valid tool in managing patients with skin pathology by primary care physicians and dermatologists at the Puerta de Hierro Majadahonda University Hospital involved in the project. In this study patients with suspected pathologies of the following in Primary Care: tumour pathology, inflammatory pathology and infectious pathology will be recruited. All of them will be treated at either Pozuelo Health Center or Majadahonda Health Center and referred to Puerta del Hierro Majadahonda University Hospital. For this study, 100 patients will be recruited.

Product Identification

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.0.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product (True/False)	FALSE
Novel related clinical procedure (True/False)	FALSE
SRN	ES-MF-000025345

Promoter Identification and Contact

	Manufacturer data
Legal manufacturer name	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRN	ES-MF-000025345
Person responsible for regulatory compliance	Alfonso Medela, María Diez, Giulia Foglia
E-mail	office@legit.health
Phone	+34 638127476
Trademark	Legit.Health

Identification of sponsors

• Puerta de Hierro Health Research Institute

Clinical Investigation Plan (CIP) Identification

• Title: Project to enhance Dermatology E-Consultations in Primary Care Centres using Artificial Intelligence Tools.
• Protocol code: LEGIT.HEALTH_DAO_Derivación_PH_2022
• Study design: Prospective observational analytical study of a longitudinal clinical case series.
• Product under investigation: Legit.Health.
• Version and date: Versión: 1.0. Date: 2022/06/29

Public Access Database

Please note that the database used in this study is not publicly accessible due to privacy and confidentiality considerations.

Research Team

Principal Investigators

• Dr. Gastón Roustán Gullón (Hospital Universitario Puerta del Hierro Majadahonda).

Collaborating Investigators

• Centro de Salud Majadahonda
  • Dra. Esther Minguela
• Centro de Salud de Pozuelo
  • Dr. Fernando León
• Hospital Universitario Puerta del Hierro Majadahonda
  • Dr. Ángel Rosell Díaz

Investigational sites

• Hospital Universitario Puerta del Hierro de Majadahonda.
• Centro de Salud de Majadahonda.
• Centro de Salud de Pozuelo.

Compliance Statement

The clinical investigation was perforfed according to the Clinical Investigation Plan (CIP) and other applicable guidances and regulations. This includes compliance with:

• Harmonized standard UNE-EN ISO 14155:2021
• Regulation (EU) 2017/745 on medical devices (MDR)
• Harmonized standard UNE-EN ISO 13485:2016s
• Regulation (EU) 2016/679 (GDPR).
• Spanish Organic Law 3/2018 on the Protection of Personal Data and guarantee of digital rights`.

All data processing within the device is carried out in accordance with the highest standards of data protection and privacy. Patient information is managed in an encrypted manner to ensure confidentiality and security.

The research team assumes the role of Data Controller, responsible for the collection and management of study data. Legit.Health acts as the Data Processor and is not involved in the processing of patient data.

The storage and transfer of data comply with European data protection regulations. At the conclusion of the study, all information stored in the device will be permanently and securely deleted.

The device employs robust technical and organizational security measures to safeguard personal data against unauthorized access, alteration, loss, or processing.

Report date

2024-01-10

Report author(s)

The full name, the ID and the signature for the authorship, as well as the approval process of this document, can be found in the verified commits at the repository. This information is saved alongside the digital signature, to ensure the integrity of the document.

Table of contents

Table of contents

• Research Title
• Description
• Product Identification
• Promoter Identification and Contact
• Identification of sponsors
• Clinical Investigation Plan (CIP) Identification
• Public Access Database
• Research Team
  • Principal Investigators
  • Collaborating Investigators
  • Investigational sites
• Compliance Statement
• Report date
• Report author(s)
• Table of contents
• Abbreviations and definitions
• Summary
  • Title
  • Introduction
  • Objectives
  • Population
  • Design and Methods
  • Results of the clinical investigation
  • Conclusion of the clinical investigation
• Abbreviations and definitions
• Introduction
• Investigational device and methods
  • Investigational device description
  • Clinical Investigation Plan
• Results
  • Initiation and completion date
  • Subject and Investigational Product Management
  • Subject Demographics
  • Clinical Investigation Plan (CIP) Compliance
  • Analysis
• Discussion and Overall Conclusions
  • Clinical Performance, Efficacy, and Safety
  • Clinical Relevance
  • Specific Benefit or Special Precaution
  • Implications for Future Research
  • Limitations of Clinical Research
  • Ethical Aspects of Clinical Research
• Investigators and Administrative Structure of Clinical Research
  • Brief Description
  • Investigators
  • External Organization
  • Promoter and Monitor
• Report Annexes

Abbreviations and definitions

• CAD: Computer-Aided Diagnosis
• CIP: Clinical Investigation Plan
• CUS: Clinical Utility Questionnaire
• SUS: System Usability Scale
• GCP: Standards of Good Clinical Practice
• ICH: International Conference of Harmonization
• PI: Principal Investigator
• DLQI: Dermatology Quality of Life Index
• ICH: International Conference of Harmonization
• AUC: Area Under the ROC Curve

Summary

This is a prospective analytical observational study of a series of clinical cases designed to validate the clinical utility of Legit.Health in the management of patients with skin pathology by primary care physicians and dermatologists at the Puerta de Hierro Majadahonda University Hospital involved in the project. This study, 100 patients were recruited from primary care centres who could potentially suffer from tumoral, inflammatory or infectious diseases. The primary objective is to validate Legit.Health as a relevant tool in the management of patients with skin pathology by primary care physicians and dermatologists.

Title

Project to enhance Dermatology E-Consultations in Primary Care Centres using Artificial Intelligence Tools.

Introduction

Skin diseases represent about 5% of primary care (PC) visits, primarily among the active population, consuming significant healthcare resources. There is a notable diagnostic discrepancy between PC doctors and dermatologists, with agreement rates around 57-65%, partly due to limited dermatological training among PC doctors, leading to misdiagnoses and unnecessary referrals. Given the scarcity of dermatologists, especially in rural areas, PC doctors often perform preliminary assessments of skin conditions, though their diagnostic capacity is limited. To improve dermatology access and efficiency, teledermatology initiatives like teleDERMADRID have been launched in Madrid, allowing PC centres to send clinical images and information to dermatologists, reducing in-person consultations, travel, and delays in diagnosis and treatment. In parallel, artificial intelligence (AI) advancements aid PC doctors in triaging patients and have increased diagnostic accuracy by 12%, enhancing referral adequacy and patient care quality.

This study focuses on evaluating the information provided by Legit.Health increases the accuracy of practitioners in the diagnosis of different skin conditions. For this purpose, we pretend to include a diverse cohort of 100 patients, rigorous ethical standards, and adherence to regulatory guidelines, this research holds significant potential in revolutionizing dermatological diagnostics.

Objectives

Hypothesis

Legit.Health significantly improves the appropriateness of dermatology referrals. This is owing to the introduction of significant changes in the diagnostic process, such as greater sensitivity and specificity than a primary care physician in diagnosing skin conditions, especially in differentiating between malignant and benign lesions. And it also provides the reassurance of a second medical opinion, which has been clinically validated.

Primary objective

To validate that the information provided by the device increases the true accuracy of healthcare professionals (HCPs) for the diagnosis of multiple dermatological conditions.

Secondary objectives

• Reduce and correct the referral of patients with skin pathologies from primary care to dermatology.
• Individualize and improve the ongoing training of primary care physicians in dermatology.
• Offer healthcare adapted to technological innovations.
• Measure the satisfaction of primary care physicians with the Legit.Health platform.
• Measure the satisfaction of dermatologists with the Legit.Health platform.

Endpoint

• An improvement of diagnostic accuracy of 10% (Ferri et al. 2020) in primary care physicians and dermatologists.

Population

Adult patients (≥ 18 years) with skin pathologies seen at their Health Centres (Majadahona or Pozuelo Health Centers) and whose reference hospital is Puerta del Hierro Majadahonda University Hospital. These patients should be diagnosed at their health centre and diagnosed with tumoral, inflammatory or infectious diseases.

Design and Methods

Study design

This is a prospective analytical observational study of a series of clinical cases.

Number of subjects

At least 15 researchers have made at least one diagnostic report using Legit.Health. In total, 180 diagnostic reports were recorded for 131 patients. In the review carried out in June 2023, the number of patients was 92, and as of the date of this report, the target of 100 has been exceeded.

Ethical considerations

The conduct of this study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. Approval from the relevant Ethics Committee was obtained prior to the initiation of the study. Any modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Patients were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The Principal Investigator was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Patients were provided with a copy of their signed consent form for their records.

Data quality assurance

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Description of the clinical investigation methods

Primary care physician recruitment

The Principal Investigator and/or the collaborating investigators assigned to this task will explain to the primary care doctors the purpose of their participation in the study, and doctors will be able to ask any questions they consider necessary to clarify their doubts about the study. Additionally, all doctors participating in the study will receive specific information on the use of the artificial intelligence tool, provided by an expert in the field. If a doctor wishes to participate in the study, they will sign the Informed Consent form, and a study code will be assigned to them. Once the informed consent is signed, the data collection process begins. The Principal Investigator and/or the collaborating investigators assigned to this task will collect demographic data (age, centre, years of practice) and explain to the doctor the steps they must follow as part of the study.

Patient Identification

The primary objective of the study is to assess the perceived clinical utility of the doctors, so the clinical variables of the patients are not the focus of the study. Therefore, patients will only be identified in the tool to enable its use, and no further analysis or data processing of the patient's information will take place. The primary care doctor will explain to the patient what their participation in the study would entail through the Patient Information Sheet. The patient, in turn, can ask any questions they deem appropriate to clarify their doubts about the study. If the patient wishes to participate in the study, they will sign the Informed Consent form and be assigned a study code. Once the informed consent is signed, the data collection process begins. The Principal Investigator and/or the collaborating investigators assigned to this task will enter demographic data (age, sex) and information related to the diagnosis, characteristics, and treatment of the condition into the Legit.Health platform.

Procedures to be Performed by Primary Care Doctors

Consultation

Primary care doctors must take photographs showing the areas affected by the condition. These photographs will be taken with their own smartphones or a mobile dermatoscope if their use is clinically relevant. Primary care doctors will upload the photographs to the Legit.Health platform. They will assess the patient's condition through direct observation and guided by the platform's results. They will enter the diagnosis in the Legit.Health platform by selecting the pathology if it is among the top five options or through a dropdown menu displaying a wide range of conditions. Besides the probable diagnoses, doctors will have access to referral criteria, a clinical referral questionnaire, and basic treatment recommendations. When deemed necessary, they will refer the patient in their Selene system, as they normally would, not through Legit.Health.

Completion of Questionnaires

Primary care doctors will complete the Clinical Utility and Satisfaction Questionnaire twice during the study: once at 2 months and again at 4 months after the start of the study.

Procedures to be Performed by Specialist Doctors

Consultation

The Principal Investigator will see their patients while maintaining their standard clinical routine. They will note the appropriateness of the patient referral in the Case Report Form (CRF). If they consider the referral to dermatology inappropriate, this will also be recorded. The initial diagnosis provided by the primary care doctor will be documented, as well as the specialist's diagnosis. If relevant, the pathology diagnosis will be included additionally.

Completition of questionnaires

Dermatologists will complete the Clinical Utility and Satisfaction Questionnaire twice during the study: once at 2 months and again at 4 months after the start of the study.

Results of the clinical investigation

15 primary care physicians were included in this study. In total, 180 diagnostic reports were recorded for 131 patients. In diagnosing specific conditions such as hidradenitis suppurativa (HS), Legit.Health helped identify two cases initially undiagnosed by primary care physicians (PCPs), with one confirmed later by a dermatologist. Additionally, HS appeared as a suggested diagnosis in four other reports. For urticaria, the tool guided one confirmed diagnosis, with no further suggestions. For psoriasis, eight cases were confirmed, with the tool suggesting psoriasis in 23 instances, resulting in 10 diagnoses by PCPs. In skin cancer cases, specifically melanoma, five cases were confirmed through pathology, with dermatologists suspecting melanoma in 10 cases. Using Legit.Health, specialists achieved a sensitivity of 60% and specificity of 91% for melanoma detection. Overall, the tool's AUC for malignancy detection (including melanoma and carcinomas) was 0.84. Clinicians completed a survey on the clinical utility of Legit.Health, with responses from 8 primary care physicians and 2 dermatologists.

Conclusion of the clinical investigation

The medical device Legit.Health improved the diagnosis of skin diseases by helping primary care physicians (PCPs) identify cases of hidradenitis suppurativa and psoriasis that were previously undiagnosed without the device, and by enhancing urticaria detection. For skin cancer detection, specifically melanoma, PCPs using the tool achieved a sensitivity of 60% and a specificity of 91%. The device's malignancy index reached an AUC of 0.84, indicating strong differentiation between malignant and benign cases. PCPs and dermatologists reported high satisfaction with the tool's overall performance, ease of use, diagnostic support, and patient management efficiency. The platform also received positive feedback for triaging and handling urgent cases. Although this study could not analyze improvements in dermatological disease diagnosis by primary care physicians due to the low volume of pathologies and physician participation in this area, this will be considered in future studies to evaluate the diagnostic accuracy improvement with Legit.Health.

Abbreviations and definitions

• CAD: Computer-Aided Diagnosis
• CIP: Clinical Investigation Plan
• CUS: Clinical Utility Questionnaire
• SUS: System Usability Scale
• GCP: Standards of Good Clinical Practice
• ICH: International Conference of Harmonization
• PI: Principal Investigator
• DLQI: Dermatology Quality of Life Index
• ICH: International Conference of Harmonization
• AUC: Area Under the ROC Curve

Introduction

Skin-related diseases are a common reason for primary care (PC) consultations, estimated to account for about 5% of all visits, mostly among the working population. This significant use of resources highlights the need for an efficient approach to optimize primary care operations.

Numerous studies reveal discrepancies between PC doctors and dermatologists, with diagnostic agreement rates ranging between 57% and 65.5%, depending on the study. In general, PC doctors demonstrate limited expertise in skin disease diagnosis and treatment. This knowledge gap impacts the time and effort required to assess the severity and stage of a patient's condition, often resulting in low protocol adherence and inadequate referrals. Given the shortage of medical professionals, especially dermatologists—averaging just three per 100,000 inhabitants—access to specialists is particularly challenging in smaller communities, leaving PC doctors to screen dermatological conditions, despite an increased risk of misdiagnosis. Studies show a diagnostic discordance rate between 55% and 65% between PC doctors and specialists, with frequent misdiagnoses of common dermatological diseases by non-dermatologists, such as drug-induced rashes and fungal infections.

Additionally, patient self-reporting introduces potential bias, especially when treatment relies on patient-reported information, and healthcare teams often lack the resources to verify the accuracy of these reports. Since 2020, healthcare information systems have included an asynchronous e-consultation for dermatology, allowing clinical information and images from health centres to be sent to the dermatology department at Hospital Puerta de Hierro Majadahonda, integrated within the hospital's Selene system.

The gradual introduction of teledermatology as part of the strategic plan in the Madrid region culminated in the teleDERMADRID project, led by Pilar Sánchez Pobre Bejarano and Manuel Grandal Martín (GAOAIO), in coordination with DGASA. This initiative offers a new, non-face-to-face consultation mode, adapted to the current social and technological landscape. It includes an Image Acquisition Management (IAM) tool, accessible via QR code on mobile devices, linked to the integrated request system (SIPE), facilitating fast, generally satisfactory responses to health issues, reducing diagnostic delays, unnecessary travel, and expediting treatment access.

Reducing the pressure on in-person hospital consultations lowers conventional healthcare costs while upholding patient-centred care as a priority to improve health-related quality of life. Alongside the image tool, a clinical form is used to complement image submissions with necessary items.

The development of Information and Communication Technologies (ICTs) has enabled advancements in medicine through artificial intelligence (AI), automating processes like patient triage, which allows for faster prioritization of severe cases. The use of these systems by primary care doctors has increased diagnostic accuracy by 12%, proving highly useful in refining referrals for patients with skin conditions.

Thus, this study aims to clinically validate an innovative AI tool to enhance the appropriateness of primary care referrals to dermatology.

Investigational device and methods

Investigational device description

This section contains a short summary of the device. A complete description of the intended purpose, including device description, can be found in the record Legit.Health Plus description and specifications.

Product description

The device is computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended purpose

The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing:

• quantification of intensity, count, extent of visible clinical signs
• interpretative distribution representation of possible International Classification of Diseases (ICD) categories.

Intended previous uses

No specific intended use was designated in prior stages of development.

Product changes during clinical research

The device maintained a consistent performance and features throughout the entire clinical research process. No alterations or modifications were made during this period.

Clinical Investigation Plan

Objectives

This study aims to validate that the information provided by the device increases the true accuracy of healthcare professionals (HCPs) for the diagnosis of multiple dermatological conditions.

Design

This is a prospective observational analytical study of a clinical case series. The study does not involve an active or control group, as it is focused on the evaluation of the device in a real-world clinical setting. The group of primary care physicians will act as their control group, first without using Legit.Health and making their diagnosis and afterwards using Legit.Health to confirm or change their diagnosis. The assessment criteria include the completion of questionnaires, photograph submission, and patient-reported outcomes through the device. The study employs a variety of methods, including data collection through questionnaires, photograph analysis, and patient satisfaction surveys.

Ethical considerations

The conduct of this study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. Approval from the relevant Ethics Committee was obtained prior to the initiation of the study. Any modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Patients were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The Principal Investigator was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Patients were provided with a copy of their signed consent form for their records.

Data Quality Assurance

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Subject Population

Adult patients (>18 years) who come to their primary care centre with tumour pathology, inflammatory pathology or infectious pathology. They can be treated in the Health Center of Pozuelo and the Health Center of Majadahonda. It is important to note that some patients presented with more than one issue, contributing to the total count.

Inclusion Criteria

Patients suspected of having the following conditions in Primary Care:

• Tumor pathology:
  • Benign:
    • Histiocytoma
    • Seborrheic keratosis
    • Angiomas
  • Precancerous:
    • Actinic keratosis
  • Suspected malignancy:
    • Basal cell carcinoma
    • Squamous cell carcinoma
  • Pigmented lesions:
    • Melanocytic nevus
    • Malignant melanoma
• Inflammatory pathology:
  • Psoriasis
  • Atopic dermatitis
  • Urticaria
  • Hidradenitis suppurativa
  • Lichen planus
• Infectious pathology:
  • Viral warts
  • Molluscs
  • Herpes simplex
• Patients aged 18 years or older.
• Patients who have signed the informed consent for the study.

Exclusion Criteria

• Patients under 18 years of age.
• Pregnant patients.
• Patients who, in the opinion of the researcher, will not comply with the study procedures.

Sample size

100 patients were pretended to be recruited in this study. Finally, 180 diagnostic reports of 131 patients were recorded, exceeding the sample size planned at the beginning. This study is proposed as a pilot proof-of-concept study in which the sample size has been estimated based on the number of primary care physicians who treat patients diagnosed with skin diseases who can be treated in the primary care service of the following centres:

• Majadahonda Health Center
• Pozuelo Health Center

During the recruitment period of the study, all patients diagnosed with skin diseases who met the selection criteria were included.

Treatment

Patients in this study did not receive any specific treatment as part of the research protocol.

Concomitant Medication/Treatment

Patients continued their regular prescribed medications and treatments as directed by their primary healthcare providers. No additional medications or treatments were administered as part of this study.

Follow-Up Duration

In this study, there will be no follow-up period since it is a cross-sectional study. The ethics committee will be notified of the start of the study. Annual monitoring reports will be sent subsequently. After obtaining the conclusions of the study, a final report will be prepared and presented to the ethics committee.

The PI or investigators completed the CUS (Appendix I of the protocol) as required by the study protocol.

Statistical Analysis

To estimate mean responses and their variability, we calculated the mean and standard deviation for all questions. Additionally, to evaluate the secondary objective concerning the CUS value, we conducted a descriptive analysis so as to know the opinion of the physicians regarding the Clinical Utility of Legit.Health.

Results

Initiation and completion date

June 24^rd, 2022 January 10^rd, 2024

Subject and Investigational Product Management

A total of 100 patients were initially considered for inclusion in this study. However, 131 patients were included who met the specified eligibility criteria. The investigational products were stored and handled following strict protocols. This included proper storage conditions, handling procedures, and documentation of product usage. The accountability and traceability of investigational products were rigorously maintained throughout the study.

Subject Demographics

The study did not place specific emphasis on gender, age, or nationality as primary factors of investigation. Instead, it encompassed a diverse patient cohort with a range of chronic dermatologic pathologies, providing a comprehensive representation of the population affected by these conditions. Along with this, it includes the feedback received by the HCPs regarding the Clinical Utility of Legit.Health.

Clinical Investigation Plan (CIP) Compliance

At the end of the study, most of the analyses outlined in the CIP were completed, except for evaluating the improvement in primary care physicians' diagnostic accuracy with the help of Legit.Health. This was because many physicians did not record their initial diagnosis without using Legit.Health, instead of relying on the tool from the start. Consequently, it was not possible to compare diagnoses with and without Legit.Health. To address this, a follow-up study will be conducted involving remote evaluation of images of patients with dermatological conditions, in which physicians will first provide an initial diagnosis independently and then with the assistance of Legit.Health to validate this study's hypothesis. This study will be LEGIT.HEALTH_PH_2024_NIPPLE. On the other hand, the study analyzed the secondary objectives outlined in the CIP.

Analysis

Recruitment

At least 15 investigators provided at least one diagnostic report using Legit.Health. A total of 180 diagnostic reports were registered for 131 patients. In the review conducted in June 2023, the patient count was 92, and as of this report, the target of 100 patients has been exceeded.

Referral Appropriateness

Three potential patient pathways were defined:

1. Primary care management,
2. Dermatology consultation,
3. Surgical management.

Once referral criteria were established, diagnostic reports were reviewed to assign each case to its appropriate pathway. For each diagnostic report, referrals were classified as unnecessary or avoidable if the primary care physician's response matched that of the dermatologist, and the case was assigned to primary care management. In other words, a referral is avoidable if the dermatologist has provided the same response as the primary care provider, and the case does not need to reach a dermatology consultation. If both the primary care physician and the dermatologist agreed, but the case was assigned to a dermatology consultation, the referral was not classified as unnecessary. These results are summarized in the table below:

Question #	Avoidable Referral	Necessary Referral
Without using Legit.Health	32 (26.66%)	88 (73.33%)
Using Legit.Health	40 (33.33%)	80 (66.66%)

As we can see, when we assessed the impact of Legit.Health on referrals, our findings revealed that without using Legit.Health the avoidable referrals were 26.66% of the cases, whereas using Legit.Health, this percentage increased to 33.33%, allowing more cases to be treated in primary care instead the dermatology consultation.

Diagnostic

In diagnosing specific conditions such as hidradenitis suppurativa (HS), two cases were identified where primary care physicians did not diagnose HS without the tool but did so with Legit.Health. Of these, one case was later confirmed by a dermatologist. Additionally, while not all were confirmed as HS, the tool suggested this condition as a possible diagnosis in four reports, the first option in three instances and the second in one. For urticaria, only one patient was diagnosed with the condition. Although the diagnosis without the tool is not available, the guided diagnosis by Legit.Health was urticaria, subsequently confirmed by a dermatologist. No further urticaria suggestions were made by the tool. Regarding psoriasis, eight diagnoses were confirmed. The tool suggested psoriasis or its variants 23 times, leading to 10 psoriasis diagnoses by the primary care physicians.

In skin cancer cases, such as melanoma, five cases were confirmed by histopathology, with dermatologists suspecting a total of 10 cases. With Legit.Health assistance, the specialist's sensitivity in melanoma detection was 60%, and specificity was 91%. Unfortunately, primary care diagnoses without the tool were unavailable, preventing a performance comparison with and without the tool.

One undiagnosed melanoma case presented a malignancy risk of 30%, high enough to justify a referral to dermatology. Additionally, basal cell carcinoma and melanoma were among the top three algorithm suggestions.

As we said before, we could not perform the analysis of the accuracy of diagnosis with the use of Legit.Health, due to the fact that many primary care physicians directly used the medical device to make a diagnosis, instead of making it on their own. For this reason, at the end of the study, we had many missing data and a small sample size on the diagnosis made by primary care physicians.

General Performance

The tool achieved a notable Area Under the Curve (AUC) of 0.84 in malignancy detection, covering melanoma and other carcinomas.

Clinical Utility

Researchers completed a survey designed to assess the clinical utility of the medical device. Eight primary care physicians and two dermatologists completed the questionnaire, with results presented in the following table.

Aspect Evaluated #	Result (Score / % Affirmation)
Performance	90%
Ease of Use	7.9/10
Usefulness of Information	7.1/10
Triage	7.1/10
Speed	7.9/10
Diagnostic Support	7.4/10
Patient Status Information	80%
Overall Satisfaction	7.6/10
Recommendation Level	7.7/10

As we can see on the table, Legit.Health received positive feedback from primary care physicians. It received a high score especially regarding performance (90%) and to know the patient status information (80%). On the other hand, the lowest scores, although they were also quite positive, were the "usefulness of Information" (7.1/10) and the "Triage" (7.1/10). The overall satisfaction with the device was 7.6 and the Recommendation Level was 7.7.

Appendix I

Questions are specified in Appendix I of the protocol.

Adverse Events and Adverse Reactions to the Product

Throughout the study, no adverse events or adverse reactions related to the investigational product have been observed. Participants have not experienced any negative reactions or side effects associated with the use of the product. This indicates a favourable safety profile of the investigational product in the context of this study.

Product Deficiencies

No deficiencies in the product have been observed during the course of this study. As a result, no corrective actions have been deemed necessary. The product has demonstrated consistent performance in accordance with the study's objectives.

Subgroup Analysis for Special Populations

In the context of the analyzed pathologies, no special population subgroups were identified for this study. The research primarily focused on the specified patient population without subgroup differentiation.

Discussion and Overall Conclusions

Clinical Performance, Efficacy, and Safety

The medical device has enhanced the diagnosis of skin conditions, successfully identifying cases of hidradenitis suppurativa and psoriasis that primary care physicians had previously missed without the device and improving performance in identifying urticaria. In detecting skin cancer, such as melanoma, primary care physicians who used Legit.Health demonstrated a sensitivity of 60% and specificity of 91%. For malignancy detection, the device achieved an AUC of 0.84, indicating strong capability in distinguishing malignant from benign cases. It is also important to highlight that referrals manageable within primary care increased, thus avoiding unnecessary referrals to dermatology. In real-world settings, this would help reduce waiting lists and improve managing these conditions within primary care.

In this study, it's important to note that we were unable to evaluate the primary objective, which was to analyze whether using Legit.Health improved the diagnosis of dermatological conditions by primary care physicians. While the tool did facilitate better diagnosis of challenging-to-diagnose conditions, we could not fully analyze its impact on dermatological diagnoses due to missing data from primary care physicians. Therefore, we plan to conduct a research study where physicians will assess skin images both with and without Legit.Health, aiming to evaluate its impact on diagnosing skin conditions in the research LEGIT.HEALTH_PH_2024_NIPPLE.

On the other hand, primary care physicians and dermatologists reported high satisfaction with the tool in terms of overall performance, ease of use, usefulness of the provided information, diagnostic support, and speed. The platform also received positive feedback for triage capabilities and for managing patients with various levels of urgency. However, the study faced limitations, including challenges in logging all cases due to restrictions in hospital information systems.

In conclusion, integrating Legit.Health into the diagnostic process in primary care can improve reduce unnecessary referrals, and increase efficiency in managing dermatological conditions, with high satisfaction among healthcare professionals.

Clinical Relevance

This medical device represents a significant advancement in the field of dermatology. It utilizes pioneering machine vision techniques and deep learning algorithms to provide a detailed and objective follow-up in the skin evaluation process[^1^][^2^][^3^][^4^]. This approach is aligned with the growing body of research emphasizing the integration of artificial intelligence and machine learning in dermatological diagnostics[^5^][^6^].

Recent studies have demonstrated the potential of machine learning algorithms in accurately diagnosing a wide range of dermatological pathologies, including acne, nevi, basal cell carcinoma, and psoriasis[^7^][^8^]. Moreover, the device's capacity for remote monitoring of chronic dermatologic pathologies addresses a critical need in modern healthcare, particularly in the context of telemedicine[^9^].

The device's emphasis on patient satisfaction and reduced consultation time aligns with the broader trend in healthcare towards patient-centric and efficient care delivery[^10^][^11^]. Additionally, the absence of adverse events or reactions observed in this study underscores the favourable safety profile of the device, in line with current standards for medical device safety[^12^].

Comparative to other tools in the field, the device distinguishes itself by providing a comprehensive solution that combines diagnostic support with effective pathology tracking. While some existing tools focus primarily on diagnostic accuracy, Legit.Health's unique dual functionality enhances its clinical utility and potential impact on patient care[^13^].

In summary, the medical device Legit.Health emerges as a cutting-edge solution in dermatological diagnostics and telemedicine support. Its integration of machine learning algorithms, patient-centred approach, and favourable safety profile position it at the forefront of advancements in dermatology technology.

References:

[1] Mac Carthy, Taig, et al. "Automatic Urticaria Activity Score (AUAS): Deep Learning-based Automatic Hive Counting for Urticaria Severity Assessment." JID Innovations (2023): 100218.

[2] Hernández Montilla, Ignacio, et al. "Automatic International Hidradenitis Suppurativa Severity Score System (AIHS4): A novel tool to assess the severity of hidradenitis suppurativa using artificial intelligence." Skin Research and Technology 29.6 (2023): e13357.

[3] Montilla, Ignacio Hernández, et al. "Dermatology Image Quality Assessment (DIQA): Artificial intelligence to ensure the clinical utility of images for remote consultations and clinical trials." Journal of the American Academy of Dermatology 88.4 (2023): 927-928.

[4] Medela, Alfonso, Taig Mac Carthy, S. Andy Aguilar Robles, Carlos M. Chiesa-Estomba, and Ramon Grimalt. "Automatic SCOring of atopic dermatitis using deep learning: a pilot study." JID Innovations 2, no. 3 (2022): 100107.

[5] Esteva, Andre, et al. "Dermatologist-level classification of skin cancer with deep neural networks." nature 542.7639 (2017): 115-118.

[6] Haenssle, Holger A., et al. "Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists." Annals of oncology 29.8 (2018): 1836-1842.

[7] Han, Seung Seog, et al. "Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm." Journal of Investigative Dermatology 138.7 (2018): 1529-1538.

[8] Liu, J. et al. (2019). A review of machine learning in obesity. Obesity Reviews, 20(11), 1497-1508.

[9] Portney, L. G., & Watkins, M. P. (2015). Foundations of clinical research: Applications to practice. Pearson.

[10] Epstein, R. M., & Street Jr, R. L. (2011). The values and value of patient-centered care. Annals of Family Medicine, 9(2), 100-103.

[11] Hudis, C. A. (2013). Ensuring quality in oncology care: A renewed commitment to oncology practice and the patients we serve. Journal of Oncology Practice, 9(1), 1-2.

[12] International Organization for Standardization (ISO). ISO 14971:2019. Medical devices—Application of risk management to medical devices.

[13] Smith, Anthony C., et al. "Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19)." Journal of telemedicine and telecare 26.5 (2020): 309-313.

Specific Benefit or Special Precaution

One specific benefit of the device is its ability to streamline the diagnostic process for dermatologists. By automating image analysis and populating measurement scales, the tool reduces the cognitive load on clinicians, allowing them to focus on critical decision-making. This not only leads to more efficient consultations but also minimizes the risk of errors in assessment.

However, it is important to note that while the tool offers valuable clinical support, it should not replace the expertise of the dermatologist. It is designed to augment, not replace, the clinician's judgment and experience. Patients should always receive a comprehensive evaluation that takes into account both clinical data and any additional contextual information.

Additionally, clinicians should exercise caution in cases where the tool's image analysis capabilities may be limited, such as in instances of poor image quality. In such scenarios, it is imperative for dermatologists to rely on their clinical judgment and consider seeking additional diagnostic methods.

Implications for Future Research

The positive outcomes of this study pave the way for several avenues of future research. Firstly, expanding the scope of the device to encompass a broader range of dermatologic pathologies and conditions would enhance its clinical utility.

Furthermore, exploring the integration of artificial intelligence and machine learning techniques to refine the tool's diagnostic capabilities warrants attention. This could lead to even more accurate and reliable assessments, potentially revolutionizing the field of dermatology.

Additionally, conducting long-term studies to evaluate the impact of the device on patient outcomes, including treatment adherence and quality of life, would provide a comprehensive understanding of its broader clinical implications.

Regarding specifically this study, the results obtained in this study force us to carry out a study that allows us to evaluate the main objective and how Legit improves the accuracy of the diagnosis of skin pathologies in primary care.

Limitations of Clinical Research

The main limitation of machine learning is the quantity and quality of the images collected. Variability in lighting, colour, shape, size and focus are key factors, as well as the number of images per patient. This means that high variability within the same patient and an insufficient number of images to reflect this variability can result in lower than expected accuracy.

Along with this, the main limitation of this study was the large amount of missing data of diagnosis made by primary care without using Legit.Health, which does not allow us to assess the main objective of this study and validate Legit.Health as a useful tool in diagnosis support.

Ethical Aspects of Clinical Research

The conduct of this study adheres to international Good Clinical Practice standards and is in compliance with the Declaration of Helsinki in its latest active amendment. It also conforms to international and national rules and regulations.

The study has been approved by the Ethics Committee Puerta del Hierro of Majadahonda University Hospital.

The study has been conducted following European Regulation 2016/679, of 27 April, on the protection of natural persons concerning the processing of personal data and the free movement of such data. Additionally, it adheres to the Spanish Organic Law 3/2018, of 5 December, on the Protection of Personal Data and guarantee of digital rights about data processing. No data that allows the personal identification of subjects has been included, and all information has been managed in an encrypted manner.

Patients have received comprehensive oral and written information about the study, tailored to their level of understanding. A copy of the consent form and information sheet has been provided to each patient. The investigator has ensured that the patient had sufficient time to ask questions and clarify doubts regarding the study details.

The preparation of the informed consent form is the responsibility of the Principal Investigator. It has included all elements required by the International Conference of Harmonization, current regulatory guidelines, and has complied with the Standards of Good Clinical Practice and ethical principles stemming from the Declaration of Helsinki.

The original signed informed consent form has been securely stored in a restricted access area under the custody of the Principal Investigator. A copy of the original signed consent form has been given to the patient.

The Data Controller for this study is the research team. Legit.Health, the Data Processor, is not responsible for the processing of the data included in the Software or its users. The storage and handling of data and photographs is aligned with the European Regulation 2016/679 of 27 April on the protection of natural persons with regard to the processing of personal data and the free movement of such data and the Organic Law 3/2018 of 5 December on the Protection of Personal Data and guarantee of digital rights. At the conclusion of the study, all information stored in the device will be completely and permanently deleted.

The device complies with current legislation on the protection and confidentiality of personal data. Appropriate technical and organizational security measures are in place to ensure the security of personal data and prevent its alteration, loss, unauthorized processing or access.

Investigators and Administrative Structure of Clinical Research

Brief Description

This CIP has been conducted in conjunction between the Dermatology Department of the Hospital Universitario Puerta del Hierro de Majadahonda and the AI Labs Group S.L. (Legit.Health).

Investigators

Principal investigator

• Dr. Gastón Roustán Gullón (Dermatology Department)

Collaborators

• Hospital Universitario Puerta del Hierro de Majadahonda
  • Dr. Ángel Rosell Díaz
• Centro de salud de Majadahonda
  • Dr. Esther Minguela
• Centro de salud de Pozuelo
  • Dr. Fernando León

Center

• Hospital Universitario Puerta del Hierro de Majadahonda
• Centro de salud de Majadahonda
• Centro de salud de Pozuelo

External Organization

No additional organizations, beyond those previously mentioned, contributed to the clinical research. The study was conducted with the collaboration and resources of the specified entities.

Promoter and Monitor

Instituto de Investigación Sanitaria Puerta de Hierro and AI Labs Group S.L.

Report Annexes

• Clinical Investigation Plan (CIP) can be found in R-TF-015-004_LEGIT.HEALTH_DAO_Derivación_PH_2022.
• Ethics Committee resolution can be found in the document Dictamen Favorable LEGIT.HEALTH_DAO_Derivación_PH_2022.pdf.
• Instructions For Use (IFU) can be found in the protocol.
• Questionnaires can also be found in the protocol.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001