R-TF-015-006 Clinical investigation report LEGIT.HEALTH_SAN_2024

Quick access

Jump to Clinical Performance, Efficacy, and Safety.

Research Title

Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of skin pathologies in primary care and dermatology

Product Identification

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.0.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product (True/False)	FALSE
Novel related clinical procedure (True/False)	FALSE
SRN	ES-MF-000025345

Promoter Identification and Contact

	Manufacturer data
Legal manufacturer name	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRN	ES-MF-000025345
Person responsible for regulatory compliance	Alfonso Medela, María Diez, Giulia Foglia
E-mail	office@legit.health
Phone	+34 638127476
Trademark	Legit.Health

Identification of sponsors

Sanitas

Clinical Investigation Plan (CIP) Identification

• Title: Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of skin pathologies in primary care and dermatology.
• Protocol code: LEGIT.HEALTH_SAN_2024.
• Study design: Prospective observational analytical and cross-sectional study.
• Product under investigation: Legit.Health Plus.
• Version and date: Version 1.0, 2024-07-04.

Public Access Database

Please note that the database used in this study is not publicly accessible due to privacy and confidentiality considerations.

Research Team

Principal investigator

• Dr. Antonio Martorell Calatayud

Collaborators

• Medical staff
  • Dr. Adriana Vasconcelos
  • Dr. María Porriño
  • Dr. Gustazo
  • Dr. Josefina Sanz
  • Dr. Andrés
  • Dr. Gerald Selda
  • Dr. Helena Bahachille
  • Dr. Mitchell Ignacio Leal Betancourt
  • Dr. Marianela del Castillo
  • Dr. María Pilar Martínez Marta
  • Dr. Nadia Hayajneh Carrillo
  • Dr. Carmen Arsuaga
  • Dr. Elena Sánchez Largo
  • Dr. María Gómez
  • Dr. Pedro Ortega Lozano
• AI Labs Group S.L.
  • Mr. Alfonso Medela
  • Mr. Taig Mac Carthy

Center

• This study was conducted remotely and sending the images to the participating dermatologists.

Compliance Statement

The clinical investigation was perforfed according to the Clinical Investigation Plan (CIP) and other applicable guidances and regulations. This includes compliance with:

• Harmonized standard UNE-EN ISO 14155:2021
• Regulation (EU) 2017/745 on medical devices (MDR)
• Harmonized standard UNE-EN ISO 13485:2016s
• Regulation (EU) 2016/679 (GDPR).
• Spanish Organic Law 3/2018 on the Protection of Personal Data and guarantee of digital rights`.

All data processing within the device is carried out in accordance with the highest standards of data protection and privacy. Patient information is managed in an encrypted manner to ensure confidentiality and security.

The research team assumes the role of Data Controller, responsible for the collection and management of study data. Legit.Health acts as the Data Processor and is not involved in the processing of patient data.

The storage and transfer of data comply with European data protection regulations. At the conclusion of the study, all information stored in the device will be permanently and securely deleted.

The device employs robust technical and organizational security measures to safeguard personal data against unauthorized access, alteration, loss, or processing.

Report Date

October 18th, 2024

Report Author(s)

Table of contents

• Research Title
• Product Identification
• Promoter Identification and Contact
• Identification of sponsors
• Clinical Investigation Plan (CIP) Identification
• Public Access Database
• Research Team
  • Principal investigator
  • Collaborators
  • Center
• Compliance Statement
• Report Date
• Report Author(s)
• Abbreviations and Definitions
• Summary
  • Title
  • Introduction
  • Objectives
  • Population
  • Design and Methods
  • Results
  • Conclusions
• Introduction
• Material and methods
  • Product Description
  • Clinical Investigation Plan
• Results
  • Initiation and Completion Date
  • Subject and Investigational Product Management
  • Subject Demographics
  • Clinical Investigation Plan (CIP) Compliance
  • Analysis
• Discussion and Overall Conclusions
  • Conclusions
• References
  • Implications for Future Research
  • Limitations of Clinical Research
  • Ethical Aspects of Clinical Research
• Investigators and Administrative Structure of Clinical Research
  • Brief Description
  • Investigators
  • External Organization
  • Promoter and Monitor
• Report Annexes

Abbreviations and Definitions

• CAD: Computer-Aided Diagnosis
• CIP: Clinical Investigation Plan
• CUS: Clinical Utility Questionnaire
• SUS: System Usability Scale
• GCP: Standards of Good Clinical Practice
• ICH: International Conference of Harmonization
• PI: Principal Investigator
• DLQI: Dermatology Quality of Life Index
• ICH: International Conference of Harmonization
• AUC: Area Under the ROC Curve

Summary

Title

Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of skin pathologies in primary care and dermatology.

Introduction

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. These discrepancies can lead to misdiagnoses or incorrect referrals, which affect the quality of life of patients with skin pathologies. This investigation pretends to assess whether the use of the Legit.Health medical device can improve the accuracy on the diagnosis of different skin conditions. 16 HCPs were recruited and each one of them will be presented with 29 images to review. This research follows rigorous ethical standards, and adherence to regulatory guidelines, this research holds significant potential in revolutionizing dermatological diagnostics.

Objectives

Primary objective

• To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Secondary objectives

• To validate what percentage of cases should be refered according the HCP with the information provided by the device.
• To validate what percentage of cases could be handled remotely with the information provided by the device.
• Confirm that the use of the medical device is perceived by specialists as being of great clinical utility.

Population

In this study the population will consist of primary care physicians and dermatologists. A minimum of 15 physicians will be selected.

Design and Methods

Design

This investigation proceeds as follows:

Healthcare practicioners recruitment and image presentation

We developed a website for conducting the experiment. Participants, including dermatologists and primary care physicians, were required to log in to the website initially. Subsequently, they were presented with a series of questions structured as follows:

1. Based on the provided image, what diagnosis do you consider most appropriate? This question was accompanied by anamnesis inquiries regarding allergies, ongoing treatments, and other relevant medical history.
2. Considering both the image and the analysis provided by the AI, what diagnosis do you deem most appropriate? In this instance, the same information from question 1 was supplemented with the top 5 diagnoses and their respective confidence levels, calculated by Legit.Health's diagnosis support algorithm based on the image.
3. Based on the AI's provided information, does this patient require a dermatology referral? Additional information such as the malignancy index and a referral recommendation by the tool were provided in this step.
4. According to the AI's provided information, can remote diagnosis and treatment be confirmed? No additional information was provided at this stage.

Each participant was presented with a total of 29 cases or images to review. These images had been previously confirmed by dermatologists and by anatomical pathology for cases of skin cancer. The conditions were distributed as follows:

Condition	Number of images
Dermatitis	5
Melanoma	3
Alopecia	2
Urticaria	1
Granuloma annulare	1
Seborrheic keratosis	1
Herpes	2
Tiña	2
Psoriasis	3
Onychomycosis	2
Acne	2
Pressure ulcer	1
Nevus	4

All this information was recorded in a database and exported to a CSV file, which was subsequently used for further analysis. The analysis was conducted using the Python programming language and statistical libraries such as numpy and pandas. Statistical measures, including the P-Value, were calculated to either accept or reject the hypothesis. Additionally, metrics were calculated based on pathology and medical specialization.

Number of subjects

A total of 15 HCPs (10 primary care physicians and 6 dermatologists) were recruited in this study.

Initiation Date

June 01^rd, 2024

Completion Date

October 10^rd, 2024

Duration

The study spanned a total duration of 4 months, encompassing the time needed for tasks such as the creation of the website, database closure and editing, data analysis, and the preparation of the final study report after the recruitment of the last subject.

Methods

The study employed a prospective observational analytical and cross-sectional design to evaluate whether the use of Legit.Health improved the accuracy on the diagnosis of different skin pathologies by HCPs. This investigation encompassed a diverse cohort of 16 HCPs (dermatologists and primary care physicians). Data collection included the accuracy on the diagnosis of different pathologies with and without the use of Legit.Health: if Legit.Health reinforces, improves, worses or has no impact over the diagnosis made by the physicians. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards.

Results

For this study, 16 HCPs (10 primary care physicians and 6 dermatologists) were included. Among them, 12 completed the entire process, while the remaining 4 reviewed a partial number of images, specifically 28, 15, 9, and 1 respectively.

When HCPs did not use Legit.Health they showed an accuracy of 68.08% on their diagnosis. On the other hand, when they integrated Legit.Health on their diagnosis, the accuracy increased to 88.78%. This increase was higher for primary care physicians, who showed an accuracy of 62.90% rising to 89.92%%. Meanwhile for dermatologists. The diagnostic accuracy increased from 76.47% to 86.93%.

In relation with the referrals, 58.1% of cases did not needed a referral, although this variated between primary care physicians and dermatologists. Finally, regarding remote consultations 55.11% of cases can be handled remotely. But as previously seen in the referrals, this percentage variated between primary care physicians and dermatologists.

Regarding the Clinical Utility questionnaire, six dermatologists, six general practitioners, and three family doctors completed the questionnaire, taking an average of 2 minutes and 38 seconds. Healthcare professionals rated the utility of the data with an average score of 7.3. Dermatologists and general practitioners also rated it 7.3, while family doctors rated it 7. The design and usability of Legit.Health received an average score of 8, with all respondents giving it the same rating, indicating strong consensus.

Conclusions

In conclusion, Legit.Health demonstrated significant enhancements in diagnostic accuracy and efficiency for healthcare professionals across multiple skin conditions. Primary care physicians, in particular, benefited from an increase in diagnostic precision, making them more equipped to manage cases remotely or without specialist referrals. Dermatologists also experienced improvement in diagnostic accuracy, especially for complex or less commonly encountered conditions. The tool facilitated remote management for over half of the cases, with healthcare professionals finding it effective and user-friendly, which reduced patient handling time to under ten minutes in many cases. These results suggest that Legit.Health has strong potential as a diagnostic support tool, especially in primary care settings and remote consultations, streamlining workflow while supporting accurate, timely patient care.

Introduction

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions. The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.

Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care physicians. Legit.Health, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of diverse patholohies. This pilot study aims to evaluate whether the use of the Legit.Health medical device can increase the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Material and methods

Product Description

This section contains a short summary of the device. A complete description of the intended purpose, including device description, can be found in the record Legit.Health Plus description and specifications.

Product description

The device is computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended purpose

The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing:

• quantification of intensity, count, extent of visible clinical signs
• interpretative distribution representation of possible International Classification of Diseases (ICD) classes.

Intended previous uses

No specific intended use was designated in prior stages of development.

Product changes during clinical research

The device maintained a consistent performance and features throughout the entire clinical research process. No alterations or modifications were made during this period.

Clinical Investigation Plan

Objectives

This study aims to validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Design (type of research, assessment criteria, methods, active group, and control group)

This is a prospective observational and cross-sectional study. The study does not involve an active or control group, as the physicians will be their own control group, firstly without using Legit.Health and after making their diagnosis, using Legit.Health to assess if they want to change their diagnosis or keep it. The assessment criteria include the assessment of different images with pathologies such as GPP or HS and their diagnosis. The study employs a variety of methods, including data collection through websites or photograph analysis.

Ethical considerations

The conduct of this study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. Approval from the relevant Ethics Committee was obtained prior to the initiation of the study. Any modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Patients were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The Principal Investigator was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Patients were provided with a copy of their signed consent form for their records.

Data Quality Assurance

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Subject Population

This study enrolled both primary care physicians and dermatologists. At the end of the study, 16 HCPs were enrolled. These doctors should evaluate 29 images of different skin pathologies and diagonse them.

Inclusion Criteria

• Board-certified primary care physicians and dermatologists, regardless of their professional experience.
• High quality images of patients different skin conditions.

Exclusion Criteria

• Low quality images of patients which can not be properly analyzed.

Statistical Analysis

For the purpose of estimating the correlation between diagnoses with and without using Legit.Health, we analyzed the concordance between diagnoses for both primary care physicians and dermatologists. We analyzed if Legit.Health reinforced the diagnosis of the physicians (after observing the results of Legit.Health, the doctor maintains the diagnosis when his answer matches that of the solution), if Legit.Health improved the practicioner's diagnosis (the doctor changes the diagnosis when his answer does not match that of the solution), if it has no impact (The doctor does not change the diagnosis even though his answer does not match that of the solution) and if it has a negative impact (The doctor changes the diagnosis for an answer that does not match that of the solution). We also analyzed if physicians considered if this case should be referred to the specialist or it could be handled remotely.

Results

Initiation and Completion Date

June 01^rd, 2024 October 10^rd, 2024

Subject and Investigational Product Management

A total of 16 HCPs were recruited for this study, 10 primary care physicians and 6 dermatologists. Each physician was presented with 29 images in order to review them and make a diagnosis of the pathology. Furthermore, they should indicate if this patients should be referred to dermatology or can be handled in primary care, or also if it can be handled remotely. This meticulous selection process ensures the integrity and validity of the study's findings. The investigational products were stored and handled following strict protocols. This included proper storage conditions, handling procedures, and documentation of product usage. The accountability and traceability of investigational products were rigorously maintained throughout the study.

Subject Demographics

The study did not place specific emphasis on gender, age, or nationality as primary factors of investigation. Instead, it encompassed a diverse physician cohort with different specialties.

Clinical Investigation Plan (CIP) Compliance

The study adhered to all aspects outlined in the CIP. This ensured that the research was conducted in accordance with established protocols, procedures, and ethical standards. Any deviations from the CIP were duly documented and appropriately addressed. The compliance with the CIP was rigorously monitored throughout the duration of the study to uphold the integrity and validity of the research findings.

Analysis

Primary Analyses

Diagnosis

In this study, a total of 16 healthcare professionals (HCPs) participated, comprising 10 primary care doctors and 6 dermatologists. Among them, 12 completed the entire process, while the remaining 4 reviewed a partial number of images, specifically 28, 15, 9, and 1 respectively.

We conducted a McNemar test in order to analyze if the information provided by Legit.Health impacts on the healthcare professionals diagnostics'. Overall, the HCPs demonstrated an accuracy of 68.08%, which notably increased to 88.78% with the integration of Legit.Health. Our analysis, supported by an extremely low p-value (0.00000000000000000022), revealed the following key findings:

• Legit.Health bolstered practitioners' diagnostics in 67.83% of cases.
• It enhanced practitioners' diagnostics in 20.95% of cases.
• Legit.Health had no discernible impact on practitioners' diagnostics in 10.97% of cases.
• In a small fraction of cases (0.25%), Legit.Health had a negative impact on practitioners' diagnostics.

When focusing on primary care physicians, the disparity was even more pronounced, with an initial accuracy of 62.9% rising to 89.92% with Legit.Health. Consequently:

• Legit.Health reinforced practitioners' diagnostics in 62.50% of cases.
• It improved practitioners' diagnostics in 27.42% of cases.
• Legit.Health did not affect practitioners' diagnostics in 9.68% of cases.
• In a negligible 0.40% of cases, Legit.Health had a negative impact.

For dermatologists, the diagnostic accuracy increased from 76.47% to 86.93%:

• Legit.Health reinforced practitioners' diagnostics in 76.47% of cases.
• It improved practitioners' diagnostics in 10.46% of cases.
• Legit.Health had no impact on practitioners' diagnostics in 13.07% of cases.
• There were no instances where Legit.Health had a negative impact.

The results of diagnostic accuracy are summarized in the table below:

HCP	Accuracy (%)	Accuracy with Legit.Health (%)	Difference (%)
All specialties	68.08	88.78	20.70
Primary care	62.90	89.92	27.02
Dermatologist	76.47	86.93	10.46

An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:

Condition	Accuracy (%)	Accuracy with Legit.Health (%)	Difference (%)	p-value
Pressure ulcer	76.92	100.00	23.08	0.25000
Urticaria	85.71	100.00	14.29	0.50000
Tinea	62.96	100.00	37.04	0.00195
Seborrheic keratosis	33.33	73.33	40	0.03125
Psoriasis	40.00	77.50	37.5	0.00006
Onychomycosis	76.92	88.46	11.54	0.25000
Nevus	70.37	83.33	12.96	0.01562
Melanoma	66.67	85.71	19.04	0.00781
Herpes	100.00	100.00	0	1.00000
Granuloma annulare	33.33	93.33	60	0.00391
Dermatitis	68.06	93.06	25	0.00004
Alopecia	96.55	100.00	3.45	1.00000
Acne	65.38	69.23	3.85	1.00000

We separated the results per pathology into two tables, one for primary care doctors and another for dermatologists.

Primary care doctors

Condition	Accuracy (%)	Accuracy with Legit.Health (%)
Pressure ulcer	75.00	100.00
Urticaria	88.89	100.00
Tinea	58.82	100.00
Seborrheic keratosis	22.22	77.78
Psoriasis	24.00	72.00
Onychomycosis	81.25	100.00
Nevus	72.73	84.85
Melanoma	65.38	92.31
Herpes	100.00	100.00
Granuloma annulare	11.11	88.89
Dermatitis	53.33	91.11
Alopecia	94.44	100.00
Acne	68.75	75.00

Dermatologists

Because of the quantity of images per pathology and the total number of dermatologists involved, the evidence is inconclusive and may be biased.

Condition	Accuracy (%)	Accuracy with Legit.Health (%)
Pressure ulcer	80.00	100.00
Urticaria	80.00	100.00
Tinea	70.00	100.00
Seborrheic keratosis	50.00	66.67
Psoriasis	66.67	86.67
Onychomycosis	70.00	70.00
Nevus	66.67	80.95
Melanoma	68.75	75.00
Herpes	100.00	100.00
Granuloma annulare	66.67	100.00
Dermatitis	92.59	96.30
Alopecia	100.00	100.00
Acne	60.00	60.00

Referral

In assessing the impact of Legit.Health on referrals, our findings revealed that 58.1% of cases did not necessitate a referral. However, this percentage varied slightly to 60.89% for primary care doctors and 53.59% for dermatologists. These results are summarized in the table below:

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	168	41.9	233	58.1
Primary care	97	39.11	151	60.89
Dermatologists	71	46.41	82	53.59

Nevus

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	45	83.33	9	16.67
Primary care	27	81.82	6	18.18
Dermatologists	18	85.71	3	14.29

Melanoma

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	42	100.00	0	0.00
Primary care	26	100.00	0	0.00
Dermatologists	16	100.00	0	0.00

Alopecia

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	19	65.52	10	34.48
Primary care	9	50.00	9	50.00
Dermatologists	10	90.91	1	9.09

Urticaria

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	2	14.29	12	85.71
Primary care	0	0.00	9	100.00
Dermatologists	2	40.00	3	60.00

Granuloma annulare

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	7	46.67	8	53.33
Primary care	5	55.56	4	44.44
Dermatologists	2	33.33	4	66.67

Seborrheic keratosis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	26.67	11	73.33
Primary care	2	22.22	7	77.78
Dermatologists	2	33.33	4	66.67

Herpes

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	0	0.00	28	100.00
Primary care	0	0.00	17	100.00
Dermatologists	0	0.00	11	100.00

Tinea

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	0	0.00	27	100.00
Primary care	0	0.00	17	100.00
Dermatologists	0	0.00	10	100.00

Psoriasis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	25	62.5	15	37.5
Primary care	19	76.00	6	24.00
Dermatologists	6	40.00	9	60.00

Onychomycosis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	15.38	22	84.62
Primary care	1	6.25	15	93.75
Dermatologists	3	30.00	7	70.00

Acne

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	15.38	22	84.62
Primary care	0	0.00	16	100.00
Dermatologists	4	40.00	6	60.00

Pressure ulcer

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	2	15.38	11	84.62
Primary care	1	12.5	7	87.50
Dermatologists	1	20.00	4	80.00

Dermatitis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	14	19.44	58	80.56
Primary care	7	15.56	38	84.44
Dermatologists	7	25.93	20	74.07

Remote consultations

Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results are presented in the subsequent table:

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	221	55.11	180	44.89
Primary care	148	59.68	100	40.32
Dermatologists	73	47.71	80	52.29

Conducting a Pearson's chi-squared test on the necessity for referrals and teleconsultations, we concluded with 95% confidence that a strong association exists between referrals and remote consultations. Specifically:

• 50.62% of cases do not require referral and can be followed up remotely.
• 7.48% of cases do not require referral but necessitate an in-person appointment.
• 4.49% of cases require referral and remote consultation.
• 37.41% of cases require a referral in addition to an in-person appointment.

However, when analyzing by HCP specialty, we observed differing patterns. For dermatologists:

• 43.14% of cases do not require referral and can be followed up remotely.
• 10.46% of cases do not require referral but require an in-person appointment.
• 4.58% of cases require referral and remote consultation.
• 41.83% of cases require a referral in addition to an in-person appointment.

In contrast, for primary care physicians:

• 55.24% of cases do not require referral and can be followed up remotely.
• 5.65% of cases do not require referral but necessitate an in-person appointment.
• 4.44% of cases require referral and remote consultation.
• 34.68% of cases require a referral in addition to an in-person appointment.

Nevus

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	7	12.96	47	87.04
Primary care	6	18.18	27	81.82
Dermatologists	1	4.76	20	95.24

Melanoma

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	1	2.38	41	97.62
Primary care	0	0.00	26	100.0
Dermatologists	1	6.25	15	93.75

Alopecia

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	15	51.72	14	48.28
Primary care	13	72.22	5	27.78
Dermatologists	2	18.18	9	81.82

Urticaria

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	11	78.57	3	21.43
Primary care	9	100.00	0	0.00
Dermatologists	2	40.00	3	60.00

Granuloma annulare

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	4	26.67	11	73.33
Primary care	3	33.33	6	66.67
Dermatologists	1	16.67	5	83.33

Seborrheic keratosis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	9	60.00	6	40.00
Primary care	4	44.44	5	55.56
Dermatologists	4	66.67	2	33.33

Herpes

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	27	96.43	1	3.57
Primary care	16	94.12	1	5.88
Dermatologists	11	100.00	0	0.00

Tinea

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	24	88.89	3	11.11
Primary care	14	82.35	3	17.65
Dermatologists	10	100.00	0	0.00

Psoriasis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	17	42.50	23	57.50
Primary care	8	32.00	17	68.00
Dermatologists	9	60.00	6	40.00

Onychomycosis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	18	69.23	8	30.77
Primary care	15	93.75	1	6.25
Dermatologists	3	30.00	7	70.00

Acne

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	20	76.92	6	23.08
Primary care	13	81.25	3	18.75
Dermatologists	7	70.00	3	30.00

Pressure ulcer

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	10	76.92	3	23.08
Primary care	7	87.50	1	12.50
Dermatologists	3	60.00	2	40.00

Dermatitis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	58	80.56	14	19.44
Primary care	39	86.67	6	13.33
Dermatologists	19	70.37	8	29.63

HCP feedback

Six dermatologists, six general practitioners, and three family doctors completed the questionnaire, taking an average of 2 minutes and 38 seconds.

Healthcare professionals rated the utility of the data with an average score of 7.3. Dermatologists and general practitioners also rated it 7.3, while family doctors rated it 7. Regarding the need for additional data, three respondents indicated they did not require more information. There was a consensus among participants on the necessity for additional patient history data, particularly regarding symptoms.

67% percent of practitioners reported that they could manage consultations in 5 to 10 minutes using Legit.Health, while 20% believed they could do so in under 5 minutes. On confidence in making clinical decisions remotely using Legit.Health, the average score was 6.4. Dermatologists rated it at 6.2, general practitioners at 6.3, and family doctors at 7.

When asked about potential uses for the medical device, the most common response (mentioned 12 times) was for diagnosis support in remote consultations and video calls. Eight respondents mentioned its application for automatic referrals to dermatology and remote follow-up of previously diagnosed patients. Seven respondents mentioned its use for diagnosis support in in-person consultations and automatic triage.

The design and usability of Legit.Health received an average score of 8, with all respondents giving it the same rating, indicating strong consensus.

Adverse Events and Adverse Reactions to the Product

Throughout the study, no adverse events or adverse reactions related to the investigational product have been observed. Participants have not experienced any negative reactions or side effects associated with the use of the product. This indicates a favorable safety profile of the investigational product in the context of this study.

Product Deficiencies

No deficiencies in the product have been observed during the course of this study. As a result, no corrective actions have been deemed necessary. The product has demonstrated consistent performance in accordance with the study's objectives.

Subgroup Analysis for Special Populations

In the context of the analyzed pathologies, no special population subgroups were identified for this study. The research primarily focused on the specified physicians population without subgroup differentiation.

Discussion and Overall Conclusions

Conclusions

Legit.Health significantly enhanced diagnostic accuracy across all healthcare professionals, increasing it from 68.08% to 88.78%. Notably, primary care physicians experienced a substantial improvement in diagnostic accuracy, rising from 62.90% to 89.92%, with the integration of Legit.Health. Similarly, dermatologists saw their diagnostic accuracy improve from 76.47% to 86.93% with the utilization of Legit.Health.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in cases such as tinea, granuloma annulare, and seborrheic keratosis. It's noteworthy that all conditions included in the test showed increased accuracy.

Approximately 58.1% of cases did not necessitate a referral, with minor differences observed between primary care physicians and dermatologists. Additionally, a substantial portion of cases (55.11%) across all specialties could be effectively managed remotely, with primary care physicians exhibiting a slightly higher percentage compared to dermatologists.

When examining the results categorized by pathology, notable discrepancies emerged between primary care doctors and dermatologists in certain cases. It's worth mentioning that the limited number of dermatologists impedes reaching a definitive conclusion at the pathology level. For instance, unanimous agreement exists among experts on the feasibility of remote management for conditions like acne, herpes, and tinea. Conversely, there's a consensus favoring in-person consultations for melanoma, granuloma annulare, and nevus.

Furthermore, dermatologists recommend referring patients with nevus, melanoma, and alopecia to their care, while suggesting that primary care doctors can manage most other pathologies effectively.

Regarding healthcare professional (HCP) feedback, Legit.Health is viewed as a useful and user-friendly tool for its intended purpose, with users highlighting its role in diagnosis support during remote consultations. 87% of HCPs believe they could manage a patient in less than 10 minutes, despite the average consultation time typically being at least 15 minutes.

Reduction of referral and use of remote consultation

Previous works reported that 66% of the patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of the patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, as reported above, we found out that according to the primary HCP with the information provided by the device 39.11% of the cases should be referred, which is a 41.34% lower than the aforementioned referral rates. In addition, our results improve the remote consultation rates, which suggests that diagnosis support tools can help fostering remote consultations.

References

1. González-López G, Descalzo-Gallego MÁ, Arias-Santiago S, Molina-Leyva A, Gilaberte Y, Fernández-Crehuet P, Husein-El Ahmed H, Viera-Ramírez A, Fernández-Peñas P, Taberner R, García-Doval I. Derivación de pacientes en consulta de dermatología y de teledermatología en España. Estudio DIADERM. Actas Dermo-Sifiliográficas. 2019 Mar 1;110(2):146-52.
2. Pagani K, Lukac D, Olbricht SM, Aronson MD, Benneyan JC, Fernandez L, Salant T, Schiff GD, Shafiq U, Sternberg SB, McGee JS. Urgent referrals from primary care to dermatology for lesions suspicious for skin cancer: patterns, outcomes, and need for systems improvement. Archives of dermatological research. 2023 Jul;315(5):1397-400.

Implications for Future Research

The positive outcomes of this study pave the way for several avenues of future research. Firstly, helping to improve the diagnosis of different skin pathologies, which can significantly impact the quality of life of patients who suffer from them.

On the other hand, exploring the integration of artificial intelligence and machine learning techniques to refine the tool's diagnostic capabilities warrants attention. This could lead to even more accurate and reliable assessments, potentially revolutionizing the field of dermatology.

Additionally, conducting long-term studies to evaluate the impact of the device on patient outcomes, including treatment adherence and quality of life, would provide a comprehensive understanding of its broader clinical implications.

Limitations of Clinical Research

The main limitations of the pilot include several factors that may influence the perception and effectiveness of the AI-based device. Firstly, the acceptance and trust of healthcare professionals in these emerging technologies can vary significantly. The device's effectiveness may be compromised if users are not fully convinced of its accuracy or usefulness, thereby affecting the overall perception of its performance.

Additionally, image quality is crucial for the device's performance. Issues such as low-quality photographs, errors in cropping lesions, or variations in lighting and focus can deteriorate the quality of the data received by the system, which may negatively influence the evaluation and perception of its effectiveness by the researchers.

Variability in image conditions is also an important aspect to consider. Differences in lighting, color, shape, size, and focus of the images, along with the number of images available for each patient, can affect the accuracy of the results. High variability in images of the same patient or an insufficient number of representative images can lead to a decrease in the expected diagnostic accuracy of the device.

Additionally, the consistency of investigators in using Legit.Health is crucial. Variations in how diligently investigators use the device can impact the pilot's findings. If the investigators are not consistent in their use of the device, it can lead to unreliable results and affect the overall assessment of its efficacy.

Another limitation is the Hawthorne effect, where pilot subjects may change their behavior simply because they know they are being observed. This awareness can influence their decisions and actions within the pilot, potentially skewing the results and not accurately reflecting how the device would be used in a non-study environment.

Ethical Aspects of Clinical Research

The conduct of this study adheres to international Good Clinical Practice standards and is in compliance with the Declaration of Helsinki in its latest active amendment. It also conforms to international and national rules and regulations.

The study did not require the approvation by an Ethics Committee due to its observational character and not allowing patients' identification.

The study has been conducted in accordance with European Regulation 2016/679, of 27 April, on the protection of natural persons with regard to the processing of personal data and the free movement of such data. Additionally, it adhere to the Spanish Organic Law 3/2018, of 5 December, on the Protection of Personal Data and guarantee of digital rights with regard to data processing. No data that allows the personal identification of subjects has been included, and all information has been managed in an encrypted manner.

Physicians have received comprehensive oral and written information about the study, tailored to their level of understanding. The main investigator has ensured that the pariticipants had sufficient time to ask questions and clarify any doubts regarding the study details.

The original signed informed consent form has been securely stored in a restricted access area under the custody of the Principal Investigator. A copy of the original signed consent form has been given to the patient.

The Data Controller for this study is the research team. Legit.Health, the Data Processor, is not responsible for the processing of the data included in the Software or its users. The storage and handling of data and photographs is aligned with the European Regulation 2016/679 of 27 April on the protection of natural persons with regard to the processing of personal data and the free movement of such data and the Organic Law 3/2018 of 5 December on the Protection of Personal Data and guarantee of digital rights. At the conclusion of the study, all information stored in the device will be completely and permanently deleted.

The device complies with current legislation on the protection and confidentiality of personal data. Appropriate technical and organizational security measures are in place to ensure the security of personal data and prevent its alteration, loss, unauthorized processing or access.

Investigators and Administrative Structure of Clinical Research

Brief Description

This CIP has been conducted in conjunction between the participating medical staff and AI Labs Group S.L. (Legit.Health).

Investigators

Principal investigator

• Dr. Antonio Martorell Calatayud

Collaborators

• Medical staff
  • Dr. Adriana Vasconcelos
  • Dr. María Porriño
  • Dr. Gustazo
  • Dr. Josefina Sanz
  • Dr. Andrés
  • Dr. Gerald Selda
  • Dr. Helena Bahachille
  • Dr. Mitchell Ignacio Leal Betancourt
  • Dr. Marianela del Castillo
  • Dr. María Pilar Martínez Marta
  • Dr. Nadia Hayajneh Carrillo
  • Dr. Carmen Arsuaga
  • Dr. Elena Sánchez Largo
  • Dr. María Gómez
  • Dr. Pedro Ortega Lozano
• AI Labs Group S.L.
  • Mr. Alfonso Medela
  • Mr. Taig Mac Carthy

External Organization

No additional organizations, beyond those previously mentioned, contributed to the clinical research. The study was conducted with the collaboration and resources of the specified entities.

Promoter and Monitor

Sanitas

Report Annexes

• Instructions For Use (IFU) can be found in the protocol.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001