R-TF-015-006 Clinical investigation report

Research Title

Non-invasive prospective Pilot in a Live Environment for the Improvement of the diagnosis of skin pathologies in primary care and dermatology

Product Identification

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.1.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product (True/False)	FALSE
Novel related clinical procedure (True/False)	FALSE
SRN	ES-MF-000025345

Promoter Identification and Contact

	Manufacturer data
Legal manufacturer name	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRN	ES-MF-000025345
Person responsible for regulatory compliance	Alfonso Medela, Saray Ugidos
E-mail	office@legit.health
Phone	+34 638127476
Trademark	Legit.Health

Identification of sponsors

Sanitas Hospitales SA

Clinical Investigation Plan (CIP) Identification

• Title: Non-invasive prospective Pilot in a Live Environment for the Improvement of the diagnosis of skin pathologies in primary care and dermatology.
• Protocol code: LEGIT.HEALTH_SAN_2024.
• Study design: Prospective observational analytical and cross-sectional study.
• Product under investigation: Legit.Health Plus.
• Version and date: Version 1.0, 2024-07-04.

Public Access Database

Please note that the database used in this study is not publicly accessible due to privacy and confidentiality considerations.

Research Team

Principal investigator

• Dr Antonio Martorell Calatayud

Collaborators

• Medical staff
  • Dr Adriana Vasconcelos
  • Dr María Porriño
  • Dr Gustazo
  • Dr Josefina Sanz
  • Dr Andrés
  • Dr Gerald Selda
  • Dr Helena Bahachille
  • Dr Mitchell Ignacio Leal Betancourt
  • Dr Marianela del Castillo
  • Dr María Pilar Martínez Marta
  • Dr Nadia Hayajneh Carrillo
  • Dr Carmen Arsuaga
  • Dr Elena Sánchez Largo
  • Dr María Gómez
  • Dr Pedro Ortega Lozano
• AI Labs Group S.L.
  • Mr Alfonso Medela
  • Mr Taig Mac Carthy

Centre

• This study was conducted remotely by sending the images to the participating dermatologists.

Compliance Statement

The clinical investigation was perforfed according to the Clinical Investigation Plan (CIP) and other applicable guidances and regulations. This includes compliance with:

• Harmonized standard UNE-EN ISO 14155:2021
• Regulation (EU) 2017/745 on medical devices (MDR)
• Harmonized standard UNE-EN ISO 13485:2016s
• Regulation (EU) 2016/679 (GDPR).
• Spanish Organic Law 3/2018 on the Protection of Personal Data and guarantee of digital rights`.

All data processing within the device is carried out in accordance with the highest standards of data protection and privacy. Patient information is managed in an encrypted manner to ensure confidentiality and security.

The research team assumes the role of Data Controller, responsible for the collection and management of study data. Legit.Health acts as the Data Processor and is not involved in the processing of patient data.

The storage and transfer of data comply with European data protection regulations. At the conclusion of the study, all information stored in the device will be permanently and securely deleted.

The device employs robust technical and organizational security measures to safeguard personal data against unauthorized access, alteration, loss, or processing.

Report Date

October 18th, 2024

Report author(s)

The full name, the ID and the signature for the authorship, as well as the approval process of this document, can be found in the verified commits at the repository. This information is saved alongside the digital signature, to ensure the integrity of the document.

Table of contents

Table of contents

• Research Title
• Product Identification
• Promoter Identification and Contact
• Identification of sponsors
• Clinical Investigation Plan (CIP) Identification
• Public Access Database
• Research Team
  • Principal investigator
  • Collaborators
  • Centre
• Compliance Statement
• Report Date
• Report author(s)
• Table of contents
• Abbreviations and Definitions
• Summary
  • Title
  • Introduction
  • Objectives
  • Population
  • Design and Methods
  • Results
  • Conclusions
• Introduction
• Material and methods
  • Product Description
  • Clinical Investigation Plan
  • Sample size
• Results
  • Initiation and Completion Date
  • Subject and Investigational Product Management
  • Subject Demographics
  • Clinical Investigation Plan (CIP) Compliance
  • Analysis
• Discussion and Overall Conclusions
  • Conclusions
• References
  • Implications for Future Research
  • Limitations of Clinical Research
  • Ethical Aspects of Clinical Research
• Investigators and Administrative Structure of Clinical Research
  • Brief Description
  • Investigators
  • External Organization
  • Promoter and Monitor
• Report Annexes

Abbreviations and Definitions

• AE: Adverse Event
• AEMPS: Spanish Agency of Medicines and Medical Devices
• AEP: Adverse Reaction to Product
• AUC: Area Under the ROC Curve
• CAD: Computer-Aided Diagnosis
• CMD: Data Monitoring Committee
• CIP: Clinical Investigation Plan
• CUS: Clinical Utility Questionnaire
• DLQI: Dermatology Quality of Life Index
• GCP: Standards of Good Clinical Practice
• ICH: International Conference of Harmonization
• IFU: Instructions For Use
• IRB: Institutional Review Board
• N/A: Not Applicable
• NCA: National Competent Authority
• PI: Principal Investigator
• SAE: Serious Adverse Events
• SAEP: Serious Adverse Event to Product
• SUAEP: Serious and Unexpected Adverse Event to the Product
• SUS: System Usability Scale

Summary

Title

Non-invasive prospective Pilot in a Live Environment for the Improvement of the diagnosis of skin pathologies in primary care and dermatology.

Introduction

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care practitioners and dermatologists remain substantial, with concordance rates between 57% and 65.52%. These discrepancies can lead to misdiagnoses or incorrect referrals, which affect the quality of life of patients with skin pathologies.

This investigation pretends to assess whether the use of the device can improve the accuracy of the diagnosis of different skin conditions. 16 HCPs were recruited and each one of them will be presented with 29 images to review. This research follows rigorous ethical standards and adherence to regulatory guidelines, this research holds significant potential in revolutionizing dermatological diagnostics.

Objectives

Primary objective

• To validate that the information provided by the device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Secondary objectives

• To validate what percentage of cases should be referred according to the HCP with the information provided by the device.
• To validate what percentage of cases could be handled remotely with the information provided by the device.
• Confirm that the use of the medical device is perceived by specialists as being of great clinical utility.

Population

In this study, the population will consist of primary care practitioners and dermatologists. A minimum of 15 HCPs will be selected.

Design and Methods

Design

This investigation proceeds as follows:

Healthcare practitioners recruitment' and image presentation

We developed a website to conduct the experiment. Participants, including dermatologists and primary care practitioners, were required to log in to the website initially. Subsequently, they were presented with a series of questions structured as follows:

1. Based on the provided image, what diagnosis do you consider most appropriate? This question was accompanied by anamnesis inquiries regarding allergies, ongoing treatments, and other relevant medical history.
2. Considering both the image and the analysis provided by the AI, what diagnosis do you deem most appropriate? In this instance, the same information from question 1 was supplemented with the top 5 diagnoses and their respective confidence levels, calculated by the device's diagnosis support algorithm based on the image.
3. Based on the the information provided by the AI, does this patient require a dermatology referral? Additional information such as the malignancy index and a referral recommendation by the tool were provided in this step.
4. According to the information provided by the AI, can remote diagnosis and treatment be confirmed? No additional information was provided at this stage.

Each participant was presented with 29 cases or images to review. These images had been previously confirmed by dermatologists and by anatomical pathology for cases of skin cancer. The conditions were distributed as follows:

Condition	Number of images
Dermatitis	5
Melanoma	3
Alopecia	2
Urticaria	1
Granuloma annulare	1
Seborrheic keratosis	1
Herpes	2
Tiña	2
Psoriasis	3
Onychomycosis	2
Acne	2
Pressure ulcer	1
Nevus	4

All this information was recorded in a database and exported to a CSV file, which was subsequently used for further analysis. The analysis was conducted using the Python programming language and statistical libraries such as numpy and pandas. Statistical measures, including the P-value, were calculated to either accept or reject the hypothesis. Additionally, metrics were calculated based on pathology and medical specialization.

Number of subjects

A total of 16 HCPs (10 primary care practitioners and 6 dermatologists) were recruited in this study.

Initiation Date

June 01^rd, 2024

Completion Date

October 10^rd, 2024

Duration

The study spanned a total duration of 4 months, encompassing the time needed for tasks such as the creation of the website, database closure and editing, data analysis, and the preparation of the final study report after the recruitment of the last subject.

Methods

The study employed a prospective observational analytical and cross-sectional design to evaluate whether the use of the device improved the accuracy of the diagnosis of different skin pathologies by HCPs. This investigation encompassed a diverse cohort of 16 HCPs (dermatologists and primary care practitioners). Data collection included the accuracy of the diagnosis of different pathologies with and without the use of the device: if the device reinforces, improves, worsens or has no impact on the diagnosis made by the HCPs. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards.

Results

For this study, 16 HCPs (10 primary care practitioners and 6 dermatologists) were included. Among them, 12 completed the entire process, while the remaining 4 reviewed a partial number of images, specifically 28, 15, 9, and 1 respectively.

When HCPs did not use the device they showed an accuracy of 68.08% on their diagnosis. On the other hand, when they integrated the device on their diagnosis, the accuracy increased to 88.78%. This increase was higher for primary care practitioners, who showed an accuracy of 62.90% rising to 89.92%. Meanwhile for dermatologists. The diagnostic accuracy increased from 76.47% to 86.93%.

In relation to the referrals, 58.1% of cases did not need a referral, although this varied between primary care practitioners and dermatologists. Finally, regarding remote consultations, 55.11% of cases can be handled remotely. However as previously seen in the referrals, this percentage variated between primary care practitioners and dermatologists.

Regarding the Clinical Utility questionnaire, six dermatologists, six general practitioners, and three family doctors completed the questionnaire, taking an average of 2 minutes and 38 seconds. Healthcare professionals rated the utility of the data with an average score of 7.3. Dermatologists and general practitioners also rated it 7.3, while family doctors rated it 7. The design and usability of the device received an average score of 8, with all respondents giving it the same rating, indicating strong consensus.

Conclusions

In conclusion, the device demonstrated significant enhancements in diagnostic accuracy and efficiency for healthcare professionals across multiple skin conditions. Primary care practitioners, in particular, benefited from an increase in diagnostic precision, making them more equipped to manage cases remotely or without specialist referrals. Dermatologists also experienced an improvement in diagnostic accuracy, especially for complex or less commonly encountered conditions. The tool facilitated remote management for over half of the cases, with healthcare professionals finding it effective and user-friendly, which reduced patient handling time to under ten minutes in many cases. These results suggest that the device has strong potential as a diagnostic support tool, especially in primary care settings and remote consultations, streamlining workflow while supporting accurate, timely patient care.

Introduction

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care practitioners and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions. The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.

Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care practitioners. The device, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of diverse pathologies. This pilot study aims to evaluate whether the use of the device can increase the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Material and methods

Product Description

This section contains a short summary of the device. A complete description of the intended purpose, including device description, can be found in the record Legit.Health Plus description and specifications.

Product description

The device is a computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended purpose

The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing:

• quantification of intensity, count, extent of visible clinical signs
• interpretative distribution representation of possible International Classification of Diseases (ICD) categories.

Intended previous uses

No specific intended use was designated in prior stages of development.

Product changes during clinical research

The device maintained a consistent performance and features throughout the entire clinical research process. No alterations or modifications were made during this period.

Clinical Investigation Plan

Objectives

This study aims to validate that the information provided by the device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Design (type of research, assessment criteria, methods, active group, and control group)

This is a prospective observational and cross-sectional study. The study does not involve an active or control group, as the HCPs will be their control group, firstly without using the device and after making their diagnosis, using the device to assess if they want to change their diagnosis or keep it. The assessment criteria include the assessment of different images with pathologies such as GPP or HS and their diagnosis. The study employs a variety of methods, including data collection through websites or photograph analysis.

Ethical considerations

This study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. As applicable, approval from the relevant Ethics Committee was obtained prior to the initiation of the study. When applicable, modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Participants were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The PI was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Participants were provided with a copy of their signed consent form for their records.

Data Quality Assurance

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Subject Population

This study enrolled both primary care practitioners and dermatologists. At the end of the study, 16 HCPs were enrolled. These doctors should evaluate 29 images of different skin pathologies and diagnose them.

Sample size

This study aims to evaluate whether the use of the device improves diagnostic accuracy by at least 10% among healthcare professionals, providing evidence to support its integration into clinical practice. To achieve this, 16 HCPs will participate in the study, a sample size determined through statistical power analysis to achieve 80% power at an alpha level of 0.05 for detecting a minimum 10% improvement in diagnostic accuracy. This design balances practical feasibility with robust statistical sensitivity.

Each HCP will review 29 dermatological images representing a diverse range of skin pathologies. These images will be carefully selected to encompass varying severity levels and types of conditions, ensuring that the device's performance is tested across a wide spectrum of clinical scenarios. The decision to use 29 images per HCP reflects a balance between ensuring sufficient variability in case profiles and minimizing cognitive fatigue, thereby maintaining the quality of the evaluations.

Limiting the study to 16 HCPs allows for a focused and manageable cohort, reducing inter-observer variability and enhancing the consistency of results. This approach also enables precise estimates of diagnostic accuracy while ensuring that each HCP gains substantial practical experience with the device. The smaller, well-defined group of experienced healthcare professionals ensures high-quality data collection, providing reliable insights into the device’s impact on clinical decision-making.

By combining a smaller HCP cohort with a robust volume of cases, this study design ensures statistically reliable and clinically relevant findings. The combination of diverse patient profiles and concentrated HCP evaluations strengthens the validity of the conclusions, supporting a meaningful assessment of the device's potential to improve diagnostic accuracy.

Inclusion Criteria

• Board-certified primary care practitioners and dermatologists, regardless of their professional experience.
• High-quality images of patients with different skin conditions.

Exclusion Criteria

• Low-quality images of patients which can not be properly analyzed.

Statistical Analysis

To estimate the correlation between diagnoses with and without using the device, we analyzed the concordance between diagnoses for both primary care practitioners and dermatologists. We analyzed if the device reinforced the diagnosis of the HCPs (after observing the results of the device, the doctor maintains the diagnosis when his answer matches that of the solution) if the device improves the practitioner's diagnosis (the doctor changes the diagnosis when his answer does not match that of the solution), if it has no impact (The doctor does not change the diagnosis even though his answer does not match that of the solution) and if it has a negative impact (The doctor changes the diagnosis for an answer that does not match that of the solution). We also analyzed if HCPs considered if this case should be referred to the specialist or it could be handled remotely.

Results

Initiation and Completion Date

June 01^rd, 2024 October 10^rd, 2024

Subject and Investigational Product Management

A total of 16 HCPs were recruited for this study, 10 primary care practitioners and 6 dermatologists. Each HCP was presented with 29 images in order to review them and make a diagnosis of the pathology. Furthermore, they should indicate if these patients should be referred to dermatology or can be handled in primary care, or also if it can be handled remotely. This meticulous selection process ensures the integrity and validity of the study's findings. The investigational products were stored and handled following strict protocols. This included proper storage conditions, handling procedures, and documentation of product usage. The accountability and traceability of investigational products were rigorously maintained throughout the study.

Subject Demographics

The study did not place specific emphasis on gender, age, or nationality as primary factors of investigation. Instead, it encompassed a diverse HCP cohort with different specialities.

Clinical Investigation Plan (CIP) Compliance

The study adhered to all aspects outlined in the CIP. This ensured that the research was conducted in accordance with established protocols, procedures, and ethical standards. Any deviations from the CIP were duly documented and appropriately addressed. Compliance with the CIP was rigorously monitored throughout the study to uphold the integrity and validity of the research findings.

Analysis

Primary Analyses

Diagnosis

In this study, a total of 16 healthcare professionals (HCPs) participated, comprising 10 primary care doctors and 6 dermatologists. Among them, 12 completed the entire process, while the remaining 4 reviewed a partial number of images, specifically 28, 15, 9, and 1 respectively.

We conducted a McNemar test in order to analyze the information provided by the device impacts on the healthcare professionals' diagnostics. Overall, the HCPs demonstrated an accuracy of 68.08%, which notably increased to 88.78% with the integration of the device. Our analysis, supported by an extremely low p-value (0.00000000000000000022), revealed the following key findings:

• The device bolstered practitioners' diagnostics in 67.83% of cases.
• It enhanced practitioners' diagnostics in 20.95% of cases.
• The device had no discernible impact on practitioners' diagnostics in 10.97% of cases.
• In a small fraction of cases (0.25%), the device had a negative impact on practitioners' diagnostics.

When focusing on primary care practitioners, the disparity was even more pronounced, with an initial accuracy of 62.9% rising to 89.92% with the device. Consequently:

• The device reinforced practitioners' diagnostics in 62.50% of cases.
• It improved practitioners' diagnostics in 27.42% of cases.
• The device did not affect practitioners' diagnostics in 9.68% of cases.
• In a negligible 0.40% of cases, the device had a negative impact.

For dermatologists, the diagnostic accuracy increased from 76.47% to 86.93%:

• The device reinforced practitioners' diagnostics in 76.47% of cases.
• It improved practitioners' diagnostics in 10.46% of cases.
• The device had no impact on practitioners' diagnostics in 13.07% of cases.
• There were no instances where the device had a negative impact.

The results of diagnostic accuracy are summarized in the table below:

HCP	Accuracy (%)	Accuracy with the device (%)	Difference (%)
All specialties	68.08	88.78	20.70
Primary care	62.90	89.92	27.02
Dermatologist	76.47	86.93	10.46

An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:

Condition	Accuracy (%)	Accuracy with the device (%)	Difference (%)	p-value
Pressure ulcer	76.92	100.00	23.08	0.25000
Urticaria	85.71	100.00	14.29	0.50000
Tinea	62.96	100.00	37.04	0.00195
Seborrheic keratosis	33.33	73.33	40	0.03125
Psoriasis	40.00	77.50	37.5	0.00006
Onychomycosis	76.92	88.46	11.54	0.25000
Nevus	70.37	83.33	12.96	0.01562
Melanoma	66.67	85.71	19.04	0.00781
Herpes	100.00	100.00	0	1.00000
Granuloma annulare	33.33	93.33	60	0.00391
Dermatitis	68.06	93.06	25	0.00004
Alopecia	96.55	100.00	3.45	1.00000
Acne	65.38	69.23	3.85	1.00000

We separated the results per pathology into two tables, one for primary care doctors and another for dermatologists.

Primary care practitioners

Condition	Accuracy (%)	Accuracy with the device (%)
Pressure ulcer	75.00	100.00
Urticaria	88.89	100.00
Tinea	58.82	100.00
Seborrheic keratosis	22.22	77.78
Psoriasis	24.00	72.00
Onychomycosis	81.25	100.00
Nevus	72.73	84.85
Melanoma	65.38	92.31
Herpes	100.00	100.00
Granuloma annulare	11.11	88.89
Dermatitis	53.33	91.11
Alopecia	94.44	100.00
Acne	68.75	75.00

Dermatologists

Because of the quantity of images per pathology and the total number of dermatologists involved, the evidence is inconclusive and may be biased.

Condition	Accuracy (%)	Accuracy with the device (%)
Pressure ulcer	80.00	100.00
Urticaria	80.00	100.00
Tinea	70.00	100.00
Seborrheic keratosis	50.00	66.67
Psoriasis	66.67	86.67
Onychomycosis	70.00	70.00
Nevus	66.67	80.95
Melanoma	68.75	75.00
Herpes	100.00	100.00
Granuloma annulare	66.67	100.00
Dermatitis	92.59	96.30
Alopecia	100.00	100.00
Acne	60.00	60.00

Referral

In assessing the impact of the device on referrals, our findings revealed that 58.1% of cases did not necessitate a referral. However, this percentage varied slightly to 60.89% for primary care doctors and 53.59% for dermatologists. These results are summarized in the table below:

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	168	41.9	233	58.1
Primary care	97	39.11	151	60.89
Dermatologists	71	46.41	82	53.59

Nevus

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	45	83.33	9	16.67
Primary care	27	81.82	6	18.18
Dermatologists	18	85.71	3	14.29

Melanoma

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	42	100.00	0	0.00
Primary care	26	100.00	0	0.00
Dermatologists	16	100.00	0	0.00

Alopecia

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	19	65.52	10	34.48
Primary care	9	50.00	9	50.00
Dermatologists	10	90.91	1	9.09

Urticaria

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	2	14.29	12	85.71
Primary care	0	0.00	9	100.00
Dermatologists	2	40.00	3	60.00

Granuloma annulare

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	7	46.67	8	53.33
Primary care	5	55.56	4	44.44
Dermatologists	2	33.33	4	66.67

Seborrheic keratosis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	26.67	11	73.33
Primary care	2	22.22	7	77.78
Dermatologists	2	33.33	4	66.67

Herpes

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	0	0.00	28	100.00
Primary care	0	0.00	17	100.00
Dermatologists	0	0.00	11	100.00

Tinea

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	0	0.00	27	100.00
Primary care	0	0.00	17	100.00
Dermatologists	0	0.00	10	100.00

Psoriasis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	25	62.5	15	37.5
Primary care	19	76.00	6	24.00
Dermatologists	6	40.00	9	60.00

Onychomycosis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	15.38	22	84.62
Primary care	1	6.25	15	93.75
Dermatologists	3	30.00	7	70.00

Acne

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	4	15.38	22	84.62
Primary care	0	0.00	16	100.00
Dermatologists	4	40.00	6	60.00

Pressure ulcer

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	2	15.38	11	84.62
Primary care	1	12.5	7	87.50
Dermatologists	1	20.00	4	80.00

Dermatitis

HCP	Should refer	Should refer (%)	Should not refer	Should not refer (%)
All specialties	14	19.44	58	80.56
Primary care	7	15.56	38	84.44
Dermatologists	7	25.93	20	74.07

Remote consultations

Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results are presented in the subsequent table:

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	221	55.11	180	44.89
Primary care	148	59.68	100	40.32
Dermatologists	73	47.71	80	52.29

Conducting a Pearson's chi-squared test on the necessity for referrals and teleconsultations, we concluded with 95% confidence that a strong association exists between referrals and remote consultations. Specifically:

• 50.62% of cases do not require a referral and can be followed up remotely.
• 7.48% of cases do not require a referral but necessitate an in-person appointment.
• 4.49% of cases require a referral and remote consultation.
• 37.41% of cases require a referral in addition to an in-person appointment.

However, when analyzing depending on HCP speciality, we observed differing patterns. For dermatologists:

• 43.14% of cases do not require a referral and can be followed up remotely.
• 10.46% of cases do not require a referral but require an in-person appointment.
• 4.58% of cases require a referral and remote consultation.
• 41.83% of cases require a referral in addition to an in-person appointment.

In contrast, for primary care practitioners:

• 55.24% of cases do not require a referral and can be followed up remotely.
• 5.65% of cases do not require a referral but necessitate an in-person appointment.
• 4.44% of cases require a referral and remote consultation.
• 34.68% of cases require a referral in addition to an in-person appointment.

Nevus

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	7	12.96	47	87.04
Primary care	6	18.18	27	81.82
Dermatologists	1	4.76	20	95.24

Melanoma

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	1	2.38	41	97.62
Primary care	0	0.00	26	100.0
Dermatologists	1	6.25	15	93.75

Alopecia

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	15	51.72	14	48.28
Primary care	13	72.22	5	27.78
Dermatologists	2	18.18	9	81.82

Urticaria

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	11	78.57	3	21.43
Primary care	9	100.00	0	0.00
Dermatologists	2	40.00	3	60.00

Granuloma annulare

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	4	26.67	11	73.33
Primary care	3	33.33	6	66.67
Dermatologists	1	16.67	5	83.33

Seborrheic keratosis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	9	60.00	6	40.00
Primary care	4	44.44	5	55.56
Dermatologists	4	66.67	2	33.33

Herpes

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	27	96.43	1	3.57
Primary care	16	94.12	1	5.88
Dermatologists	11	100.00	0	0.00

Tinea

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	24	88.89	3	11.11
Primary care	14	82.35	3	17.65
Dermatologists	10	100.00	0	0.00

Psoriasis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	17	42.50	23	57.50
Primary care	8	32.00	17	68.00
Dermatologists	9	60.00	6	40.00

Onychomycosis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	18	69.23	8	30.77
Primary care	15	93.75	1	6.25
Dermatologists	3	30.00	7	70.00

Acne

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	20	76.92	6	23.08
Primary care	13	81.25	3	18.75
Dermatologists	7	70.00	3	30.00

Pressure ulcer

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	10	76.92	3	23.08
Primary care	7	87.50	1	12.50
Dermatologists	3	60.00	2	40.00

Dermatitis

HCP	Can be handled remotely	Can be handled remotely (%)	Can't be handled remotely	Can't be handled remotely (%)
All specialties	58	80.56	14	19.44
Primary care	39	86.67	6	13.33
Dermatologists	19	70.37	8	29.63

HCP feedback

Six dermatologists, six general practitioners, and three family doctors completed the questionnaire, taking an average of 2 minutes and 38 seconds.

Healthcare professionals rated the utility of the data with an average score of 7.3. Dermatologists and general practitioners also rated it 7.3, while family doctors rated it 7. Regarding the need for additional data, three respondents indicated they did not require more information. There was a consensus among participants on the necessity for additional patient history data, particularly regarding symptoms.

67% per cent of practitioners reported that they could manage consultations in 5 to 10 minutes using the device, while 20% believed they could do so in under 5 minutes. In relation to confidence in making clinical decisions remotely using the device, the average score was 6.4. Dermatologists rated it at 6.2, general practitioners at 6.3, and family doctors at 7.

When asked about potential uses for the medical device, the most common response (mentioned 12 times) was for diagnosis support in remote consultations and video calls. Eight respondents mentioned its application for automatic referrals to dermatology and remote follow-up of previously diagnosed patients. Seven respondents mentioned its use for diagnosis support in in-person consultations and automatic triage.

The design and usability of the device received an average score of 8, with all respondents giving it the same rating, indicating strong consensus.

Adverse Events and Adverse Reactions to the Product

Throughout the study, no adverse events or adverse reactions related to the investigational product have been observed. Participants have not experienced any negative reactions or side effects associated with the use of the product. This indicates a favourable safety profile of the investigational product in the context of this study.

Product Deficiencies

No deficiencies in the product have been observed during the course of this study. As a result, no corrective actions have been deemed necessary. The product has demonstrated consistent performance in accordance with the study's objectives.

Subgroup Analysis for Special Populations

In the context of the analyzed pathologies, no special population subgroups were identified for this study. The research primarily focused on the specified HCP population without subgroup differentiation.

Discussion and Overall Conclusions

Conclusions

The device significantly enhanced diagnostic accuracy across all healthcare professionals, increasing it from 68.08% to 88.78%. Notably, primary care practitioners experienced a substantial improvement in diagnostic accuracy, rising from 62.90% to 89.92%, with the integration of the device. Similarly, dermatologists saw their diagnostic accuracy improve from 76.47% to 86.93% with the utilization of the device.

The impact of the device varied across different skin conditions, demonstrating significant improvements in cases such as tinea, granuloma annulare, and seborrheic keratosis. It's noteworthy that all conditions included in the test showed increased accuracy.

Approximately 58.1% of cases did not necessitate a referral, with minor differences observed between primary care practitioners and dermatologists. Additionally, a substantial portion of cases (55.11%) across all specialities could be effectively managed remotely, with primary care practitioners exhibiting a slightly higher percentage compared to dermatologists.

When examining the results categorized by pathology, notable discrepancies emerged between primary care doctors and dermatologists in certain cases. It's worth mentioning that the limited number of dermatologists impedes reaching a definitive conclusion at the pathology level. For instance, unanimous agreement exists among experts on the feasibility of remote management for conditions like acne, herpes, and tinea. Conversely, there's a consensus favouring in-person consultations for melanoma, granuloma annulare, and nevus.

Furthermore, dermatologists recommend referring patients with nevus, melanoma, and alopecia to their care, while suggesting that primary care doctors can manage most other pathologies effectively.

Regarding healthcare professional (HCP) feedback, the device is viewed as a useful and user-friendly tool for its intended purpose, with users highlighting its role in diagnosis support during remote consultations. 87% of HCPs believe they could manage a patient in less than 10 minutes, despite the average consultation time typically being at least 15 minutes.

Reduction of referral and use of remote consultation

Previous works reported that 66% of the patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of the patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, as reported above, we found out that according to the primary HCP with the information provided by the device 39.11% of the cases should be referred, which is a 41.34% lower than the aforementioned referral rates. In addition, our results improve the remote consultation rates, which suggests that diagnosis support tools can help foster remote consultations.

References

1. González-López G, Descalzo-Gallego MÁ, Arias-Santiago S, Molina-Leyva A, Gilaberte Y, Fernández-Crehuet P, Husein-El Ahmed H, Viera-Ramírez A, Fernández-Peñas P, Taberner R, García-Doval I. Derivación de pacientes en consulta de dermatología y de teledermatología en España. Estudio DIADERM. Actas Dermo-Sifiliográficas. 2019 Mar 1;110(2):146-52.
2. Pagani K, Lukac D, Olbricht SM, Aronson MD, Benneyan JC, Fernandez L, Salant T, Schiff GD, Shafiq U, Sternberg SB, McGee JS. Urgent referrals from primary care to dermatology for lesions suspicious for skin cancer: patterns, outcomes, and need for systems improvement. Archives of dermatological research. 2023 Jul;315(5):1397-400.

Implications for Future Research

The positive outcomes of this study pave the way for several avenues of future research. Firstly, helping to improve the diagnosis of different skin pathologies, which can significantly impact the quality of life of patients who suffer from them.

On the other hand, exploring the integration of artificial intelligence and machine learning techniques to refine the tool's diagnostic capabilities warrants attention. This could lead to even more accurate and reliable assessments, potentially revolutionizing the field of dermatology.

Additionally, conducting long-term studies to evaluate the impact of the device on patient outcomes, including treatment adherence and quality of life, would provide a comprehensive understanding of its broader clinical implications.

Limitations of Clinical Research

The main limitations of the pilot include several factors that may influence the perception and effectiveness of the AI-based device. Firstly, the acceptance and trust of healthcare professionals in these emerging technologies can vary significantly. The device's effectiveness may be compromised if users are not fully convinced of its accuracy or usefulness, thereby affecting the overall perception of its performance.

Additionally, image quality is crucial for the device's performance. Issues such as low-quality photographs, errors in cropping lesions, or variations in lighting and focus can deteriorate the quality of the data received by the system, which may negatively influence the evaluation and perception of its effectiveness by the researchers.

Variability in image conditions is also an important aspect to consider. Differences in lighting, colour, shape, size, and focus of the images, along with the number of images available for each patient, can affect the accuracy of the results. High variability in images of the same patient or an insufficient number of representative images can decrease the expected diagnostic accuracy of the device.

Additionally, the consistency of investigators in using the device is crucial. Variations in how diligently investigators use the device can impact the pilot's findings. If the investigators are not consistent in their use of the device, it can lead to unreliable results and affect the overall assessment of its efficacy.

Another limitation is the Hawthorne effect, where pilot subjects may change their behaviour simply because they know they are being observed. This awareness can influence their decisions and actions within the pilot, potentially skewing the results and not accurately reflecting how the device would be used in a non-study environment.

Ethical Aspects of Clinical Research

This study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. As applicable, approval from the relevant Ethics Committee was obtained prior to the initiation of the study. When applicable, modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Participants were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The PI was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Participants were provided with a copy of their signed consent form for their records.

Data quality assurance

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Investigators and Administrative Structure of Clinical Research

Brief Description

This CIP has been conducted between the participating medical staff and AI Labs Group S.L. (the manufacturer).

Investigators

Principal investigator

• Dr Antonio Martorell Calatayud

Collaborators

• Medical staff
  • Dr Adriana Vasconcelos
  • Dr María Porriño
  • Dr Gustazo
  • Dr Josefina Sanz
  • Dr Andrés
  • Dr Gerald Selda
  • Dr Helena Bahachille
  • Dr Mitchell Ignacio Leal Betancourt
  • Dr Marianela del Castillo
  • Dr María Pilar Martínez Marta
  • Dr Nadia Hayajneh Carrillo
  • Dr Carmen Arsuaga
  • Dr Elena Sánchez Largo
  • Dr María Gómez
  • Dr Pedro Ortega Lozano
• AI Labs Group S.L.
  • Mr Alfonso Medela
  • Mr Taig Mac Carthy

External Organization

No additional organizations, beyond those previously mentioned, contributed to the clinical research. The study was conducted with the collaboration and resources of the specified entities.

Promoter and Monitor

Sanitas Hospitales SA

Report Annexes

• Instructions For Use (IFU) can be found in the protocol.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-005