R-TF-015-006 Clinical investigation report LEGIT.HEALTH_PH_2024

Product Identification

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.0.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product (True/False)	FALSE
Novel related clinical procedure (True/False)	FALSE
SRN	ES-MF-000025345

Promoter Identification and Contact

	Manufacturer data
Legal manufacturer name	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRN	ES-MF-000025345
Person responsible for regulatory compliance	Alfonso Medela, María Diez, Giulia Foglia
E-mail	office@legit.health
Phone	+34 638127476
Trademark	Legit.Health

Summary

Objectives

Primary objective

To validate that the information provided by device increases the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions

Secondary objectives

• To validate what percentage of cases should be referred according the HCP with the information provided by the device
• To validate what percentage of cases could be handled remotely with the information provided by the device

Design and Methods

Design

We developed a website for conducting the experiment. Primary care physicians were required to log in to the website initially. Subsequently, they were presented with a series of questions structured as follows:

1. Based on the provided image, what diagnosis do you consider most appropriate? This question was accompanied by anamnesis inquiries regarding allergies, ongoing treatments, and other relevant medical history.
2. Considering both the image and the analysis provided by the AI, what diagnosis do you deem most appropriate? In this instance, the same information from question 1 was supplemented with the top 5 diagnoses and their respective confidence levels, calculated by Legit.Health's diagnosis support algorithm based on the image.
3. Based on the AI's provided information, does this patient require a dermatology referral? Additional information such as the malignancy index and a referral recommendation by the tool were provided in this step.
4. According to the AI's provided information, can remote diagnosis and treatment be confirmed? No additional information was provided at this stage.

Each participant was presented with a total of 30 cases or images to review. These images had been previously confirmed by dermatologists and by anatomical pathology for cases of skin cancer. The conditions were distributed as follows:

Condition	ICD-11 code	Number of images
Atypical melanocytic nevus	2F20.1	2
Melanocytic nevus	2F20	3
Melanoma	2C30	5
Basal cell carcinoma	2C32	3
Urticaria	EB05	5
Pustular psoriasis	EA90.4	2
Actinic keratosis	EK90.0	2
Plaque psoriasis	EA90.0	3
Hidradenitis suppurativa	ED92.0	5

All this information was recorded in a database and exported to a CSV file, which was subsequently used for further analysis. The analysis was conducted using the Python programming language and statistical libraries such as numpy and pandas. Statistical measures, including the P-Value, were calculated to either accept or reject the hypothesis. Additionally, metrics were calculated based on pathology and medical specialization. Atypical melanocytic nevus and melanocytic nevus were evaluated together under a single category: XH4L78 Pigmented Nevus, referred to in the following sections simply as "nevus".

Results

Diagnosis

In this study, nine primary care doctors reviewed all the images.

We conducted a McNemar test in order to analyze if the information provided by Legit.Health impacts on the healthcare professionals diagnostics'. Overall, primary care doctors demonstrated an accuracy of 72.96%, which notably increased to 82.22% with the integration of Legit.Health. Our analysis, supported by a very low p-value (0.0001), revealed the following key findings:

• Legit.Health reinforces practitioners' diagnostics in 69.26% of the cases
• Legit.Health improves practitioners' diagnostics in 12.96% of the cases
• Legit.Health does not have any impact on practitioners' diagnostics in 14.07% of the cases
• Legit.Health has a negative impact on practitioners' diagnostics in 3.70% of the cases

An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:

Condition	Accuracy (%)	Accuracy with Legit.Health (%)	Relative difference (%)	p-value
Actinic keratosis	55.56	83.33	49.98	0.125
Pustular psoriasis	5.56	22.22	299.64	0.25
Plaque psoriasis	96.30	96.30	0.00	1.00
Nevus	75.56	77.78	2.91	1.00
Melanoma	86.67	91.11	5.12	0.69
Urticaria	73.33	91.11	24.24	0.02
Hidradenitis suppurativa	64.44	80.00	24.14	0.04
Basal cell carcinoma	91.67	88.89	-3.00	1.00

Referral

In assessing the impact of Legit.Health on referrals, our findings revealed that 48.89% of cases did not necessitate a referral.

Condition	Do not require referral (%)
Nevus	60.00
Melanoma	2.22
Basal cell carcinoma	7.41
Urticaria	88.89
Pustular psoriasis	11.11
Actinic keratosis	33.33
Plaque psoriasis	81.48
Hidradenitis suppurativa	71.11

Remote consultations

Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results show that 60.74% of the cases can be handled remotely.

Conducting a Pearson's chi-squared test on the necessity for referrals and teleconsultations, we concluded with 95% confidence that a strong association exists between referrals and remote consultations. Specifically:

• 36.67% of the cases do not require referral and can have follow-up remotely
• 12.22% of the cases do not require referral but require an in-person appointment
• 24.07% of the cases require referral and remote consultation
• 27.04% of the cases require a referral in addition to an in-person appointment

Pathology	Can be handled remotely (%)
Nevus	55.56
Melanoma	42.22
Basal cell carcinoma	44.44
Urticaria	75.56
Pustular psoriasis	38.89
Actinic keratosis	61.11
Plaque psoriasis	81.48
Hidradenitis suppurativa	75.56

Conclusions

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialties could be effectively managed remotely.

Reduction of referral and use of remote consultation

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

References

1. González-López G, Descalzo-Gallego MÁ, Arias-Santiago S, Molina-Leyva A, Gilaberte Y, Fernández-Crehuet P, Husein-El Ahmed H, Viera-Ramírez A, Fernández-Peñas P, Taberner R, García-Doval I. Derivación de pacientes en consulta de dermatología y de teledermatología en España. Estudio DIADERM. Actas Dermo-Sifiliográficas. 2019 Mar 1;110(2):146-52.
2. Pagani K, Lukac D, Olbricht SM, Aronson MD, Benneyan JC, Fernandez L, Salant T, Schiff GD, Shafiq U, Sternberg SB, McGee JS. Urgent referrals from primary care to dermatology for lesions suspicious for skin cancer: patterns, outcomes, and need for systems improvement. Archives of dermatological research. 2023 Jul;315(5):1397-400.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001