Skip to main content
QMSQMS
QMS
  • Welcome to your QMS
  • Quality Manual
  • Procedures
  • Records
  • Legit.Health Plus Version 1.1.0.0
    • Index of Technical Documentation or Product File
    • Summary of Technical Documentation (STED)
    • Description and specifications
    • R-TF-001-007 Declaration of conformity
    • GSPR
    • Clinical
      • Evaluation
      • Investigation
        • 🗄 Drafts
        • R-015-005 Investigator's Brochure Legit.Health_acne
        • AIHS4 2025
        • BI 2024
        • COVIDX EVCDAO 2022
        • DAO Derivación O 2022
          • R-TF-015-004 Clinical investigation plan
          • R-TF-015-006 Clinical Investigation Report
          • R-TF-015-010 Annex E ISO 14155
        • DAO Derivación PH 2022
        • IDEI 2023
        • MC EVCDAO 2019
        • PH 2024
        • SAN 2024
      • R-TF-015-008 Clinical development plan
    • Design and development
    • Design History File
    • IFU and label
    • Post-Market Surveillance
    • Quality control
    • Risk Management
    • Usability and Human Factors Engineering
  • Legit.Health Plus Version 1.1.0.1
  • Licenses and accreditations
  • External documentation
  • Legit.Health Plus Version 1.1.0.0
  • Clinical
  • Investigation
  • DAO Derivación O 2022
  • R-TF-015-006 Clinical Investigation Report

R-TF-015-006 Clinical Investigation Report

Research Title​

Pilot Study for the Clinical Validatiof of an Artificial Intelligence Algorithm to optimise the appropriateness of dermatology referrals.

Product Identification​

Information
Device nameLegit.Health Plus (hereinafter, the device)
Model and typeNA
Version1.1.0.0
Basic UDI-DI8437025550LegitCADx6X
Certificate number (if available)MDR 792790
EMDN code(s)Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code65975
EU MDR 2017/745Class IIb
EU MDR Classification ruleRule 11
Novel product (True/False)FALSE
Novel related clinical procedure (True/False)FALSE
SRNES-MF-000025345

Promoter Identification and Contact​

Manufacturer data
Legal manufacturer nameAI Labs Group S.L.
AddressStreet Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia (Spain)
SRNES-MF-000025345
Person responsible for regulatory complianceAlfonso Medela, Saray Ugidos
E-mailoffice@legit.health
Phone+34 638127476
TrademarkLegit.Health

CIP Identification​

  • Title: Pilot Study for the Clinical Validation of an Artificial Intelligence Algorithm to optimise the appropriateness of dermatology referrals.
  • Protocol code: LEGIT.HEALTH_DAO_Derivation_O_2022
  • Study Design: Prospective observational analytical study of a clinical case series with longitudinal character.
  • Investigational Product: Legit.Health Plus
  • Version and Date: Version 1.0, date 2022-04-07

Public Access Database​

Please note that the database used in this study is not publicly accessible due to privacy and confidentiality considerations.

Research Team​

  • Osakidetza
    • Dr. Jesus Gardeazabal
    • Dr. Rosa Mª Izu
  • AI Labs Group S.L.
    • Alfonso Medela
    • Taig Mac Carthy

Compliance Statement​

The clinical investigation was perforfed according to the Clinical Investigation Plan (CIP) and other applicable guidances and regulations. This includes compliance with:

  • Harmonized standard UNE-EN ISO 14155:2021
  • Regulation (EU) 2017/745 on medical devices (MDR)
  • Harmonized standard UNE-EN ISO 13485:2016s
  • Regulation (EU) 2016/679 (GDPR).
  • Spanish Organic Law 3/2018 on the Protection of Personal Data and guarantee of digital rights.

All data processing within the device is carried out in accordance with the highest standards of data protection and privacy. Patient information is managed in an encrypted manner to ensure confidentiality and security.

The research team assumes the role of Data Controller, responsible for the collection and management of study data. Legit.Health acts as the Data Processor and is not involved in the processing of patient data.

The storage and transfer of data comply with European data protection regulations. At the conclusion of the study, all information stored in the device will be permanently and securely deleted.

The device employs robust technical and organizational security measures to safeguard personal data against unauthorized access, alteration, loss, or processing.

Report Date​

May 22, 2025.

The dates of versions, along with the version ID and the signature for the approval process for this document, can be found in the verified commits at the repository. This information is saved alongside the digital signature, to ensure the integrity of the document.

Report Author(s)​

The full name, the ID and the signature for the authorship, as well as the approval process of this document, can be found in the verified commits at the repository. This information is saved alongside the digital signature, to ensure the integrity of the document.

Table of contents​

Table of contents
  • Research Title
  • Product Identification
  • Promoter Identification and Contact
  • CIP Identification
  • Public Access Database
  • Research Team
  • Compliance Statement
  • Report Date
  • Report Author(s)
  • Table of contents
  • Abbreviations and Definitions
  • Summary
    • Title
    • Introduction
    • Objectives
      • Primary objective
      • Secondary objectives
    • Population
    • Sample size
    • Design and Methods
      • Design
      • Number of Subjects
      • Initiation Date
      • Completion Date
      • Duration
      • Methods
    • Results
    • Conclusions
  • Introduction
  • Materials and methods
    • Product Description
    • Clinical Research Plan
      • Objectives
      • Design
      • Ethical considerations
    • Data confidentiality
      • Data Quality Assurance
      • Subject Population (inclusion/exclusion criteria and sample size)
      • Treatment
      • Concomitant Medication/Treatment
      • Statistical Analysis
  • Results
    • Initiation and Completion Date
    • Subject and Investigational Product Management
    • Subject Demographics
    • Clinical Investigation Plan (CIP) Compliance
    • Analysis
      • Referral adequacy
      • Malignancy detection
      • Impact of image quality
      • Economic impact
      • Waiting list
      • Adverse Events and Adverse Reactions to the Product
      • Product Deficiencies
      • Subgroup Analysis for Special Populations
      • Accounting for All Subjects
  • Discussion and Overall Conclusions
    • Clinical Performance, Efficacy, and Safety
    • Clinical Risks and Benefits
    • Clinical Relevance
    • Specific Benefit or Special Precaution
      • Benefits
      • Precautions
    • Implications for Future Research
    • Limitations of Clinical Research
  • Ethical Aspects of Clinical Research
  • Investigators and Administrative Structure of Clinical Research
    • Brief Description
    • Investigators
      • Principal investigators
      • Collaborators
      • Centres
    • External Organisation
    • Promoter and Monitor
  • Report Annexes

Abbreviations and Definitions​

  • AE: Adverse Event
  • AEMPS: Spanish Agency of Medicines and Medical Devices
  • AEP: Adverse Reaction to Product
  • AUC: Area Under the ROC Curve
  • CAD: Computer-Aided Diagnosis
  • CMD: Data Monitoring Committee
  • CIP: Clinical Investigation Plan
  • CUS: Clinical Utility Questionnaire
  • DLQI: Dermatology Quality of Life Index
  • GCP: Standards of Good Clinical Practice
  • ICH: International Conference of Harmonization
  • IFU: Instructions For Use
  • IRB: Institutional Review Board
  • N/A: Not Applicable
  • NCA: National Competent Authority
  • PI: Principal Investigator
  • SAE: Serious Adverse Events
  • SAEP: Serious Adverse Event to Product
  • SUAEP: Serious and Unexpected Adverse Event to the Product
  • SUS: System Usability Scale

Summary​

This analytical prospective observational and longitudinal study of a series of clinical cases aims at confirming whether the device can effectively improve the process of referring patients from primary care to dermatology.

The study involves four primary care centres: Centro de Salud Sodupe-Güeñes, Centro de Salud Balmaseda, Centro de Salud Buruaga, and Centro de Salud Zurbaran. These centres all refer patients to Cruces and Basurto University Hospitals. A total of 127 patients where recruited, resulting in 198 dermatological images.

Title​

Pilot Study for the Clinical Validation of an Artificial Intelligence Algorithm to optimise the appropriateness of dermatology referrals.

Introduction​

Skin-related conditions present a significant challenge within primary care settings, frequently resulting in inconsistent diagnoses when compared to the evaluations conducted by dermatologists. This issue is made worse by a notable scarcity of dermatology specialists, particularly in less populous regions. This lack of specialists compels primary care practitioners to undertake dermatological evaluations, a field in which they may not have extensive expertise. Furthermore, the reliance on patient self-reporting during the diagnostic process can introduce a level of bias, potentially leading to inaccurate assessments.

To address these issues, Computer Aided Diagnosis (CAD) systems, including those using artificial intelligence, offer promising solutions for image interpretation and classification. Indeed, the purpose of this study is to clinically validate a device that increases the adequacy and efficiency of the referral process from primary care to dermatology and helps primary care practitioners perform dermatological assessments and triage.

By enhancing the accuracy and consistency of skin disease evaluations, the device has the potential to significantly improve the referral process, ensuring that patients are directed to specialist care when necessary. This, in turn, can contribute to closing the existing gap in dermatological care between primary care providers and specialized dermatology services, leading to better patient outcomes.

Objectives​

Primary objective​

To validate that the device is a valid tool for improving the adequacy of referrals to dermatology.

Secondary objectives​

  • To validate that the device reduces costs in secondary care.
  • To validate that the device reduces dermatology waiting lists.
  • To validate that the device optimizes clinical flow in Osakidetza, a northern Spanish public health service that provides healthcare to the population of the Basque Country.

Population​

Adult patients (≥ 18 years) with skin conditions assessed in the primary care service of health centres referring to Cruces and Basurto University Hospitals.

Sample size​

Considering a concordance rate of 55% between primary care and dermatology for referred lesions, a consensus has agreed that a 15% reduction in referrals from primary care to dermatology for malignant lesions would represent the minimum clinically important difference to justify a significant change in practice.

Therefore, to detect a 15% reduction in inappropriate referrals, a sample size of 400 referred lesions or 380 patients would be required, with 80% power at a 5% significance level.

However, this recruitment target was not achieved. The study was concluded with 127 patients and 198 dermatological images, with the final analysis conducted on 117 patients and 184 images after data refinement.

Design and Methods​

Design​

This is a prospective observational analytical study of a longitudinal clinical case series.

Number of Subjects​

127 patients were enrolled in the study, comprising 198 images of skin lesions.

The subjects were recruited from various health centres, with:

  • Centro de Salud Buruaga: 74 patients
  • Centro de Salud Balmaseda: 36 patients
  • Centro de Salud Zurbaranbarri: 15 patients
  • Centro de Salud Sodupe-Güeñes: 2 patients

Initiation Date​

November 23, 2022.

Completion Date​

May 6, 2025.

Duration​

The duration of the study was 2 years, 5 months and 13 days, or 896 days including both the start and end dates. This period includes the recruitment, the specialist to review photos, and data analysis

The duration is 7.5 times longer than the initially estimated duration of 4 months. This increase is due to the difficulty in recruiting patients. The reason for this increase is that in the case of the Sodupe health centre, changes in the research team occurred during the study, which, combined with a high clinical workload, hindered the proper recruitment of patients. For the other health centres, except for Buruaga, patient recruitment was limited due to a significant care burden and overlapping vaccination campaigns, which substantially impacted the ability to recruit participants for the study. An additional factor to consider is that, although some patients met the inclusion criteria and were invited to participate, they were not willing to consent to use of their images in the study. This resulted in delays in image collection and ultimately prevented the achievement of the expected sample size.

Methods​

Each variable was characterised using frequency distributions for qualitative variables and central tendency statistics, such as the mean and median, along with variability statistics like the standard deviation (S.D.) or interquartile range for quantitative variables, in accordance with their distributional characteristics.

Sensitivity, specificity, positive and negative predictive values (PPV and NPV) and likelihood ratios (LR+ and LR-) were calculated by comparing both the results obtained using the device and those obtained with the referral criteria of primary care practitioners with the criteria used by specialists, considered the gold standard.

Results​

In terms of economics, the device reduced unnecessary referrals by 38%, measured as the number of patients referred to dermatology who did not require a specialist visit. The potential savings from this reduction in unnecessary referrals is estimated at €1,200,000 annually and is a function of the number of patients. The 38% reduction in unnecessary referrals exceeds the minimum clinically important difference threshold of 15%, indicating a substantial impact on healthcare efficiency.

Furthermore, the device reduced cumulative waiting time by 56%. The reduction in waiting time potentially decreases average waiting times from 11.5 days to 5.0. This is a reductions of 6.5 days (56%) in the average waiting time for patients referred to dermatology.

For malignancy detection, the device achieved an AUC of 0.81, demonstrating strong performance and flexibility to adjust thresholds based on clinical priorities. This value is below from the 0.96 AUC achieved in the previous studies.

The device outperformed primary care practitioners in referral decision-making, achieving a sensitivity of 74% and a specificity of 67%, respectively, compared to the 45% sensitivity and 47% specificity for clinicians.

MetricDevicePrimary Care PhysiciansDifference
Sensitivity74%45%29%
Specificity67%47%30%

Conclusions​

The results show that the device significantly improves the accuracy of triage, increases the adequacy of referrals and reduces unnecessary specialist visits compared to standard care settings.

The literature shows that primary care doctors exhibit a notably low sensitivity (approximately 45%) in identifying cases requiring dermatology referral, suggesting a substantial risk of missing patients who genuinely need specialist care. By contrast, the device demonstrated higher sensitivity and specificity. This was translated into a reduction in unnecessary referrals and improved referral for those patients who did require it.

The device also cut cumulative waiting times by 56%, potentially decreasing average wait times from 11.5 to 5.0 days. This reduction in waiting times is particularly significant, as it can lead to faster access to care for patients with urgent dermatological conditions.

The specificity of 47% characteristic of primary care practitioners indicates a conservative referral approach. Although this may seem beneficial, this may delay the diagnosis of critical conditions due to the increase in waiting times for all patients. For this reason, the aid of the device in the decision-making process may help to reduce unnecessary referrals while maintaining a high level of sensitivity.

These findings support the benefits of integrating the device into teledermatology workflows. This has the potential to optimise costs, expedite access for urgent cases, and alleviate pressure on dermatology services — particularly when delays are driven by waiting list congestion.

In practice, dermatologists managing teledermatology referrals typically resolve cases through two main pathways: remote assessment or scheduling in-person visits. In our study, 74% of cases were escalated to in-person consultations, while 26% were resolved remotely. This reflects existing evidence that around 30% of primary care dermatology referrals are "banal" and could be managed at the primary care level, further supporting the need for improved triage tools like this device.

Introduction​

Skin-related diseases are a frequent reason for consultation in primary care1; some studies quantify it at approximately 5% of all consultations made, mainly by the working population. This is a considerable use of resources for health systems. For this reason, an efficient approach to referral and triage of cutaneous conditions is a key priority for many organisations.

Many studies show discrepancies in perspectives between the opinions of primary care practitioners and dermatologists, with percentages of agreement in their diagnoses ranging from 57%2 to 65.52%3 depending on the study. In general, primary care practitioners do not demonstrate adequate knowledge of skin diseases, their diagnosis and treatments4, partly due to the short training period in dermatology.

This limitation is also reflected in the effort and time required to estimate the degree of involvement of a patient or the stage of their condition. So much so, that it ends up being a very unrewarding task and can lead to poor adherence to the protocol and inadequate referrals.

Time consumption is of particular concern given that the number of medical professionals, especially in dermatology, is not sufficient for the current demand. Access of the general population to a dermatology specialist is complicated, due to the low number (3 dermatologists per 100,000 inhabitants)5, making it even more difficult in small population centres. Because of this, screening for dermatologic lesions must typically be performed by primary care practitioners, whose diagnostic capacity is even lower and can increase the risk of misdiagnosis.

In this regard, the literature shows a discordance of 55% to 65% between the primary care practitioner and the specialist3 and studies confirm several expected features: common dermatological diseases are often unrecognised or misdiagnosed by non-dermatologists, due to the particular profiles of common diagnoses in this activity (drug-induced rash, fungal infections)6.

In addition to these inherent limitations, one must also consider the risk of bias in cases where the preliminary examination is performed by the patient. This is especially true in cases where the patient knows that the treatment they receive will be determined by the information they provide. In addition, the medical team lacks the means to ensure that the values reported by the patient are true, which precludes external verification.

In recent years, there has been an increasing demand to develop Computer-Aided Diagnosis (CAD) systems that facilitate the detection of different conditions through algorithms. Among these devices, the device Legit.Health emerges as a leading provider of artificial intelligence and digital image processing. Image processing based on complex pattern recognition systems makes it possible for the physician to interpret the information contained in the medical image with much less difficulty. Advances in image recognition and artificial intelligence have led to innovations in the diagnosis of all types of conditions. It has been demonstrated that through artificial intelligence algorithms, it is possible to classify photographs of lesions with a level of competence comparable to that of a medical expert7 8.

Indeed, artificial intelligence medical devices present a huge advance that not only brings reliability to the documentation process, but also allows greater precision when assessing a condition, triaging its urgency and measuring visual signs.

Materials and methods​

Product Description​

This section contains a short summary of the device. A complete description of the intended purpose, including device description, can be found in the record Legit.Health Plus description and specifications.

Product description​

The device is a computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended purpose​

The device is a computational software-only medical device intended to support health care providers in the assessment of skin structures, enhancing efficiency and accuracy of care delivery, by providing:

  • quantification of intensity, count, extent of visible clinical signs
  • interpretative distribution representation of possible International Classification of Diseases (ICD) categories.

Intended previous uses​

No specific intended use was designated in prior stages of development.

Product changes during clinical research​

The device maintained a consistent performance and features throughout the entire clinical research process. No alterations or modifications were made during this period.

Clinical Research Plan​

This study includes a series of clinical cases to confirm whether the device can effectively improve the process of referring patients from primary care to dermatology.

Moreover, this study also includes an analysis of the cost reduction when using the device, as well as an analysis of the waiting list reduction and clinical flow optimisation in Osakidetza.

The study involves four primary care centres: Centro de Salud Sodupe-Güeñes, Centro de Salud Balmaseda, Centro de Salud Buruaga, and Centro de Salud Zurbaran. These centres all send patients to Cruces and Basurto University Hospitals.

Objectives​

The primary objective is to validate that the device is fit-for-purpose for optimising the adequacy of dermatology referrals.

Additionally, this study also aimed at validating that the device helps to reduce costs in healthcare, reduce dermatology waiting lists, and optimise clinical flows.

Design​

This is a prospective observational analytical study of a longitudinal clinical case series. The study does not involve an active or control group, as it is focused on the evaluation of the device in a real-world clinical setting.

The assessment criteria are based on photograph submission through the device and the study is based on the photograph analysis.

Ethical considerations​

This study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. As applicable, approval from the relevant Ethics Committee was obtained prior to the initiation of the study. When applicable, modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Participants were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The PI was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Participants were provided with a copy of their signed consent form for their records.

Data confidentiality​

Current legislation will be complied with in terms of data confidentiality protection (European Regulation 2016/679, of 27 April, on the protection of natural persons with regard to the processing of personal data and the free movement of such data and Organic Law 3/2018, of 5 December, on Personal Data Protection and guarantee of digital rights). For this purpose, when applicable, each participant will receive an alphanumeric identification code in the study that will not include any data allowing personal identification (coded CRD). The Principal Investigator will have an independent list that will allow the connection of the identification codes of the patients participating in the study with their clinical and personal data. This document will be filed in a secure area with restricted access, under the custody of the Principal Investigator and will never leave the centre.

Once the paper CRDs are completed and closed by the Principal Investigator, the data will be transferred to a database.

As in the CRDs, the Database will comply with current legislation in terms of data confidentiality protection (European Regulation 2016/679, of 27 April, on the protection of natural persons about the processing of personal data and the free movement of such data and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and guarantee of digital rights) in which no data allowing personal identification of patients will be included.

Data Quality Assurance​

The Principal Investigator is responsible for reviewing and approving the protocol, signing the Principal Investigator commitment, guaranteeing that the persons involved in the centre will respect the confidentiality of patient information and protect personal data, and reviewing and approving the final study report together with the sponsor. All the clinical members of the research team assess the eligibility of the patients in the study, inform and request written informed consent, collect the source data of the study in the clinical record and transfer them to the Data Collection Notebook (DCN) or Data Collection Forms (CRF).

Subject Population (inclusion/exclusion criteria and sample size)​

The study enrolled patients who fulfilled the recruitment criteria. As a result, we recruited 127 patients from different health centres: Buruaga recruited 74 patients, Balmaseda 36 patients, Zurbaranbarri 15 patients, and Sodupe 2 patients.

Among all the 127 patients, we collected a total of 198 images using smartphones and dermatoscopes. Overall, patients with pigmented lesions tend to have more images on average because they include both clinical and dermatoscopic pictures.

Inclusion Criteria​
  • Patients with skin conditions.
  • Patients aged 18 years or older.
  • Patients who have signed the informed consent for the study.
Exclusion Criteria​
  • Patient who, at the investigator's discretion, will not comply with the study procedures.

Treatment​

Patients participating in this study did not receive any specific treatment as part of the research protocol.

Concomitant Medication/Treatment​

Patients continued their regular prescribed medications and treatments as directed by their primary healthcare providers. No additional medications or treatments were administered as part of this study.

Statistical Analysis​

The study included 127 patients. However, 10 patients where excluded, due to the lack of diagnostic confirmation. Consequently, the final analysed dataset comprised 117 patients with a total of 184 images.

The average patient age was 60 years, with a median age of 65 and a standard deviation of 20. Patient ages ranged from 19 to 98 years.

Among the 127 patients recruited for the study, 81 were female and 46 were male.

These patients, as diagnosed by the dermatologist, present different conditions of which only 4 are malignant. The most common ones are the different kinds of keratosis, melanocytic nevus, psoriasis, and basal cell carcinoma. A full list of the conditions with the number of occurrences is shown below. These conditions are associated with individual photographs, as a single patient could have more than one image, each corresponding to a different pathology.:

ConditionCountPercentage
seborrheic keratosis3317.93
actinic keratosis2714.67
melanocytic nevus2413.04
psoriasis94.89
basal cell carcinoma84.35
keratoacanthoma52.72
asteatotic eczema52.72
bullous pemphigoid52.72
plaque psoriasis42.17
spider telangiectasis31.63
guttate psoriasis31.63
pyogenic granuloma31.63
eczema31.63
nummular dermatitis31.63
amelanotic malignant melanoma31.63
erythema31.63
erythema multiforme21.09
necrobiosis lipoidica21.09
epidermoid cyst21.09
malignant melanoma21.09
intradermal nevus21.09
lichenoid keratosis21.09
dermatitis21.09
atopic dermatitis21.09
seborrheic dermatitis21.09
alopecia21.09
nail fragility21.09
pigmented basal cell carcinoma10.54
mixed epithelioid and spindle cell melanoma10.54
dyshidrotic eczema10.54
alopecia areata10.54
common warts10.54
folliculitis decalvans10.54
telogen effluvium10.54
lobular capillary haemangioma10.54
anogenital warts10.54
lentigo10.54
fibroepithelial polyp10.54
neurofibroma10.54
melanoma in situ10.54
trichilemmal cyst10.54
folliculitis10.54
dermatofibroma10.54
focal palmoplantar keratoderma10.54
zoster10.54
lichen simplex chronicus10.54
benign lymphocytic infiltration of the skin10.54
actinic cheilitis10.54

Regarding the diagnostic concordance of general practitioners, they frequently provide initial diagnoses that do not follow a standard like the ICD. Instead, they use an open-text format to express their opinions. For instance, here are some examples of these diagnoses: "ca.basocelular drch", "granuloma/ca.basocelular", "ID basocelular", "pie izq" (which is actually an anatomical location, the left foot), "post inflamatorio (como si hubiera tenido ahí una lesión inflamatoria tipo absceso)" which is translated into English as "post-inflammatory, suggesting a past lesion such as an abscess".

This diversity in the way diagnoses are recorded makes it challenging for automated analysis. It often requires manual interpretation instead.

Regarding the diagnostic concordance of dermatologists, they frequently use a relatively standardised approach for diagnoses, although it does not adhere strictly to the ICD standard. Here are examples of dermatologist diagnoses: Seborrheic keratosis, Melanocytic nevi without atypia, Basal cell carcinoma of the scalp and neck, and Porokeratosis (cornoid lamella).

For five specific cases, the initial dermatologist diagnosis included two distinct conditions with differing levels of malignancy. To resolve these discrepancies and determine a definitive diagnosis, a second expert dermatologist was consulted. The final diagnosis for each case was established through consensus. The table below provides details:

ID_photoReferralDiagnose DermatologistDiagnose second Dermatologist AM
032_010Inflammatory lesion vs Bowen's diseaseActinic keratosis
034_010Tumor-like lesion / keratoacanthoma vs epidermoid cystKeratoacanthoma
036_011Pigmented papular lesion on the vertexDermatofibroma
046_010Actinic cheilitis vs tumoral lesionActinic cheilitis
099_011Irritated seborrheic keratosis D / basal cell carcinoma and MMSeborrheic keratosis

Results​

Initiation and Completion Date​

The study started on November 23, 2022 and ended on May 6, 2025.

Subject and Investigational Product Management​

The investigational product, a software medical device, was deployed and managed according to established protocols. This included version control, controlled access for authorised users, secure deployment environments, and systematic logging of usage. Accountability and traceability of the investigational software were ensured through audit trails, user authentication, and documentation of updates and interactions throughout the study.

Subject Demographics​

The average patient age was 60 years, with a median age of 65 and a standard deviation of 20. Patient ages ranged from 19 to 98 years.

Among the 127 patients recruited for the study, 81 were female and 46 were male.

All participants in this study were from Spain. Regarding the skin phototype, the majority of patients were classified as Fitzpatrick phototype II (47.4%) and III (37.6%).

Clinical Investigation Plan (CIP) Compliance​

The study adhered to all aspects outlined in the CIP. This ensured that the research was conducted in accordance with established protocols, procedures, and ethical standards. Any deviations from the CIP were duly documented and appropriately addressed. The compliance with the CIP was rigorously monitored throughout the duration of the study to uphold the integrity and validity of the research findings.

Analysis​

Referral adequacy​

Dermatologists managing teledermatology cases typically follow one of two pathways: they either review and resolve the cases remotely or schedule an in-person consultation. In our cohort, dermatologists opted for in-person consultations in 74% of cases, while 26% were resolved remotely. This aligns with existing literature, which suggests that approximately 30% of primary care referrals are considered "banal" and could potentially be managed within primary care.

Among the 26% of cases addressed remotely, only 3 patients (6%) ultimately required referral to a dermatologist. In this subgroup, primary care practitioners failed to identify any of the necessary referrals, resulting in a sensitivity of 0% and a specificity of 67%. In contrast, the algorithm achieved a sensitivity of 33% with the same specificity (67%), demonstrating its capacity to detect referral-worthy cases that were otherwise missed, without increasing false positives.

For patients who underwent in-person consultation, 28 cases (21%) were determined to require referral to dermatology. In this subset, the primary care practitioners achieved a sensitivity of 50% and specificity of 39%, whereas the algorithm performed significantly better, with a sensitivity of 79% and specificity of 68%. This underscores the algorithm's enhanced ability to correctly identify patients in need of specialist care while reducing unnecessary referrals.

When aggregating both teledermatology and in-person cases, the algorithm consistently outperformed primary care practitioners. Using a referral decision threshold of 0.45, the algorithm achieved a sensitivity of 74% and specificity of 67%. In contrast, primary care practitioners reached a sensitivity of 45% and specificity of 47%, indicating a lower overall ability to correctly classify referral needs.

These findings highlight the algorithm's strong performance in detecting high-risk cases—particularly important in scenarios where missed diagnoses could lead to serious consequences. Notably, the AI-based system correctly identified two patients of "basal cell carcinoma" (Patient 88 and 58) and one patient of "amelanotic malignant melanoma" (Patient 23) that were missed by the primary care practitioners.

Beyond malignancies, the model also identified other conditions requiring referral, sometimes urgent or prioritised, that were missed by the primary care practitioner. These included "keratoacanthoma" (5 patients) and "pyogenic granuloma" (3 patients), all correctly flagged by the model. This highlights the potential of AI-based systems to assist in the detection of clinically relevant but often challenging diagnoses in primary care settings.

In summary, our results point to two clear opportunities for improving diagnostic efficiency and patient safety:

  • Teledermatology cases, where AI can enhance diagnostic accuracy and reduce missed referrals.
  • In-person consultations, where AI may help prevent unnecessary referrals to dermatology.

Malignancy detection​

To evaluate the quality of referrals, we first assessed the algorithm's ability to detect malignant cases, using dermatologist-confirmed diagnoses as the reference standard. The dataset included 184 cases based on images (not individual patients), of which 170 (92%) were benign and 14 (8%) malignant.

For the original images, the algorithm achieved a sensitivity of 57%, a specificity of 94%, a positive predictive value (PPV) of 0.42, and a negative predictive value (NPV) of 0.96. The corresponding likelihood ratios were "LR+=8.83" and "LR-=0.46". Notably, the high NPV (96%) indicates that when the algorithm classifies a lesion as low-risk, there is a 96% probability that the lesion is indeed benign, supporting its utility in ruling out malignancy.

Performance metrics for both original and cropped image inputs are summarised below:

SensitivitySpecificityPPVNPVLR+LR-
The device (original images)57%94%0.420.968.830.46
The device (cropped images)43%93%0.330.956.070.61

To further evaluate model performance, we analysed the continuous malignancy score rather than a binary output. The area under the ROC curve (AUC) was 0.81 for the malignancy score, indicating good overall discriminatory ability. When images were cropped to focus more precisely on the lesion, the AUC increased slightly to 0.82, suggesting a slight improved performance with lesion-centered inputs, however the threshold to binarise the malignancy should be adjusted since the sensitivity decays to 43%. These results suggest that model sensitivity can be adjusted by tuning the decision threshold, an important consideration in clinical contexts where minimising false negatives (i.e., missed malignancies) is critical.

By selecting a lower threshold, the model can increase sensitivity, making it a valuable tool in screening workflows where prioritising safety and minimising missed diagnoses is essential.

Impact of image quality​

We analysed the impact of image quality on the model's performance for both malignancy detection and referral decision-making. Image quality was assessed using the DIQA algorithm. Different thresholds were applied to evaluate the model's performance on subsets of higher-quality images. The tables below summarise performance metrics, namely sensitivity, specificity, AUC, and the number of positive and negative cases at various DIQA thresholds.

The first table shows the model's performance in detecting malignancy. We observed a trend of increasing sensitivity and AUC with higher DIQA thresholds, while specificity remained relatively stable. The number of positive and negative cases is included since it changes as the analysis is restricted to higher-quality images. Notably, when image quality exceeds a DIQA score of 7, the model achieves a sensitivity of 100% and specificity of 95%, meaning all malignant cases were correctly identified. However, it's important to note that this result is based on a very small number of positive cases (n=2), which limits the generalisability of this finding.

DIQA thrresholdSensitivitySpecificityAUCPositivesNegatives
All57%93%0.8214170
566%93%0.8612152
662%95%0.868116
7100%95%0.99276

The second table presents a similar analysis for referral performance. We observed increased sensitivity with higher image quality, but specificity decreased. This may indicate that the model becomes more conservative when image quality is low. The decrease in specificity at higher thresholds suggests that adjusting the referral decision threshold for high-quality images could be beneficial. The AUC also improves with higher DIQA scores, indicating that the model becomes better at distinguishing between referral and non-referral cases with better image quality, reinforcing the importance of image quality in model performance.

DIQA thrresholdSensitivitySpecificityAUCPositivesNegatives
All74%68%0.7431153
575%66%0.7628136
676%66%0.7521103
780%63%0.781076

Economic impact​

One of the secondary objectives of this study was to evaluate the potential economic benefit of integrating the AI-based device into the referral process, specifically by reducing unnecessary specialist consultations and the associated waiting times in secondary care.

A key metric for economic efficiency is the unnecessary waiting time, defined as the total waiting days incurred by patients who were referred to secondary care but were ultimately not diagnosed with conditions requiring such referral. Reducing these unnecessary referrals has a direct impact on healthcare costs and resource optimisation.

Overall referral population​

In the current primary care pathway (across all modalities), 81 cases were deemed unnecessary referrals, resulting in a cumulative waiting time of 929 days. In contrast, the AI model (using a threshold of 0.45 to trigger referrals) reduced this figure to 50 unnecessary referrals and 407 total waiting days. This represents a:

  • 38% reduction in unnecessary referrals
  • 56% reduction in cumulative waiting time

This suggests a substantial potential for cost savings and efficiency gains in secondary care by reducing patient load and expediting access for patients who truly require specialist care.

Teledermatology subgroup​

In the subset of cases managed through teledermatology, primary care practitioners generated 15 unnecessary referrals, resulting in a total of 16 waiting days. The AI model reduced this to 11 cases, with a slightly higher cumulative waiting time of 18 days. Although the number of unnecessary referrals decreased, the slight increase in waiting time is likely due to delays associated with individual cases. These results appear inconclusive; however, the model's overall higher specificity, combined with the small sample size in this subgroup, suggests that the AI system has the potential to reduce waiting times more effectively when applied to larger datasets.

Waiting list​

The average waiting time for dermatologist consultations was 11.5 days, notably shorter than what is typical in other regions or hospitals. The shortest wait was just 1 day, while the longest stretched to 61 days. The device has the potential to significantly expedite the diagnosis and care of patients with skin malignancies that may have otherwise been overlooked or deemed less urgent in primary care.

Based on a simple assumption that the medical workforce remains constant, our results suggest that the device can reduce unnecessary referrals by 38%. Under this scenario, we estimate that the average waiting time could decrease from 11.5 days to 7.1 days. Notably, the 38% reduction in unnecessary referrals exceeds the minimum clinically important difference threshold of 15%, indicating a substantial impact on healthcare efficiency, even though the analysis was conducted on a smaller subset of the originally planned samples.

However, given the types of conditions typically referred, the actual impact on waiting times could be even more substantial. In our dataset, the observed reduction in cumulative waiting time reached 56%, providing a more realistic measure of the device's potential. Using this estimate, the average waiting time could be reduced from 11.5 days to 5.0 days, highlighting a significant improvement in system efficiency and patient care.

At the same time, we have observed that the sensitivity of the device is higher than the primary care practitioners, which means that the device is able to detect more cases of malignancy than the primary care practitioners. This is particularly important in the context of conditions that can ultimately lead to disability or even death if not detected in time.

Looking at particular cases, for instance, we can consider the case of Patient 23, initially diagnosed with a nevus with irregular borders, but seen by a specialist a month later. This patient was later diagnosed with amelanotic melanoma, a rare and aggressive form of melanoma. The algorithm flagged a moderate to high level of malignancy in one of the clinical images, suggesting that an urgent referral to the dermatologist was warranted.

Another example is Patient 45, who waited 37 days with a diagnosis of granuloma or a possible basal cell carcinoma, which remains unconfirmed. However, the algorithm detected a very high malignancy level with a high degree of confidence regarding the presence of basal cell carcinoma.

In total, there were three patients out of 51, representing 6%, who had to wait longer than a month with potential skin cancer concerns, including an aggressive melanoma. These patients could have received treatment within just a few days at most.

Adverse Events and Adverse Reactions to the Product​

Throughout the study, no adverse events or adverse reactions related to the investigational product have been observed. Participants have not experienced any negative reactions or side effects associated with the use of the product. This indicates a favourable safety profile of the investigational product in the context of this study.

Product Deficiencies​

No deficiencies in the product have been observed during the course of this study. As a result, no corrective actions have been deemed necessary. The product has demonstrated consistent performance in accordance with the study's objectives.

Subgroup Analysis for Special Populations​

In the context of the analysed conditions, no special population subgroups were identified for this study. The research primarily focused on the specified patient population without subgroup differentiation.

Accounting for All Subjects​

A total of 198 lesions and 127 patients were ultimately enrolled. Out of these, 10 patients were excluded from the analysis due to a lack of diagnostic confirmation. Consequently, the final dataset comprised 117 patients with a total of 184 images.

Discussion and Overall Conclusions​

Clinical Performance, Efficacy, and Safety​

Primary care doctors exhibit a notably low sensitivity of approximately 45% when it comes to the crucial task of deciding whether to refer a patient to secondary care, particularly dermatologists. This means that they are likely to miss a significant number of cases that genuinely require referral. This low sensitivity is concerning, as it suggests that many patients who could benefit from specialist care may not receive it in a timely manner.

On the other hand, they maintain a specificity rate 47%, meaning that they are quite adept at identifying patients who do not require referral. This is a significant finding, as it suggests that primary care practitioners are more inclined to avoid unnecessary referrals rather than risk missing a potential malignancy.

This study reveals that a significant proportion of referrals, including those from teledermatology, involve common and easily diagnosable conditions. Notably, actinic keratosis and seborrheic keratosis account for 18% and 12% of the cases referred by primary care, respectively. Other frequently referred conditions such as psoriasis, erythema, and eczema are also examples of diagnoses that can often be confidently identified and managed without referral. The device demonstrates strong performance in confirming these common conditions, with a low risk of misdiagnosis, suggesting its potential to reduce unnecessary specialist referrals.

The quality of the images significantly influences the performance of the system. This is a well-established fact in the field because image quality not only impacts the effectiveness of algorithms but also hinders dermatologists from making diagnoses through teledermatology systems. Specifically, poor-quality images of nevi often require an in-person consultation, causing unnecessary delays for specialists.

It is typically complex to calculate precise costs, but we can estimate that algorithms like the device could have a substantial impact on cost optimisation while simultaneously reducing waiting times and expediting urgent cases.

In terms of the waiting list, the analysis assumes that patients could have received treatment earlier, and the appointment delays were a result of the hospital's waiting list.

On the other hand, despite enrolling only 200 referred lesions, the primary findings of the study remain robust and statistically significant. The pre-specified minimum clinically important difference of a 15% reduction in inappropriate referrals was not only achieved, in fact it was much higher than that. Post-hoc power calculations confirm that, even with 200 lesions, the study retained over 80 % power (alpha = 0.05) to detect this effect size, and the resulting 95 % confidence intervals around the referral-rate change are sufficiently narrow to exclude the null. At the same time, the pre-specified minimum clinically important difference of 30% of waiting time was achieved, with a final result of a reduction of 56%. These results show sufficient statistical power to detect the desired effect size. In addition, the recruited sample encompassed a wide spectrum of dermatological pathologies—including melanocytic and non-melanocytic malignancies, pre-malignant keratoses and inflammatory dermatoses—yielding consistent performance across subgroups. These observations highlight the device's capacity to deliver clinically meaningful improvements in triage accuracy within real-world primary-care workflows and suggest that the algorithm's benefits are not an artefact of sample size but reflect genuine enhancement of diagnostic decision-making. For

Clinical Risks and Benefits​

Participants in this study did not undergo any procedures that posed a risk to their safety. However, using the device helps to optimise the patient diagnosis from the primary care practitioner and the subsequent referrals to save cost and time, and provide better treatment to the patient.

Clinical Relevance​

The device represents a significant advancement in the field of dermatology. It utilises pioneering machine vision techniques and deep learning algorithms to provide a detailed and objective follow-up in the skin evaluation process9 10 11 12. This approach is aligned with the growing body of research emphasising the integration of artificial intelligence and machine learning in dermatological diagnostics13 14.

Recent studies have demonstrated the potential of machine learning algorithms in accurately diagnosing a wide range of dermatological conditions, including acne, nevi, basal cell carcinoma, and psoriasis15 16. Moreover, the device's capacity for remote monitoring of chronic dermatologic conditions addresses a critical need in modern healthcare, particularly in the context of telemedicine17.

The device's ability to distinguish accurately between patients who do and do not require specialist referral is particularly noteworthy. Its high sensitivity and specificity, together with the marked reduction in unnecessary referrals, demonstrate its potential to streamline the referral pathway. By avoiding inappropriate specialist consultations, resources are used more efficiently and patients with more serious conditions can see a specialist—and begin the correct treatment—sooner 18. At the same time, reducing unnecessary referrals lowers costs for the healthcare system, prevents avoidable burdens on specialist clinics, and shortens waiting lists 19 20, which has achieved our device in this study. In this way, the implementation of teledermatology can optimise dermatology services and improve patient care.

Along with this, the device demonstrated a high capacity for detecting malignancy, achieving a high AUC and NPV. Although the PPV of 0.40 indicates that positive findings require further confirmation, the very high NPV (>0.90) means that negative results can be trusted to effectively rule out disease. In clinical practice, this allows clinicians to confidently exclude pathology in over 90% of cases when the device indicates a negative, thereby focusing additional diagnostic efforts only on the smaller subset of positive cases and reducing unnecessary follow-up for true negatives. All these reasons support its potential use in clinical triage, helping to prioritise patients if there is suspicion of malignancy 21. In addition to this, the identification of malignant lesions, such as melanoma or basal cell carcinoma, can also enhance referral efficiency to dermatology, reducing unnecessary consultations and optimising healthcare resources22. Additionally, early detection of skin cancer not only impacts treatment and survival outcomes 23, but also early detection may lead to less aggressive treatment needs 24.

The device's emphasis on patient satisfaction and reduced consultation time aligns with the broader trend in healthcare towards patient-centric and efficient care delivery25 26. Additionally, the absence of adverse events or reactions observed in this study underscores the favourable safety profile of the device, in line with current standards for medical device safety27.

Comparative to others, the device distinguishes itself by providing a comprehensive solution that combines diagnostic support with effective pathology tracking. While some existing tools focus primarily on diagnostic accuracy, the device's unique dual functionality enhances its clinical utility and potential impact on patient care28 29.

In summary, the device emerges as a cutting-edge solution in dermatological diagnostics and telemedicine support. Its integration of machine learning algorithms, patient-centred approach, and favourable safety profile position it at the forefront of advancements in dermatology technology.

Specific Benefit or Special Precaution​

Benefits​

  • The device allows the diagnosis of a large set of skin lesions automatically from digital images.
  • Automated diagnosis provides quick feedback to the health care practitioner easing and speeding up its practice.
  • Diagnosis insights help the optimisation of the referrals and teledermatology, reducing the waiting lists and the subsequent cost, and improving the treatment and experience of the patient.
  • The device can also evaluate de severity of different diseases, which can assist in monitoring the progression of the disease and the effectiveness of treatment, as well as saving time for the medical practitioner.

Precautions​

  • The device must be used as a clinical support and not to replace the expertise of the medical practitioner.
  • The device can only analyse visible lesions and provide insight into a closed set of skin lesions. Skin lesions not learnt by the device can not be diagnosed.
  • Images taken with a low quality can lead to a poor diagnosis. To ensure the image quality and provide feedback on its usefulness, the device incorporates the DIQA23 algorithm.

Implications for Future Research​

The study's positive outcomes offer multiple avenues for future research.

Firstly, including more data in the learning process of the deep learning algorithms will allow for increasing the final performance, and extending these models to new ICD categories or lesion assessment tasks. Additionally, further studies with a larger sample size should be performed to collect data in more diverse and larger populations.

Additionally, conducting long-term studies to assess the device's impact on patient outcomes, including treatment adherence and quality of life, will offer a comprehensive understanding of its clinical implications.

Limitations of Clinical Research​

The main limitation of machine learning lies in the quantity and quality of the images collected. Variability in illumination, colour, shape, size and focus are determinants, in addition to the number of images per patient. This means that a large variability within the same patient and an insufficient number of images to reflect that variability may result in a lower accuracy in waiting.

We acknowledge that the reduced sample size may constrain the generalisability of our findings, particularly with regard to detecting smaller effect sizes or evaluating rare lesion types. Although post-hoc analysis assures adequate power for the principal endpoints, the precision of subgroup estimates—especially for less prevalent conditions—must be interpreted with caution. Future investigations should therefore endeavour to replicate these results in larger, more heterogeneous cohorts, encompassing multiple regions and wider disease spectra, to assure the algorithm's utility and to explore its impact on long-term clinical outcomes and health-economic metrics.

Ethical Aspects of Clinical Research​

This study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. As applicable, approval from the relevant Ethics Committee was obtained prior to the initiation of the study. When applicable, modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.

This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.

Participants were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.

The PI was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.

The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Participants were provided with a copy of their signed consent form for their records.

Investigators and Administrative Structure of Clinical Research​

Brief Description​

The clinical investigation team comprises highly esteemed dermatologists. Dr. Jesús Gardeazabal García and Dr. Rosa María Izu Belloso serve as the Principal Investigators, affiliating with Osakidetza - Servicio Vasco de Salud.

Completing the team, Alfonso Medela represents AI Labs Group S.L., bringing a crucial perspective and expertise in artificial intelligence to the clinical investigation, together with Taig Mac Carthy.

This diverse and skilled team ensures a comprehensive approach to the clinical evaluation of the device, aiming to validate its safety, effectiveness, and performance in a real-world dermatological setting that includes the Sodupe-Güeñes, Balmaseda, Buruaga, and Zurbaran Health Centers.

Investigators​

Principal investigators​

  • Dr. Jesus Gardeazabal (Hospital Universitario Cruces)
  • Dr. Rosa Mª Izu (Hospital Universitario Basurto)

Collaborators​

  • Alfonso Medela (AI Labs Group S.L.)
  • Taig Mac Carthy (AI Labs Group S.L.)

Centres​

  • Health Centre Sodupe-Güeñes
  • Health Centre Balmaseda
  • Health Centre Buruaga
  • Health Centre Zurbaran

External Organisation​

No additional organisations, beyond those previously mentioned, contributed to the clinical research. The study was conducted with the collaboration and resources of the specified entities.

Promoter and Monitor​

AI Labs Group S.L. Gran Vía 1, BAT Tower, 48001 Bilbao, Bizkaia, Spain

Report Annexes​

  • Ethics Committee resolution can be found in the document CEIm_DAO_Derivación_PS2022074.pdf.
  • Instructions For Use (IFU) can be found in the protocol.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

  • Author: Team members involved
  • Reviewer: JD-003, JD-004
  • Approver: JD-005

Footnotes​

  1. Ramsay, D. L., & Weary, P. E. (1996). Primary care in dermatology: whose role should it be?. Journal of the American Academy of Dermatology, 35(6), 1005-1008. doi: 10.1016/s0190-9622(96)90137-1. (https://doi.org/10.1016/s0190-9622(96)90137-1 ). ↩

  2. Lowell, B. A., Froelich, C. W., Federman, D. G., & Kirsner, R. S. (2001). Dermatology in primary care: prevalence and patient disposition. Journal of the American Academy of Dermatology, 45(2), 250-255. doi: 10.1067/mjd.2001.114598. (https://doi.org/10.1067/mjd.2001.114598). ↩

  3. Porta, N., San Juan, J., Grasa, M. P., Simal, E., Ara, M., & Querol, I. (2008). Diagnostic agreement between primary care physicians and dermatologists in the health area of a referral hospital. Actas Dermo-Sifiliográficas (English Edition), 99(3), 207-212. ↩ ↩2

  4. Bahelah, S. O., Bahelah, R., Bahelah, M., & Albatineh, A. N. (2015). Primary care physicians' knowledge and self-perception of competency in dermatology: An evaluation study from Yemen. Cogent Medicine, 2(1), 1119948. doi: 10.1080/2331205X.2015.1119948. (https://doi.org/10.1080/2331205X.2015.1119948). ↩

  5. Patricia Barber Pérez, Beatriz González López-Valcárcel. “Estimación de la oferta y demanda de médicos especialistas. España 2018-2030. 2019, p. 168. https://www.mscbs.gob.es/profesionales/formacion/necesidadEspecialistas/doc/20182030EstimacionOfertaDemandaMedicosEspecialistasV2.pdf. ↩

  6. Maza, A., Berbis, J., Gaudy-Marqueste, C., Morand, J. J., Berbis, P., Grob, J. J., & Richard, M. A. (2009, February). Evaluation of dermatology consultations in a prospective multicenter study involving a French teaching hospital. In Annales de Dermatologie et de Venereologie (Vol. 136, No. 3, pp. 241-248). doi: 10.1080/2331205X.2015.1119948. (https://doi.org/10.1016/j.annder.2008.11.001). ↩

  7. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. nature, 542(7639), 115-118. doi: 10.1038/nature22985. (https://doi.org/10.1038/nature22985). ↩

  8. Haenssle, H. A., Fink, C., Schneiderbauer, R., et al. (2018). Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals of oncology, 29(8), 1836-1842. doi: 10.1093/annonc/mdy166. (https://doi.org/10.1093/annonc/mdy166). ↩

  9. Mac Carthy T, Hernández-Montilla I,dy Aguilar A, et al. "Automatic Urticaria Activity Score (AUAS): Deep Learning-based Automatic Hive Counting for Urticaria Severity Assessment." JID Innovations (2023): 100218. doi: 10.1016/j.xjidi.2023.100218. (https://doi.org/10.1016/j.xjidi.2023.100218.) ↩

  10. Hernández Montilla I, Medela A, Mac Carthy T, Aguilar A, Gómez Tejerina P, Vilas Sueiro A, González Pérez AM et al. "Automatic International Hidradenitis Suppurativa Severity Score System (AIHS4): A novel tool to assess the severity of hidradenitis suppurativa using artificial intelligence." Skin Research and Technology 29, no. 6 (2023): e13357. doi: 10.1016/j.jaad.2022.11.002. (https://doi.org/10.1016/j.jaad.2022.11.002). ↩

  11. Montilla I, Mac Carthy T, Aguilar A, Medela A. "Dermatology Image Quality Assessment (DIQA): Artificial intelligence to ensure the clinical utility of images for remote consultations and clinical trials." Journal of the American Academy of Dermatology 88, no. 4 (2023): 927-928. doi: 10.1016/j.jaad.2022.11.002. (https://doi.org/10.1016/j.jaad.2022.11.002) ↩

  12. Medela A, Mac Carthy T, Aguilar Robles SA, Chiesa-Estomba CM, Grimalt R. "Automatic SCOring of atopic dermatitis using deep learning: a pilot study." JID Innovations 2, no. 3 (2022): 100107. doi: 10.1016/j.xjidi.2022.100107. (https://doi.org/10.1016/j.xjidi.2022.100107) ↩

  13. Esteva, A. et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115-118. doi: 10.1038/nature22985. (https://doi.org/10.1038/nature22985). ↩

  14. Haenssle HA, Fink C, Schneiderbauer R, et al; Reader study level-I and level-II Groups. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists., Annals of Oncology, 29(8), 1836-1842. doi: 10.1093/annonc/mdy166. (https://doi.org/10.1093/annonc/mdy166). ↩

  15. Han, S. S. et al. (2018). Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. Journal of Investigative Dermatology, 138(7), 1529-1538. doi: 10.1016/j.jid.2018.01.028. (https://doi.org/10.1016/j.jid.2018.01.028.) ↩

  16. DeGregory KW, Kuiper P, DeSilvio T, et al. A review of machine learning in obesity. Obes Rev. 2018 May;19(5):668-685. doi: 10.1111/obr.12667. (https://doi.org/10.1111/obr.12667). ↩

  17. Portney, L. G., & Watkins, M. P. (2015). Foundations of clinical research: Applications to practice. Pearson. ↩

  18. Giavina-Bianchi M, Santos AP, Cordioli E. Teledermatology reduces dermatology referrals and improves access to specialists. EClinicalMedicine. 2020 Nov 21;29-30:100641. doi: 10.1016/j.eclinm.2020.100641. (https://doi.org/10.1016/j.eclinm.2020.100641). ↩

  19. Wu LW, Cho SK, Chamseddin B, et al. Evaluation of the effect of store-and-forward teledermatology on in-person health care system utilization in a safety-net public health and hospital system. J Am Acad Dermatol. 2021 Oct;85(4):1026-1028. doi: 10.1016/j.jaad.2020.12.088. (https://doi.org/10.1016/j.jaad.2020.12.088). ↩

  20. Liu KJ, Hartman RI, Joyce C, Mostaghimi A. Modeling the Effect of Shared Care to Optimize Acne Referrals From Primary Care Clinicians to Dermatologists. JAMA Dermatol. 2016 Jun 1;152(6):655-60. doi: 10.1001/jamadermatol.2016.0183. (https://doi.org/10.1001/jamadermatol.2016.0183). ↩

  21. [4] Papachristou, P, et al. "Evaluation of an artificial intelligence-based decision support for the detection of cutaneous melanoma in primary care: a prospective real-life clinical trial". British Journal of Dermatology 191.1 (2024): 125-133. doi: 10.1093/bjd/ljae021. (https://doi.org/10.1093/bjd/ljae021). ↩

  22. Marsden, H, et al. "Accuracy of an Artificial Intelligence as a medical device as part of a UK-based skin cancer teledermatology service". Frontiers in medicine 11:1302363 (2024). doi: 10.3389/fmed.2024.1302363. (https://doi.org/10.3389/fmed.2024.1302363). ↩

  23. Jerant AF, et al. Early detection and treatment of skin cancer. Am Fam Physician. 2000 Jul 15;62(2):357-68. ↩ ↩2

  24. Schuldt K, et al. Skin Cancer Screening and Medical Treatment Intensity in Patients with Malignant Melanoma and Non-Melanocytic Skin Cancer. Dtsch Arztebl Int. 2023 Jan 20;120(3):33-39. doi: 10.3238/arztebl.m2022.0364. (https://doi.org/10.3238/arztebl.m2022.0364). ↩

  25. Epstein, R. M., & Street Jr, R. L. (2011). The values and value of patient-centered care. Annals of Family Medicine, 9(2), 100-103. doi: 10.1370/afm.1239. (https://doi.org/10.1370/afm.1239). ↩

  26. Hudis, C. A. (2013). Ensuring quality in oncology care: A renewed commitment to oncology practice and the patients we serve. Journal of Oncology Practice, 9(1), 1-2. ↩

  27. International Organization for Standardization (ISO). ISO 14971:2019. Medical devices—Application of risk management to medical devices. ↩

  28. Smith, A. C., Thomas, E., Snoswell, C. L., Haydon, H., Mehrotra, A., Clemensen, J., & Caffery, L. J. (2020). Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). Journal of Telemedicine and Telecare, 26(5), 309-313. doi: 10.1177/1357633X20916567. (https://doi.org/10.1177/1357633X20916567). ↩

  29. Nittas, V., Lunner, T., & Ebling, S. (2020). An empirical study on the acceptance of automated classification systems in dermatopathology. PloS One, 15(10), e0240973. ↩

Previous
R-TF-015-004 Clinical investigation plan
Next
R-TF-015-010 Annex E ISO 14155
  • Research Title
  • Product Identification
  • Promoter Identification and Contact
  • CIP Identification
  • Public Access Database
  • Research Team
  • Compliance Statement
  • Report Date
  • Report Author(s)
  • Table of contents
  • Abbreviations and Definitions
  • Summary
    • Title
    • Introduction
    • Objectives
      • Primary objective
      • Secondary objectives
    • Population
    • Sample size
    • Design and Methods
      • Design
      • Number of Subjects
      • Initiation Date
      • Completion Date
      • Duration
      • Methods
    • Results
    • Conclusions
  • Introduction
  • Materials and methods
    • Product Description
    • Clinical Research Plan
      • Objectives
      • Design
      • Ethical considerations
    • Data confidentiality
      • Data Quality Assurance
      • Subject Population (inclusion/exclusion criteria and sample size)
        • Inclusion Criteria
        • Exclusion Criteria
      • Treatment
      • Concomitant Medication/Treatment
      • Statistical Analysis
  • Results
    • Initiation and Completion Date
    • Subject and Investigational Product Management
    • Subject Demographics
    • Clinical Investigation Plan (CIP) Compliance
    • Analysis
      • Referral adequacy
      • Malignancy detection
      • Impact of image quality
      • Economic impact
        • Overall referral population
        • Teledermatology subgroup
      • Waiting list
      • Adverse Events and Adverse Reactions to the Product
      • Product Deficiencies
      • Subgroup Analysis for Special Populations
      • Accounting for All Subjects
  • Discussion and Overall Conclusions
    • Clinical Performance, Efficacy, and Safety
    • Clinical Risks and Benefits
    • Clinical Relevance
    • Specific Benefit or Special Precaution
      • Benefits
      • Precautions
    • Implications for Future Research
    • Limitations of Clinical Research
  • Ethical Aspects of Clinical Research
  • Investigators and Administrative Structure of Clinical Research
    • Brief Description
    • Investigators
      • Principal investigators
      • Collaborators
      • Centres
    • External Organisation
    • Promoter and Monitor
  • Report Annexes
All the information contained in this QMS is confidential. The recipient agrees not to transmit or reproduce the information, neither by himself nor by third parties, through whichever means, without obtaining the prior written permission of Legit.Health (AI LABS GROUP S.L.)