R-TF-015-004 Clinical investigation plan
Scope
This clinical Investigation Plan (CIP) sets out the rationale, objectives, design, methodology, conduct, implementation, record-keeping, and method of analysis for the clinical investigation.
CIP Identification
| CIP | |
|---|---|
| Title of the clinical investigation | Pilot study for the clinical validation of an artificial intelligence algorithm to optimize the appropriateness of dermatology referrals |
| Device under investigation | Legit.Health |
| Protocol version | Version 1.0 |
| Date | 2022-04-07 |
| Protocol code | LEGIT.HEALTH_DAO_Derivación_O_2022 |
| Sponsor | AI Labs Group S.L. |
| Coordinating Investigator | Dr. Jesús Gardeazabals and Dr. Rosa Mª Izu |
| Principal Investigator(s) | Dr. Jesús Gardeazabals and Dr. Rosa Mª Izu |
| Investigational site(s) | Health Center Sodupe-Güeñes, Health Center Balmaseda, Health Center Buruaga, and Health Center Zurbaran |
| Ethics Committee | Comité de Ética de la Investigación con Medicamentos de Euskadi |
Table of contents
- Scope
- CIP Identification
- Compliance Statement
- Abbreviations and definitions
- CIP or protocol specifications
- Product Identification and Description
- Justification of the design
- Hypothesis
- Objectives
- Summary of the study
- Design and methods
- Condition of interest
- Ethical considerations
- CIP Modification
- CIP Deviations
- Start, follow-up and end reports
- Statements of compliance
- Informed Consent process
- Adverse events, adverse product reactions and product deficiencies
- Annexes
Compliance Statement
- Harmonized standard UNE-EN ISO 14155:2021.
- Regulation (EU) 2017/745 on medical devices (MDR).
- Harmonized standard UNE-EN ISO 13485:2016s.
- Regulation (EU) 2016/679 (GDPR).
- Spanish Organic Law 3/2018 on the Protection of Personal Data and guarantee of digital rights.
- Spanish Organic Law 1090/2015 on regulating clinical trials with medicines, the Ethics Committees for Research with Medicines and the Spanish Registry of Clinical Studies.
Abbreviations and definitions
- AE: Adverse Event
- AEMPS: Spanish Agency of Medicines and Medical Devices
- AEP: Adverse Reaction to Product
- AUC: Area Under the ROC Curve
- CAD: Computer-Aided Diagnosis
- CMD: Data Monitoring Committee
- CIP: Clinical Investigation Plan
- CUS: Clinical Utility Questionnaire
- DLQI: Dermatology Quality of Life Index
- GCP: Standards of Good Clinical Practice
- ICH: International Conference of Harmonization
- IFU: Instructions For Use
- IRB: Institutional Review Board
- N/A: Not Applicable
- NCA: National Competent Authority
- PI: Principal Investigator
- SAE: Serious Adverse Events
- SAEP: Serious Adverse Event to Product
- SUAEP: Serious and Unexpected Adverse Event to the Product
- SUS: System Usability Scale
CIP or protocol specifications
Principal Investigators
- Dr. Jesús Gardeazabal (Osakidetza).
- Dr. Rosa Mª Izu (Osakidetza).
Coordinating investigators
- Dr. Jesús Gardeazabal (Osakidetza).
- Dr. Rosa Mª Izu (Osakidetza).
Collaborating Investigator(s)
- AI Labs Group S.L.
- Mr. Alfonso Medela
- Mr. Taig Mac Carthy
Investigational sites
- Health Center Sodupe-Güeñes
- Health Center Balmaseda
- Health Center Buruaga
- Health Center Zurbaran
Funding
This research was carried out without any funding or sponsorship.
Product Identification and Description
| Information | |
|---|---|
| Device name | Legit.Health Plus (hereinafter, the device) |
| Model and type | NA |
| Version | 1.0.0.0 |
| Basic UDI-DI | 8437025550LegitCADx6X |
| Certificate number (if available) | MDR 792790 |
| EMDN code(s) | Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software) |
| GMDN code | 65975 |
| Class | Class IIb |
| Classification rule | Rule 11 |
| Novel product (True/False) | FALSE |
| Novel related clinical procedure (True/False) | FALSE |
| SRN | ES-MF-000025345 |
Justification of the design
Background and rationale
Skin-related conditions account for a significant portion of primary care visits, representing about 5% of all consultations, predominantly among working-age populations. However, discrepancies between general practitioners and dermatologists in diagnosing these conditions are common, with diagnostic agreement ranging between 57% and 65%. The lack of dermatological expertise among general practitioners, coupled with time constraints and limited access to specialists, poses a challenge in effectively managing skin diseases, particularly in smaller communities where access to dermatologists is scarce. This results in misdiagnoses and unnecessary referrals, leading to inefficient healthcare delivery.
Recent advances in artificial intelligence (AI) and image recognition have shown promising potential in improving diagnostic accuracy for skin conditions. Computer-Aided Diagnosis (CAD) systems, which combine AI and digital image processing, allow for more accurate interpretation of medical images. Studies demonstrate that AI algorithms can classify skin lesions with a level of competence comparable to that of expert dermatologists. The use of AI-based tools offers a significant opportunity to improve clinical workflows, enhancing the accuracy of patient assessments and reducing the burden on healthcare professionals.
This study aims to clinically validate an innovative AI tool for grading the severity of skin conditions. By improving diagnostic precision and referral decisions, this technology could enhance medical practice, particularly in the early detection of skin cancer, while also opening new avenues for research on treatment effectiveness and disease subtypes.
Risks and benefits of the product in investigation and clinical research
Participants in this study did not undergo any procedures posing a risk to their safety. However, using the device could optimize patient diagnosis, save costs and time, and provide better treatment to patients.
Hypothesis
The device optimizes the appropriateness of referrals to dermatology. In addition, it will provide greater sensitivity and specificity than a general practitioner in the diagnosis of skin pathologies, especially differentiating malignant and benign lesions. It provides also the security of a second medical opinion, which has been clinically validated.
Objectives
Primary objective
To validate that the device is a valid tool for improving the adequacy of referrals to dermatology.
Secondary objective(s)
- To validate that the device reduces costs in secondary care.
- To validate that the device reduces dermatology waiting lists.
- To validate that the device optimizes clinical flow in Osakidetza.
Summary of the study
This is a prospective observational analytical study of a longitudinal clinical case series. It is intended to assess if the device is a valid tool to improve referrals to dermatology. This investigation includes a population of 400 patients treated in their health centers and referred to dermatology of Cruces and Basurto Hospitals. Data collection will include photograph analysis, data about clinical flow and the possible reduction of costs. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.
Design and methods
Type of clinical research
This is a prospective observational study which is intended to evaluate if the device is a valid tool to enhance the clinical flow and referrals to dermatology from primary care. It will include patients with different skin conditions. In this investigation, there will be a single group of patients and there is no control group since we want to evaluate the device in a real-world environment and with a large sample size (400 lesions, or 380 patients). We want to know also if the use of the device reduces the waiting lists and costs in secondary care.
Population
Adult patients (>18 years) with dermatological conditions that meet the inclusion criteria. These patients are attended at their Health Centers and referred to the Dermatology Departments of the Hospital Universitario Cruces and Hospital Universitario de Basurto.
Sample size
Considering a concordance rate of 55% between primary care and dermatology for referred lesions, a consensus has agreed that a 15% reduction in referrals from primary care to dermatology for malignant lesions would represent the minimum clinically important difference to justify a significant change in practice. Therefore, to detect a 15% reduction in inappropriate referrals, a sample size of 400 referred lesions or 380 patients would be required, with 80% power at a 5% significance level.
Duration
This study estimates a recruitment period of 2 months.
Specialist HCPs will have 1 month to label the photographs.
Researchers will have 1 month to close and edit the database, analyze the data, and prepare the final report of the study.
The total duration of the study is estimated at 4 months.
Acceptance criteria
- Improve the adequacy of referrals to dermatology.
- A reduction of waiting lists (at least 30% Warshaw et al. 2011).
- A reduction of the costs in secondary care.
Inclusion criteria
- Patients with skin pathologies.
- Patients aged 18 years or older.
- Patients who have signed the informed consent for the study.
Exclusion criteria
- Patient who at the investigator's discretion will not comply with the study procedures.
Variables
Main variable
The main variable aims to determine the efficiency of the device when optimizing the appropriateness of referrals to dermatology.
To do this, we will identify inappropriate referrals. We define an inappropriate referral as one that, according to the criteria of the specialists consulted in the study, does not require the attention of a dermatologist to be treated, as may be the case of seborrheic keratosis.
Secondary variables
- Patient demographic data: Sex and age.
- Data on the general practitioner: Age, years practising and centre.
- Cost reduction: Calculated as the product of the cost of a dermatological consultation by the number of consultations that the device would have avoided.
- Waiting list reduction: Calculated as the difference between the mean number of patients on the dermatology waiting list during the duration of the study minus the mean number of referrals avoided during the duration of the study.
- Clinical data: A photograph, to be taken with a dermatoscope whenever relevant, taken by the general practitioner at the time of the first consultation, whether the referral has occurred and the referral criteria.
Condition of interest
Patients with dermatological pathologies are treated at their health centres and referred to the dermatology service of Cruces and Basurto University Hospitals.
Limitations of clinical research
The main limitation of machine learning lies in the quantity and quality of the images collected. Variability in illumination, colour, shape, size and focus are determinants, in addition to the number of images per patient. This means that a large variability within the same patient and an insufficient number of images to reflect that variability may result in a lower accuracy in waiting.
Ethical considerations
This study adhered to international Good Clinical Practice (GCP) guidelines, the Declaration of Helsinki in its latest amendment, and applicable international and national regulations. As applicable, approval from the relevant Ethics Committee was obtained prior to the initiation of the study. When applicable, modifications to the protocol were reviewed and approved by the Principal Investigator (PI) and subsequently evaluated by the Ethics Committee before subjects were enrolled under a modified protocol.
This study was conducted in compliance with European Regulation 2016/679, of 27 April, concerning the protection of natural persons with regard to the processing of personal data and the free movement of such data (General Data Protection Regulation, GDPR), and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the guarantee of digital rights. In accordance with these regulations, no data enabling the personal identification of participants was collected, and all information was managed securely in an encrypted format.
Participants were informed both orally and in writing about all relevant aspects of the study, with the information being tailored to their level of understanding. They were provided with a copy of the informed consent form and the accompanying patient information sheet. Adequate time was given to patients to ask questions and fully comprehend the details of the study before providing their consent.
The PI was responsible for the preparation of the informed consent form, ensuring it included all elements required by the International Conference on Harmonisation (ICH), adhered to current regulatory guidelines, and complied with the ethical principles of GCP and the Declaration of Helsinki.
The original signed informed consent forms were securely stored in a restricted access area under the custody of the PI. These documents remained at the research site at all times. Participants were provided with a copy of their signed consent form for their records.
Data confidentiality
Current legislation will be complied with in terms of data confidentiality protection (European Regulation 2016/679, of 27 April, on the protection of natural persons with regard to the processing of personal data and the free movement of such data and Organic Law 3/2018, of 5 December, on Personal Data Protection and guarantee of digital rights). For this purpose, when applicable, each participant will receive an alphanumeric identification code in the study that will not include any data allowing personal identification (coded CRD). The Principal Investigator will have an independent list that will allow the connection of the identification codes of the patients participating in the study with their clinical and personal data. This document will be filed in a secure area with restricted access, under the custody of the Principal Investigator and will never leave the centre.
Once the paper CRDs are completed and closed by the Principal Investigator, the data will be transferred to a database.
As in the CRDs, the Database will comply with current legislation in terms of data confidentiality protection (European Regulation 2016/679, of 27 April, on the protection of natural persons about the processing of personal data and the free movement of such data and Organic Law 3/2018, of 5 December, on the Protection of Personal Data and guarantee of digital rights) in which no data allowing personal identification of patients will be included.
Bias minimization measures
In clinical research, minimizing bias is essential to ensure the validity and reliability of the study's results. In this study, patients will be randomly selected to participate in this. Thus, outcomes will not be influenced by pre-existing characteristics of the participants. Furthermore, we will use standarized protocols, Using standardized procedures for conducting the study and measuring outcomes ensures that all participants are treated and evaluated in the same way, reducing variability due to differences in how the intervention is applied or how outcomes are assessed. Finally, Collecting data prospectively reduces the chance that participants or investigators will inaccurately recall past events, which is common in retrospective studies.
Calendar
The duration of this study was estimated at 4 months, including 2 months for the recruitment time, 1 month for the specialist to review photos, and 1 month for data analysis.
However, the study will be active as long as it does not reach the specified number of recruited patients.
Monitoring plan
The team will periodically meet with researchers and review the data collected to ensure that the entire collection procedure is carried out correctly.
The periodicity of the reviews follows the following logic: - The first review will be carried out when samples from 3 patients have been collected. - Subsequently, once every 30 calendar days until the end of the study.
If the first review requires any correction, a next evaluation will be carried out on the following 3 patients. The review meetings will be held depending on the availability of the researcher.
Completition of the investigation
After the final closure of the clinical investigation, a Clinical Investigation Report (CIR: T-015-006 Clinical Investigation Report) will be drafted, even in the event of early termination or suspension. The CIR will be provided to the Ethics Committee and the Spanish Agency for Medicines and Medical Devices (AEMPS). The results obtained (whether positive, inconclusive, or negative) will be included in the previously mentioned public access database.
Additionally, if deemed appropriate, the results may be published in scientific journals. The Ethics Committee that approved this clinical investigation will be acknowledged, and any funds received by the author for the study and its source of funding will be disclosed. The anonymity of participants in the clinical investigation will be maintained at all times.
Upon completion of the study, the results of the clinical utility and satisfaction surveys, as per the annexed models, may be presented at conferences and scientific meetings, subject to prior authorization by both parties. Press releases and other communications may also be issued to share the study's results. All publications and communications must be reviewed and approved by the parties involved.
Statistical analysis
Each variable will be characterized using frequency distributions for qualitative variables and statistics of central tendency such as mean and median and variability such as standard deviation (SD) or interquartile range for quantitative variables based on their distributional characteristics.
Sensitivity, specificity, positive and negative predictive values (PPV and NPV) and likelihood ratios (LR+ and LR-) will be calculated by comparing both the results obtained using the device and those obtained with the referral criteria of general practitioners with the criteria used by specialists are considered the gold standard.
The analyses will be performed using appropriate statistical software, SPSS version 23.0 and STATA 13.0. Values of a p-value lower than 0.05 will be considered significant.
Data management
The data will be managed and tabulated with consistency rules and logical ranges to control inconsistencies during data tabulation. A validation process of the clinical data will be carried out by running computer filters based on validation rules, which will automatically identify missing values or inconsistencies of clinical data according to the protocol. Additionally, manual editing and validation will be performed using descriptive and exploratory statistical techniques to complement the detection of logical errors and inconsistent values.
The database will be considered closed upon completion of all data management processes and satisfactory resolution of discrepancies and errors in the data. Any changes in the database after its closure can only be made after written agreement between the Principal Investigator and the technical coordinators of the project.
AI LABS GROUP, S.L. (hereinafter, the manufacturer) is the owner of the device. During the period of validity of this study, the manufacturer will grant a license to use the device by the research team free of charge. The research team will be the administrator of the account. Both patients and members of the medical team will have login credentials. The manufacturer will not have access to the account or patient information.
According to the definitions and roles set out in Regulation (EU) 2016/679 (GDPR), the Data Controller is the research team and the manufacturer is the Data Processor.
The storage of data and photographs will be in line with the European Regulation 2016/679 (GDPR) and the Organic Law 3/2018 of 5 December on the Protection of Personal Data and guarantee of digital rights. At the end of the study, all information stored in the device will be totally and permanently deleted.
The device complies with current legislation on the protection and confidentiality of personal data European Regulation 2016/679 (GDPR). Appropriate technical and organizational security measures are adopted to ensure the security of personal data and prevent its alteration, loss, unauthorized processing or access, given the state of technology, the nature of the data and the risks to which they are exposed, whether from human action or the natural physical environment.
CIP Modification
As indicated in the UNE-EN ISO 14155:2021 standard, the Clinical Investigation Plan (CIP) may be modified as necessary during the clinical investigation. The changes made must be described, along with their justification, potential impact on clinical performance, efficacy, safety, or other evaluation criteria, and identification of other affected documents.
CIP modifications will be prepared by the sponsor and agreed upon and accepted between the sponsor and the principal investigator. The modifications will be recorded with a justification for each one in the form of an amendment or addendum.
The required regulatory authority (AEMPS) and the Ethics Committee or Institutional Review Board (IRB) will be notified, if necessary. Modifications to the clinical investigation may only be implemented once favourable feedback and the corresponding approval have been obtained.
- In the case of substantial modifications to authorized clinical investigations, a request must be submitted to the AEMPS.
- For non-substantial modifications to authorized clinical investigations, until the corresponding module is available in the EUDAMED database, the updated documentation will be sent to the AEMPS for inclusion in their file.
CIP Deviations
As indicated in the UNE-EN ISO 14155:2021 standard, the investigator may not deviate from the Clinical Investigation Plan (CIP), except in emergencies (section 4.5.4.b of the standard). In such cases, the investigator may proceed without prior approval from the sponsor and the Ethics Committee to protect the rights, safety, and well-being of human subjects.
These deviations must be documented by the principal investigator and notified to the sponsor and the IRB as soon as possible, and always within a maximum of 15 days. Immediately after receiving the notification, the sponsor will record and analyze the deviations carried out and their potential impact. Depending on the findings, the sponsor will take the necessary corrective and/or preventive measures.
In other circumstances, when deviations affect the rights, safety, and well-being of the subject or the scientific integrity of the clinical investigation, deviation requests and reports must be provided to the IRB if required.
Start, follow-up and end reports
The start of the study will be notified to the ethics committee. Annual follow-up reports will be submitted thereafter.
Upon obtaining the study conclusions, a final report (T-015-006 Clinical Investigation Report (CIR)) will be prepared and submitted to the ethics committee.
Statements of compliance
The present clinical investigation will be conducted following the ethical principles originating from the Declaration of Helsinki.
Additionally, the clinical investigation will comply with the harmonized standard UNE-EN ISO 14155:2021 and the European Regulation MDR 2017/745. The statement specifying compliance with the general safety and performance requirements in accordance with MDR can be found in the document Manufacturer's Declaration of Compliance with Requirements.
As appropiate, clinical investigation will not commence until approval/favourable opinion has been obtained from the Clinical Research Ethics Committee (CREC) and the required regulatory authority (Spanish Agency for Medicines and Medical Devices, AEMPS), and it must comply with any additional requirements imposed by the CREC and/or AEMPS.
Informed Consent process
The patient, or in their absence, the family member or legally authorized representative, must provide written consent before their inclusion in the clinical investigation. This will occur after they have understood, through a prior interview with the principal investigator or a member of the research team, the objectives of the investigation, its risks, inconveniences, and benefits, as well as the conditions under which it will be conducted, and after being informed of their right to withdraw from the investigation at any time without explanation and without incurring any responsibility or prejudice.
The principal investigator will discuss the study with the subject and provide the information objectively, without coercion or influence, and without offering any inappropriate or undue incentive. The principal investigator will use non-technical language in the subject's native language (or that of the spouse/closest relative or legally authorized representative) for better understanding and will allow sufficient time for reading and comprehending the information.
Each participant will document their informed consent by signing and dating the informed consent form. Each signed and dated consent will be kept by the principal investigator, and a copy of the informed consent will be provided to the subject.
If new important information arises that could affect the subject's willingness to continue participating in the clinical investigation, it will be provided in any case. If necessary, their continued informed consent will be confirmed in writing.
Adverse events, adverse product reactions and product deficiencies
Adverse Events (AE) and Adverse Events to the Product (AEP)
An AE is any unintended medical event, unanticipated illness or injury, or unintended clinical signs (including abnormal laboratory findings) in subjects, users, or other persons, whether or not related to the investigational product and whether intended or unintended.
An AEP is an adverse event related to the use of an investigational medical device.
Given these definitions, potential AEPs or AEs are documented in the product's IFU.
Product deficencies
Possible inadequacies of a medical device may relate to its identity, quality, durability, reliability, safety, or performance.
Product deficiencies in the investigation will be managed by the sponsor according to non-conforming product control procedures. When appropriate, corrective and/or preventive actions will be taken to protect the safety of subjects, users, and other individuals.
Serious Adverse Events, serious adverse events to product and serious and unexpected adverse events to the product
According to UNE-EN ISO 14155:2021:
- A Serious Adverse Product Reaction (SAEP) is a SAE that has produced any consequence characteristic of a serious adverse event.
- A Serious Adverse Event (SAE) is an AE that results in any of the following events: death, serious deterioration of the health status of the subject, users or other persons, fetal distress, fetal death, congenital anomaly or birth defect.
- A Serious Unexpected Adverse Event to the Product (SUAEP) is a SAE that, due to its nature, incidence, intensity or consequences, has not been identified in the updated risk assessment.
Taking into account these definitions, there are no SAEP, SAEs or SUAEPs related to the use of the product.
Foreseeable adverse events and adverse events to the product
The foreseeable adverse events and expected adverse reactions to the product, as well as their incidence, mitigation and treatment are documented in the T-013-002 Risk management record of the product under study.
Data Monitoring Committee (DMC)
Information on the DMC (Data Monitoring Committee), if established. This is an independent committee that the sponsor may establish to evaluate, at indicated intervals, the progress of the clinical investigation, the safety data or the critical clinical performance or efficacy endpoints and to recommend to the sponsor whether to continue, suspend, modify, or stop the clinical investigation.
Suspension or early termination of clinical research
As indicated in the UNE-EN ISO 14155:2021 regulation, the sponsor may suspend or terminate the clinical research early for significant and documented reasons. These are:
- If during the clinical research the suspicion of an unacceptable risk arises, including a serious threat to the health of the subjects. It must be suspended while the risk is determined.
- If an unacceptable risk that cannot be controlled is confirmed.
- Due to the impossibility of including a minimum number of subjects that allows the final assessment of the study within a reasonable period according to the characteristics of the study.
- When instructed by the IRB or the required regulatory authority (AEMPS).
- Due to non-compliance with the obligations assumed in the contract by any of the contracting parties.
- By mutual agreement between the parties, expressed in writing.
- By the will of one of the parties, expressed in writing at least one month in advance.
In the event of a suspension or early termination of the clinical research, the sponsor will inform the AEMPS. The sponsor must provide the resources to comply with the obligations of the CIP and with the existing agreements. The principal investigator must inform the subjects if so stipulated in the agreement between the sponsor and the research centre.
If the clinical investigation is resumed, the sponsor must inform the principal investigator and, where appropriate, the AEMPS. Approval from the IRB and, where appropriate, from the AEMPS will be required for such resumption. The principal investigator must inform the subjects.
Annexes
Annex I. Protocol of the study
Annex II. Ethics committee approval
Annex III Patient information sheet & Informed consent
Signature meaning
The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:
- Author: Team members involved
- Reviewer: JD-003, JD-004
- Approver: JD-005