R-TF-028-003 Data Collection Instructions - Custom Gathered Data

Table of contents

• Purpose and Scope
• Context and Rationale
• Objectives
• Data Sources and Study Types
  • Clinical Validation Studies
  • Dedicated Prospective Data Acquisition Studies
• Data Population Characteristics
  • Recruitment Strategy
  • Participating Institutions
  • Ethical and Legal Considerations
    • Ethical Approval
    • Informed Consent
    • Data Protection and Privacy (GDPR Compliance)
    • Data Processing Agreements
  • Inclusion Criteria
  • Exclusion Criteria
• Study Design and Protocols
  • Study Design Types
  • Study Workflow
• Image Acquisition Protocol
  • Qualified Operators
  • Standardized Acquisition Procedure
    • Number of Images
    • Image Types and Modalities
    • Technical Specifications
    • Equipment
• Data Collection and Handling Protocol
  • Data Collection Workflow
  • Collected Data Specification
  • Data Quality Assurance
    • Real-Time Quality Checks
    • Post-Acquisition Quality Control
    • Metadata Validation
    • Corrective Actions
• Data Ingestion and Management
  • De-identification Verification
  • Ingestion into Development Database
  • Documentation and Traceability
• Study-Specific Protocols
• Other Specifications
• References

Purpose and Scope

This document defines the systematic protocol for the collection of dermatological images and associated clinical metadata from custom data gathering activities conducted by or for AI Labs Group S.L. This protocol forms part of the data acquisition strategy for the development, validation, and continuous improvement of the AI algorithms integrated into Legit.Health Plus.

Custom gathered data serves as a critical complement to retrospective archive data, providing controlled, high-quality datasets from known clinical contexts with standardized acquisition protocols. These datasets are sourced from clinical validation studies of the medical device and from prospective data acquisition studies specifically designed for algorithm training and testing purposes.

Context and Rationale

The development of clinically safe, effective, and generalizable AI algorithms requires not only large-scale retrospective data but also carefully curated prospective data collected under controlled conditions [1-3]. Custom gathered data offers several key advantages:

• Acquisition Control: Standardized imaging protocols ensure consistent image quality, resolution, and metadata completeness.
• Ground Truth Quality: Direct access to clinical workflows enables the collection of robust diagnostic labels, including differential diagnoses and diagnostic confidence levels.
• Real-World Clinical Context: Data collected during actual clinical practice or under realistic clinical scenarios ensures ecological validity and reflects the intended use environment [4, 5].
• Regulatory Compliance: Custom data collection enables full control over informed consent, data protection, and ethical oversight, ensuring compliance with MDR 2017/745 and GDPR requirements.
• Performance Validation: Data from clinical validation studies provides an independent assessment of device performance in real-world conditions.

This approach ensures the creation of high-quality, well-characterized datasets that enhance model robustness, support regulatory requirements, and enable continuous device improvement.

Objectives

The primary objectives of this custom data collection protocol are:

• Controlled Data Acquisition: To collect dermatological images and metadata under standardized protocols that ensure high quality, completeness, and consistency.
• Clinical Validation Support: To gather data during clinical validation studies, enabling assessment of device performance in real-world clinical settings.
• Algorithm Enhancement: To acquire targeted datasets for specific diagnostic categories, patient demographics, or imaging conditions to address performance gaps or expand device capabilities.
• Ground Truth Establishment: To establish robust diagnostic ground truth through expert clinical assessment, differential diagnoses, and where applicable, histopathological confirmation [6].
• Regulatory Compliance: To execute all data collection activities in full compliance with ethical requirements (informed consent, IRB/CEIm approval), data protection regulations (GDPR), and quality management system procedures.

Data Sources and Study Types

Custom gathered data is collected through two primary mechanisms, with distinct intended uses for algorithm development:

Clinical Validation Studies

Data collected during clinical validation studies has a strictly limited use:

• Performance Validation: Primary and sole purpose is to assess the clinical performance of the device in real-world settings, comparing device outputs to reference standard diagnoses.
• Independent Test/Validation Sets Only: Data from clinical validation studies is used exclusively for independent testing and validation of algorithm performance.
• Regulatory Compliance: This separation ensures unbiased performance assessment and compliance with regulatory expectations for independent validation datasets.

Clinical validation studies are conducted in accordance with the device's Clinical Evaluation Plan and comply with all applicable regulatory and ethical requirements.

Dedicated Prospective Data Acquisition Studies

Prospective observational studies designed specifically for data collection purposes, conducted without intervention of the medical device. These studies:

• Are designed to capture specific types of data needed for algorithm development (e.g., rare diagnoses, specific skin types, particular imaging conditions).
• Flexible Dataset Use: Data collected through these studies may be used for algorithm training, validation, and testing as needed for algorithm development and improvement.
• Follow standardized acquisition protocols to ensure data quality and consistency.
• Require IRB/CEIm approval and informed consent from all participants.
• Do not involve any intervention, treatment, or modification of the patient's standard care pathway.

Data Population Characteristics

Recruitment Strategy

Participants are recruited from dermatology clinics and healthcare institutions where AI Labs Group S.L. conducts clinical validation or prospective data acquisition studies. Recruitment strategies vary by study type:

• Clinical Validation Studies: Consecutive enrollment ("all-comers") strategy to ensure representative patient populations and minimize selection bias [7].
• Prospective Data Acquisition Studies: May employ stratified or purposive sampling to ensure adequate representation of specific diagnostic categories, demographics, or clinical presentations.

Participating Institutions

Data collection is conducted at qualified healthcare institutions, including:

• Academic medical centers with dermatology departments.
• Community dermatology clinics with experienced dermatologists.
• Specialized skin disease clinics.

All participating institutions are selected based on:

• Availability of qualified dermatologists.
• Appropriate clinical infrastructure for image acquisition.
• Institutional capacity to comply with ethical and data protection requirements.
• Established collaboration agreements or data collection contracts with AI Labs Group S.L.

Ethical and Legal Considerations

All custom data collection activities are conducted in full compliance with ethical and legal requirements:

Ethical Approval

• All data collection protocols require prior approval from an Institutional Review Board (IRB) or Comité de Ética de la Investigación (CEIm) at the participating institution [8].
• The ethics application includes the study protocol, informed consent forms, data protection measures, and any amendments.
• Ethics approval documentation is maintained in the technical file for each data collection study.

Informed Consent

• All participants provide written Informed Consent prior to any data collection [9].
• The consent process ensures participants are fully informed about:
  • The purpose of the study (clinical validation or data collection).
  • The types of data being collected (images, clinical metadata).
  • How their de-identified data will be used (algorithm development, training, validation).
  • Their rights, including the right to withdraw consent.
• Consent forms are available in the participant's native language.
• Signed consent forms are securely stored and maintained according to retention requirements.

Data Protection and Privacy (GDPR Compliance)

• All data collection, processing, and storage activities comply with the EU General Data Protection Regulation (GDPR) [10].
• Data is de-identified at the point of collection:
  • Unique anonymized participant identifiers are assigned.
  • No personally identifiable information (PII) such as names, dates of birth, medical record numbers, or addresses is collected.
  • Image metadata (EXIF data) containing potential identifiers is stripped.
• Data transfer between institutions and AI Labs Group S.L. uses secure, encrypted channels.
• Access to identifiable data (during the active study phase) is restricted to authorized personnel under confidentiality agreements.
• Re-identification keys (linking anonymized IDs to participants) are maintained securely and separately from the research dataset, accessible only to the principal investigator at the collecting institution, and are not shared with AI Labs Group S.L.

Data Processing Agreements

Where applicable, formal Data Processing Agreements (DPAs) are established between AI Labs Group S.L. and participating institutions, clearly defining roles, responsibilities, and data handling procedures.

Inclusion Criteria

Participants and data will be included if they meet all of the following criteria:

• Eligibility: Patients attending a dermatology clinic for assessment of a skin condition.
• Age: Patients aged 18 years or older (or as specified in the study-specific protocol).
• Consent: Patients who are able and willing to provide written informed consent.
• Clinical Scope: Conditions involving the epidermis, dermis, and associated cutaneous structures, consistent with the intended use of Legit.Health Plus.
• Image Quality: Images captured must meet minimum quality standards (resolution, focus, lighting) as defined in the acquisition protocol.
• Diagnostic Information: A confirmed diagnosis or diagnostic assessment by a qualified dermatologist must be available for each case.

Exclusion Criteria

Participants and data will be excluded if they meet any of the following criteria:

• Inability to Consent: Patients unable to provide informed consent due to cognitive impairment, language barriers without interpretation, or other reasons.
• Refusal to Participate: Patients who decline to participate or withdraw consent.
• Poor Image Quality: Cases with extremely low visual quality due to patient factors (e.g., inability to remain still, uncooperative), anatomical factors, or technical issues.
• Out-of-Scope Conditions: Conditions outside the intended use of Legit.Health Plus (e.g., purely mucosal lesions, unless within device scope).
• Incomplete Data: Cases with missing critical metadata (diagnosis, imaging modality, lesion location) that cannot be recovered.

Study Design and Protocols

Study Design Types

Custom data collection studies are classified as:

• Prospective, Observational Studies: No intervention or modification of the patient's standard clinical care.
• Single-center or Multi-center: Depending on the specific study objectives and required sample size.
• Consecutive or Stratified Enrollment: Depending on whether the goal is to capture a representative general population or specific targeted subgroups.

Study Workflow

The general workflow for data collection during clinical encounters is:

1. Patient Presentation: Patient attends their standard dermatology appointment or a study-specific visit.
2. Eligibility Assessment: The attending dermatologist or study coordinator assesses eligibility based on the inclusion/exclusion criteria.
3. Informed Consent: The study is explained to eligible patients, questions are addressed, and written informed consent is obtained.
4. Image Acquisition: Study-specific images are captured according to the standardized acquisition protocol.
5. Clinical Assessment: The dermatologist performs their standard clinical assessment and records the diagnosis and relevant clinical metadata.
6. Standard Care Continuation: The patient's standard care pathway continues without alteration.
7. Data Recording: All required data is entered into the electronic Case Report Form (eCRF) or data collection system.

Image Acquisition Protocol

Qualified Operators

All images are acquired by or under the supervision of:

• Qualified Dermatologists: Board-certified or equivalent dermatologists with clinical experience in dermatological diagnosis.
• Trained Clinical Staff: Where appropriate, trained nurses or medical assistants may perform image acquisition under dermatologist supervision, following standardized protocols.

All operators receive training on the acquisition protocol and quality standards before participating in data collection.

Standardized Acquisition Procedure

To ensure consistency and quality across all data collection sites, a standardized image acquisition procedure is defined:

Number of Images

For each enrolled case, the operator captures between 1 and 5 high-resolution images of the relevant skin lesion(s) or affected area. The specific number depends on:

• The size and extent of the lesion(s).
• The number of distinct lesions requiring documentation.
• The clinical complexity of the presentation.

Image Types and Modalities

The image set must include:

• Clinical Images (Macroscopic Photographs):
  • Standard photographs of the lesion and surrounding skin.
  • Captured at a distance that provides anatomical context when needed.
  • Lighting should be adequate and even, avoiding harsh shadows or specular reflections.
• Dermoscopic Images:
  • Close-up, magnified images captured using a dermatoscope (contact or non-contact).
  • Required for pigmented lesions and other cases where dermoscopy is clinically indicated.
  • Should clearly show the morphological structures relevant for diagnosis.

Technical Specifications

Images must meet the following minimum technical requirements:

• Format: JPEG, BMP or PNG format.
• Focus: Images must be in sharp focus across the region of interest.
• Lighting: Adequate, even illumination without significant shadows, glare, or color casts.
• Framing: The lesion must occupy a significant portion of the frame while including sufficient surrounding normal skin for context.
• Artifacts: Minimize obstructions (hair, rulers, surgical markings) where possible, or ensure they do not obscure diagnostically relevant features.

Equipment

• Cameras: Digital cameras, smartphones with high-resolution cameras, or dedicated medical imaging devices.
• Dermatoscopes: Polarized or non-polarized dermatoscopes, contact or non-contact, from established manufacturers (e.g., DermLite, Heine, FotoFinder).
• Lighting: Natural lighting or standardized artificial lighting (daylight-balanced LED or flash).

Note: No specific manufacturer or model requirements are imposed to ensure real-world generalizability and device compatibility across diverse clinical settings [11, 12].

Data Collection and Handling Protocol

Data Collection Workflow

The systematic data collection process ensures completeness, quality, and traceability:

1. Participant Enrollment:

   • Upon obtaining informed consent, a unique, anonymized participant identifier is generated using a standardized format (e.g., SITE_YYMMDD_NNN).
   • The identifier is recorded in both the institution's study records and the data collection system.

2. Image Capture:

   • The operator captures images following the Standardized Acquisition Procedure (Section 6.2).
   • Images are reviewed immediately for quality (focus, lighting, framing).
   • Poor-quality images are recaptured if possible.

3. Clinical Assessment and Diagnosis:

   • The attending dermatologist performs their clinical assessment.
   • The diagnostic assessment is recorded in the electronic Case Report Form (eCRF) or data collection tool.
   • Diagnostic Information Collected:
     • Primary Diagnosis: The most certain diagnosis (ICD-11 code).
     • Differential Diagnoses (optional but encouraged), listed in descending order of certainty (ICD-11 codes).
     • Diagnostic Confidence: Where applicable, the clinician's confidence level (e.g., high, medium, low).
     • Histopathological Confirmation: If available, biopsy results and histopathological diagnosis.

4. Metadata Recording:

   • Required metadata is entered into the eCRF for each case:
     • Anonymized participant ID.
     • Demographics: age (in years or age ranges), sex, Fitzpatrick skin phototype (I-VI) [13].
     • Lesion characteristics: anatomical location, lesion type, size, duration.
     • Imaging modality: clinical, dermoscopic.
     • Acquisition date (de-identified: month and year only).
     • Device/camera model (optional but encouraged).

5. Quality Control:

   • A study coordinator or designated personnel reviews each case for completeness:
     • All required images present and meet quality standards.
     • All required metadata fields populated.
     • Diagnosis recorded in correct format (ICD-11).
   • Incomplete cases are flagged for resolution before data transfer.

6. Secure Data Transfer:

   • At regular intervals (e.g., weekly or monthly), de-identified images and metadata are securely transferred to AI Labs Group S.L.'s secure research environment.
   • Transfer methods include encrypted file transfer (SFTP, HTTPS), secure cloud storage with access controls, or encrypted physical media.
   • Data integrity is verified using checksums (e.g., SHA-256), when applicable.

7. Data Receipt and Verification:

   • Upon receipt, AI Labs Group S.L. verifies data integrity and completeness.
   • Any issues (corrupted files, missing metadata) are reported to the data collection site for resolution.

Collected Data Specification

For each case, the following data is collected:

Image Data:

• De-identified image files (JPEG or PNG format).
• Filename format: as defined in the study.

Metadata:

• Structured metadata file (CSV or JSON format) containing:
  • Anonymized participant ID.
  • Image filenames associated with the participant.
  • Primary diagnosis (ICD-11 code and description).
  • Differential diagnoses (ICD-11 codes and descriptions), if recorded.
  • Diagnostic confidence level, if recorded.
  • Histopathological confirmation, if available.
  • Patient demographics: age range, sex, Fitzpatrick skin type.
  • Lesion information: anatomical location, size, duration.
  • Acquisition metadata: imaging modality, device model, acquisition date (month/year).

Data Quality Assurance

To maintain high data quality standards, the following quality assurance measures are implemented:

Real-Time Quality Checks

• Image Review: Operators review images immediately after capture on the camera/device screen to verify focus, exposure, and framing.
• Recapture: Poor-quality images are recaptured during the patient encounter when possible.

Post-Acquisition Quality Control

• Automated Checks: Software tools are used to detect:

  • Corrupted or unreadable image files.
  • Images below minimum resolution thresholds.
  • Images with extreme visual quality distortions (overexposed, underexposed, blurred).
  • Presence of EXIF metadata containing identifiers.

• Manual Review: A trained reviewer (dermatologist, clinical researcher, or trained data specialist) inspects a sample of images (minimum 10% per batch) to assess:

  • Diagnostic quality and appropriateness.
  • Compliance with acquisition protocol.
  • Absence of identifiable information in images.

Metadata Validation

• Completeness Checks: Automated scripts verify that all required metadata fields are populated.
• Consistency Checks: Cross-field validation (e.g., anatomical location consistent with imaging modality).
• Diagnosis Code Validation: ICD-11 codes are verified against the official ICD-11 reference to ensure validity.

Corrective Actions

• Cases failing quality checks are flagged for review.
• Rectifiable issues (e.g., missing metadata) are corrected by contacting the data collection site.
• Cases with irresolvable quality issues (e.g., fundamentally poor image quality) are excluded from the dataset and documented.

Data Ingestion and Management

De-identification Verification

Before ingestion into the main AI development database, all data undergoes comprehensive de-identification verification:

1. Automated EXIF Stripping: All image metadata (EXIF, IPTC, XMP) is automatically removed using validated tools.
2. Automated Scanning: Automated tools scan for potential identifiers in filenames and metadata fields.
3. Visual Inspection: A representative sample of images (minimum 10%) is manually inspected for visible identifiers:
   • Patient faces (unless lesion is on face and face is relevant to diagnosis).
   • Identifiable tattoos with names or dates.
   • Visible medical record numbers or patient wristbands.
   • Background elements with institutional or personal identifiers.
4. Redaction: Any identified information is securely redacted (e.g., face blurring, cropping) or the image is excluded.

Ingestion into Development Database

Following successful quality assurance and de-identification verification:

1. Database Import: Images and metadata are ingested into the secure AI development database, as described in R-TF-028-004 Data Annotation Instructions - ICD-11 Mapping.
2. Unique Identifiers: Each image is assigned a globally unique identifier (UUID) within the database.
3. Source Tagging: All records are tagged with:
   • Source study identifier.
   • Data collection site.
4. Version Control: The database is version-controlled to ensure traceability and reproducibility.
5. Access Control: Strict role-based access controls limit database access to authorized personnel.

Documentation and Traceability

Comprehensive documentation is maintained for all custom data collection activities:

• Study Protocols: Detailed study protocols for each data collection study, including objectives, methods, and ethics approval.
• Data Collection Logs: Records of all data transfers, including dates, batch sizes, and any issues encountered.
• Quality Control Reports: Summary reports of quality assurance activities, including pass/fail rates and corrective actions.
• Ethics Approvals: Copies of IRB/CEIm approval letters and informed consent forms.
• Data Processing Agreements: Copies of contracts or agreements with participating institutions.
• Dataset Manifests: Comprehensive listings of all collected data, including source, collection dates, and data characteristics.

All documentation is maintained as part of the technical file for regulatory purposes and is subject to change control procedures as defined in the Quality Management System.

Study-Specific Protocols

Individual data collection studies (e.g., validation studies, targeted prospective collections) may have study-specific protocols that provide additional detail:

• Specific sample size calculations and enrollment targets.
• Inclusion/exclusion criteria tailored to study objectives.
• Detailed statistical analysis plans for validation studies.
• Site-specific procedures or institutional requirements.

These study-specific protocols are documented separately and referenced in the technical file. All study-specific protocols must comply with the general principles and requirements defined in this document.

Other Specifications

• Device Flexibility: No specific requirements are imposed regarding the make or model of camera or dermatoscope to ensure compatibility with diverse clinical settings and real-world generalizability [11, 12].
• Operator Variability: Images may be captured by different operators (dermatologists, trained clinical staff) to reflect real-world operational conditions, provided all operators are appropriately trained and qualified.
• Continuous Improvement: Data collection protocols are subject to periodic review and continuous improvement based on lessons learned, regulatory feedback, and advances in best practices.

References

1. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi:10.1038/nature21056

2. Liu Y, Jain A, Eng C, et al. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020;26(6):900-908. doi:10.1038/s41591-020-0842-3

3. Daneshjou R, Smith MP, Sun MD, Rotemberg V, Zou J. Lack of transparency and potential bias in artificial intelligence data sets and algorithms: a scoping review. JAMA Dermatol. 2021;157(11):1362-1369. doi:10.1001/jamadermatol.2021.3129

4. Winkler JK, Fink C, Toberer F, et al. Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition. JAMA Dermatol. 2019;155(10):1135-1141. doi:10.1001/jamadermatol.2019.1735

5. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. doi:10.1186/s12916-019-1426-2

6. Brinker TJ, Hekler A, Enk AH, et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer. 2019;113:47-54. doi:10.1016/j.ejca.2019.04.001

7. Haenssle HA, Fink C, Schneiderbauer R, et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol. 2018;29(8):1836-1842. doi:10.1093/annonc/mdy166

8. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. doi:10.1001/jama.2013.281053

9. European Parliament and Council. Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). Off J Eur Union. 2016;L119:1-88.

10. European Parliament and Council. Regulation (EU) 2017/745 on medical devices. Off J Eur Union. 2017;L117:1-175.

11. Combalia M, Codella NC, Rotemberg V, et al. BCN20000: Dermoscopic lesions in the wild. arXiv:1908.02288 [cs.CV]. 2019.

12. Tschandl P, Rosendahl C, Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data. 2018;5:180161. doi:10.1038/sdata.2018.161

13. Fitzpatrick TB. The validity and practicality of sun-reactive skin types I through VI. Arch Dermatol. 1988;124(6):869-871. doi:10.1001/archderm.124.6.869

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001