CAPA Plan - BSI CE Mark Closeout

Document Information

Field	Value
Manufacturer	AI Labs Group S.L.
Address	Street Gran Vía 1, BAT Tower, 48001, Bilbao, Bizkaia
Legal Manufacturer SRN	ES-MF-000025345
Customer Site(s)	AI LAB-0047909351-000
Review Reference	T0041849, SMO 30001393
Non-Conformity Reference	30365821
CAP Issue Date	Thursday, May 15, 2025
Device Name	Legit.Health Plus
Basic UDI-DI	8437025550LegitCADx6X
Certificate Number(s)	MDR 792790
Legislation	MDR Conformity Route Applied: Annex IX Ch I & III: Quality Management System

Purpose of This Document

This document serves as the CAPA Plan Closeout Report comparing:

• ✅ The original corrective actions proposed in the CAPA Plan submitted to BSI
• 🟠 The actual implementation (marked in orange) with explanations of any changes
• 📎 Evidence references to the Technical File documentation

Legend

• Original text: Exact text as proposed in the CAPA Plan submitted to BSI
• Orange text: Actual implementation or changes made
• Evidence: References to supporting documentation in the Technical File

NCR 2650005-202501-M1 (Major) - Verification and Validation of Measuring/Diagnostic Features

Non-Conformity Statement (BSI)

Documentation of verification and validation tests to demonstrate conformity of the device's measuring and diagnostic features to GSPRs 1, 15.1 and 17.2 per Annex II, 6 is inadequate.

Evidence that the device achieves the performance intended by the manufacturer for the diagnostic features described in REQ_001 (intensity of clinical signs), REQ_002 (count of clinical signs), REQ_003 (extent of clinical signs) and REQ_004 (ICD class distribution) cannot be confirmed based on the test documentation supplied.

• GSPRs Affected: GSPR 1, 15.1, 17.2
• Annex II: 6.1 (a), (b); 6.2 (f)

Root Cause (Original)

The description of the Design and Development (D&D) process in procedure GP-012 Design, Redesign, and Development does not fully meet the requirements of the MDR or those outlined in ISO 62304 and EN 82304-1, which is recognized as the state of the art for Software as a Medical Device (SaMD). As a result, the records related to Design, Development, Verification, and Validation are incomplete and insufficient.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	We will change the structure of requirements, and divide current into two: Product User Requirement, Software Requirement Specification (SRS-*).	Implemented. Created PRS (Product Requirement Specification) and SRS (Software Requirement Specification) with clear traceability. The naming "User Requirement" was refined to "PRS" which better aligns with MDR terminology.
2	Regarding GSPR 15.1, in test plans and runs we will improve the explanation on how metrics/benchmarks are appropriate to demonstrate the requirement/objective and we will provide sufficient evidence to prove the accuracy for their intended purpose, based on appropriate technical and clinical documentation.	Implemented. Created comprehensive AI/ML documentation suite (R-TF-028-*) with detailed metrics justification instead of the originally proposed R-TF-012-009.
3	The peer-reviewed publication lays out the methodology for choosing the model architecture and training method. We will add evidence in Test plans and Test runs (PLAN-, TEST-) of the verification for every software item related to this objective at the time of release/deployment.	Implemented. Test documentation now structured as R-TF-012-033 Software Test Plan, R-TF-012-034 Software Test Description, R-TF-012-035 Software Test Report instead of individual PLAN-/TEST- files.
4	The metrics/benchmarks we use in verification and validation are detailed in the record R-TF-012-009 Validation and testing of machine learning models. In that document, we will elaborate more on how those metrics demonstrate the requirement/objective.	Changed. Instead of R-TF-012-009, created dedicated AI/ML documentation suite (R-TF-028-001 AI/ML Description through R-TF-028-011 AI/ML Risk Matrix) with more comprehensive coverage.
5	In R-TF-008-001 we say that GSPR 15.1 is addressed by clinical validation, but we will change this to say that we address these in the Verification and Validation (V&V) plans and results during the verification.	Implemented. GSPR 15.1 is now addressed through both verification (Software Verification Phase 4) and validation (Phase 5) including clinical validation.
6	We will add evidence that the device has been verified on data representative of the full intended patient population. This will be seen in Test plans and Test runs (PLAN-, TEST-). This will include Fairness analysis, Differential impact tests in population subgroups.	Implemented with dedicated AI Risk Assessment. Created R-TF-028-011 AI/ML Risk Matrix with comprehensive bias analysis across skin phototypes, age groups, and demographics.
7	We will add explanations on how the test population during verification represents the real-world performance of the device. We explain that the validation was done with a representative of the full intended patient population in the intended use environment done during the clinical studies and usability testing (Software verification Phase 4) including all validation reports like usability, cybersecurity, risks and clinical.	Implemented with clearer separation. Verification (Phase 4) covers software testing; Validation (Phase 5) covers clinical validation and usability in intended environment.
8	To achieve this, we will provide data on the demographic and skin characteristic of the population in the test dataset, and how that is representative of the broad population. This will include data on age, sex and skin phototypes in the test dataset, and how it matches the population.	Implemented. Dataset characterization includes Fitzpatrick phototype distribution, age, sex, and anatomical site coverage in R-TF-028-003 Data Collection Instructions.
9	Regarding the paper referenced (Medela et al, 2022) which indicates that model performance was inferior to annotator/expert performance for images of dark skin, that was true at the time of the study, but we will add in Test plans and Test runs (PLAN-, TEST-) the performance metrics for all skins phototypes that show equiparable performance.	Implemented. R-TF-028-011 AI/ML Risk Matrix includes performance metrics stratified by Fitzpatrick phototype demonstrating equivalent performance across skin types.
10	We will change the template for Test plans and Test runs (PLAN-, TEST-) to include the requirements of annex II, 6.1 (a), (b) and 6.2 (f). In the test plan: 6.1(a) test design, 6.1(b) complete test or study protocols. In the test run: 6.2(f) methods of data analysis, 6.2(f) data summaries, 6.2(f) test conclusions.	Implemented. Consolidated into R-TF-012-033 Software Test Plan, R-TF-012-034 Software Test Description, R-TF-012-035 Software Test Report following IEC 62304 terminology.

Evidence:

• R-TF-012-031 Product Requirements Specification
• R-TF-012-028 Software Requirement Specification
• R-TF-028-001 AI/ML Description
• R-TF-028-011 AI/ML Risk Matrix
• R-TF-012-033 Software Test Plan
• R-TF-012-034 Software Test Description
• R-TF-012-035 Software Test Report
• R-TF-008-001 GSPR

NCR 2650005-202501-M2 (Major) - Software Development Lifecycle & Traceability

Non-Conformity Statement (BSI)

Evidence that the device has been developed and manufactured in accordance with state of the art taking into account the principles of development life cycle, risk management and verification per GSPRs 1 and 17.2 is inadequate.

Evidence of software design and development under EN 62304 to which compliance has been claimed to demonstrate conformity to the relevant GSPRs is inadequate.

Documentation of product verification per Annex II, 6 is inadequate.

• GSPRs Affected: GSPR 1, 17.2
• Annex II: 6.1 (a), (b)

Root Cause (Original)

The description of the Design and Development (D&D) process in procedure GP-012 Design, Redesign, and Development does not fully meet the requirements of the MDR or those outlined in ISO 62304, which is recognized as the state of the art for Software as a Medical Device (SaMD). As a result, the records related to Design, Development, Verification, and Validation are incomplete and insufficient.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	We will improve the traceability between risks and requirements. In every Software Requirement Specification (SRS-*), we will explain which risks they are related to, and how they are related. This includes: Risks that are caused by the requirement, Risks to which the requirement is the solution (mitigation or control).	Implemented. Each SRS now includes causeRequirements and mitigationRequirements fields linking to specific risks.
2	In the risk management record (R-TF-013-002), we will change the column named "Requirement(s) controlling risk" into "SRS controlling risk" and we will move it to the end of the table so it's conceptually within the risk control section.	Implemented with enhanced structure. Risk matrix now includes causeRequirements, mitigationRequirements, verificationOfImplementation, and verificationOfEffectiveness columns.
3	We will eliminate from the risk management record (R-TF-013-002) the column named "Related to" because it was a vague way of referring to the related software item, and now that information will be contained in the SRS.	Implemented. Removed redundant columns; added more relevant fields like threatModelRefs and postMarketPlanRefs.
4	We will eliminate from the risk management record (R-TF-013-002) the column named "Part affected" because it does not apply to software devices.	Implemented. Column removed.
5	We will eliminate from the risk management record (R-TF-013-002) the column named "People affected" because it's not a requirement of the ISO 14971 and it is redundant.	Implemented. Column removed.
6	In the GP-013 Risk Management general procedure we will further elaborate on how the R-TF-013-002 explains how hazardous situations, causes, risk control measures and verification are related.	Implemented. GP-013 updated with detailed explanation of risk-requirement traceability.
7	We will execute missing unit tests.	Implemented. All unit tests executed and documented in R-TF-012-035 Software Test Report.
8	In the R-TF-012-006 Lifecycle plan and report we will further elaborate on our verification tests. There can be three types: system, unit, and integration, and verification (system). We will describe them in further detail, specifically how acceptance of tests is documented.	Implemented. Test documentation clearly categorizes tests by type with full traceability to requirements in R-TF-012-033 Software Test Plan, R-TF-012-034 Software Test Description, R-TF-012-035 Software Test Report.
9	We will modify the templates of Test plans and Test runs (PLAN-, TEST-) and implement the structure to the existing test plans and runs. Test plan: Clearly indicate what is the acceptance criteria, Clearly indicate why that acceptance criteria is right, Clearly indicate what test runs arise from the plan, Clearly indicate what kind of test it is: system, unit, or integration or verification. Test run: Clearly indicate whether it was met, therefore passed, Clearly indicate from what test plan the stem.	Implemented as R-TF-012-033 Software Test Plan, R-TF-012-034 Software Test Description, R-TF-012-035 Software Test Report structure. Each test case includes requirement references, acceptance criteria, and pass/fail status.
10	Regarding the missing evidence for software integration testing, we will add to R-TF-012-006 Lifecycle plan and report the full list of integration testing for all critical components, which are all the processors identified as Class B.	Implemented. Integration tests documented for all Class B processors.
11	Regarding the software version missing in the integration test, we will modify the template T-012-017 Integration test review: It will include the version of the device being tested, It will include the IDs of the test runs under review. We will change the name to "Test review" (instead of "Integration test review") and it will include the review of all three types of tests: integration, unit or verification system.	Implemented. Test review documentation includes device version and test run IDs.
12	We will modify GP-012 to explain that the records Verified Version Release (REL-*) represents the outcome of the software verification process (5.8 of 62304). It is a candidate for a version of the device that will undergo the clinical validation, and if it's successful, will be put in the market.	Implemented as R-TF-012-038 Verified Version Release. Naming convention changed from REL-* to R-TF-012-038 for consistency.
13	The records Validated Version Transfer (TRA-*) represent the outcome of the clinical validation (61 of MDR). Also, this record reviews the R-TF-015-003 Clinical Evaluation Report, as well as other documentation. This record represents the approval of a validated (and previously verified) version that is put on the market.	Implemented as R-TF-012-039 Validated Version Transfer. Naming convention changed from TRA-* to R-TF-012-039 for consistency.
14	We will add to the GP-012, in the section "Software versioning", the clarification that the significance of changes is determined according to the guidance MDCG 2020-3 Rev.1. Any and all minor and major changes will lead to: an update of a document, or/and the creation of a new document, or/and the validation that a document is still applicable for the new version without any change.	Implemented. Version significance determined per MDCG 2020-3 Rev.1. Minor changes (X.X.X.↑) don't require duplicating all records; major changes require full documentation.
15	When the change is a new minor version (X.X.X.↑), the directory containing the files of the new version will not include the records that have not changed; only the Change Request (CR-*), since the rest of the records will remain accessible inside the directory with the name of the previous major version from which this new version stems. However, for the other minor changes (X.X.↑.X), and major changes (X.↑.X.X) or (↑.X.X.X), we will include all records under the new version, even those that have no changes.	Implemented with clarification per BSI feedback. Documentation approach updated to ensure all changes lead to document updates, new documents, or validation that documents remain applicable.
16	We will add to the GP-012, in the section "Software versioning", in accordance with guidance MDCG 2020-3 Rev.1, the clarification that updating a SOUP or fixing a bug will be subject to evaluation, and that in some cases will constitute a minor version change (X.X.X.↑).	Implemented. Each change is individually assessed for impact per EN 62304, 6.3, 8.2 and 8.3.
17	In the case where the CAPA is found before the transfer of the device into the market: we should not create a CAPA: in accordance with 62304, the right approach is to analyse it during design review.	Implemented. Pre-market issues handled through design review process.
18	We will close the CAPAs mistakenly created, for example [R006001-85] 2024-05-13_R-006-001_Anomalies detected after running penetration test. That should not be a CAPA, but a new activity stemming from an existing SRS.	Implemented. Legacy CAPAs from previous device (Legit.Health under MDD) appropriately closed and not carried forward to Legit.Health Plus.
19	In the case where the CAPA is found after the transfer of the device into the market: we will create a Change Request (CR-*), per EN 62304.	Implemented. Post-market changes follow GP-023 Change Control Management.
20	There was a misconception in our team, where we did not correctly understand the difference between "Release" and "Transfer". The document REL001 we submitted, represented the transfer. However, it was not even good as a transfer. We will submit the corrected: Verified Version Release (REL-) and the Validated Version Transfer (TRA-_).	Implemented. Clear separation between R-TF-012-038 Verified Version Release and R-TF-012-039 Validated Version Transfer.

Evidence:

• R-TF-013-002 Risk Management Record
• R-TF-012-043 Traceability Matrix
• R-TF-012-033 Software Test Plan
• R-TF-012-034 Software Test Description
• R-TF-012-038 Verified Version Release
• R-TF-012-039 Validated Version Transfer
• R-TF-012-019 SOUPs
• R-TF-012-030 Software Configuration Management Plan

NCR 2650005-202501-M3 (Major) - Clinical Evaluation

Non-Conformity Statement (BSI)

Clinical assessment of Legit.Health Plus is incomplete based on the requirements of Annex XIV and Article 61. Claims made are not supported by sufficient evidence, specified, and justified by the manufacturer.

The clinical evaluation does not provide sufficient assessment of evidence to support clinical benefits and outcomes, and safety and performance of the device under the intended purpose.

• Articles Affected: Article 7, Article 61.1, GSPR 1
• Annex XIV: 2, 3

Root Cause (Original)

The CEP and CER documents were not properly prepared in accordance with the MDR. Limited experience in the preparation of these documents and the absence of expert guidance on the appropriate presentation of clinical evidence.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	Regarding the clinical benefits: The clinical benefits and their acceptance criteria will be defined based on the state-of-the-art and substantiated through the benefit-risk analysis, in accordance with MDR requirements.	Implemented. Created R-TF-015-011 State of the Art document establishing benchmarks. Clinical benefits and acceptance criteria defined in R-TF-015-001 CEP (section "Intended clinical benefits", Table with IDs 7GH, 3KX, 8PL, 1QF, 9VW, 5RB, 0ZC) with quantified magnitude of benefit claimed.
2	We will re-write the clinical benefits in a way that they are less ambiguous and more closely tied to the outcome parameters they are measured by. This will be found in the R-TF-015-001 CEP, in the section "Device description", specifically inside the heading "Intended clinical claims".	Implemented. R-TF-015-001 CEP section "Intended clinical benefits" includes table with 7 clinical benefits (IDs 7GH, 3KX, 8PL, 1QF, 9VW, 5RB, 0ZC), each with specific "Means of measure" and "Magnitude of benefit claimed" columns.
3	We will clearly identify the outcome parameters by which we measure the clinical benefits. This will be found in the R-TF-015-001 CEP, inside the heading "Clinical benefits".	Implemented. R-TF-015-001 CEP section "Intended clinical benefits" defines outcome parameters for each benefit (e.g., Top-1 accuracy, AUC, sensitivity, specificity, ICC, reduction in waiting time).
4	We will clearly identify the acceptance criteria: the minimum required value in the above mentioned outcome parameter to determine the clinical aspects of the indication for use. This will be found in the R-TF-015-001 CEP, inside the heading "Clinical benefits".	Implemented. R-TF-015-001 CEP section "Intended clinical benefits" includes "Magnitude of benefit claimed" column with quantified acceptance criteria (e.g., AUC ≥90%, sensitivity ≥79%, specificity ≥87%, ICC ≥72.70%).
5	Regarding safety and performance outcome parameters: Safety and performance claims and their acceptance criteria will be based on the state-of-the-art.	Implemented. R-TF-015-011 State of the Art establishes benchmarks. R-TF-015-001 CEP section "Safety outcome parameters" defines safety claims with means of measure and acceptance criteria based on state-of-the-art comparison.
6	We will include, alongside the already existing list of GSPR: The outcome parameters by which we measure the safety and performance claims, the acceptance criteria: the minimum required value in the above mentioned outcome parameter.	Implemented. R-TF-008-001 GSPR updated. R-TF-015-001 CEP section "Safety outcome parameters" includes table with potential harm, means of measure, and endpoint/acceptance criteria linked to GSPR 1, 8, and 17.
7	We will clearly identify in the Risk file (new column in the risk matrix) and in the clinical file (CER) the Verification of Effectiveness of Risk Controls. We will confirm that the risk control actually reduces the identified risk to an acceptable level in the validation phase.	Implemented. Risk matrix includes verificationOfEffectiveness column with specific references.
8	The new revision of the clinical evaluation report (R-TF-015-003 CER) will include a comprehensive analysis of the available scientific literature relating to the state of the art in the relevant medical field, including clinical data on the device under evaluation and its equivalent devices.	Implemented. R-TF-015-003 CER references R-TF-015-011 State of the Art for literature analysis. CER section "Clinical data generated and held by the manufacturer" includes clinical data from 8 pivotal studies and legacy device data per equivalence claim.
9	Regarding the equivalence, we will refrain from including this information in the GP-015 Clinical evaluation, and move it to the R-TF-015-003 CER, since we understand it is confusing.	Implemented. Equivalence assessment moved to R-TF-015-003 CER section "Demonstration of equivalence" with subsections: Technical equivalence, Clinical equivalence, Biological equivalence, and Conclusions regarding equivalence.
10	We will fix the table referencing Guideline MDCG 2020-5 so that it contains all the information; now things are missing.	Implemented. R-TF-015-003 CER section "Demonstration of equivalence" includes complete tables per MDCG 2020-5: Technical equivalence table (15 characteristics) and Clinical equivalence table (6 characteristics) with side-by-side comparison.
11	We will further explain, with significantly more depth similarities and differences with the legacy device. Now we only mention them, we will include in-depth explanations to prove clinical and technical equivalence. This will be found in R-TF-015-003 CER, inside a heading called "Demonstration of equivalence", inside the section "Device description".	Implemented. R-TF-015-003 CER section "Demonstration of equivalence" includes detailed analysis with conclusions for each equivalence type and section "Justification for Additional Clinical Evidence versus the Legacy Device" explaining regulatory approach.
12	We will explain that the legacy device and the device under evaluation are identical in terms of technology, with only minor modifications described inside the section "Device description".	Implemented. R-TF-015-003 CER section "Technical equivalence" and "Technical equivalence conclusion" confirm identical core architecture, algorithms, and performance specifications. Section "Regulatory Approach to Legacy and Plus Device Technical Documentation" explains the documentation strategy.
13	We will explain the clinical validation we have conducted to validate the subject device, including clarifying what clinical evidence of the legacy device we are using to justify the intended use, providing proper scientific justification of the clinical impact of differences per Annex XIV (3).	Implemented. R-TF-015-003 CER section "Clinical data generated and held by the manufacturer" includes legacy device study (LEGIT_MC_EVCDAO_2019) and 8 pivotal studies with frozen version. Section "Justification for Additional Clinical Evidence versus the Legacy Device" provides scientific justification per Annex XIV (3) for reclassification from Class I to Class IIb.
14	Clarifying the new ad-hoc clinical studies we have performed with the subject device to fulfill the specific new regulatory requirements of the MDR. This will be found in the R-TF-015-008 Clinical development plan, inside the section "Clinical investigation".	Implemented. Nine clinical investigations documented in R-TF-015-001 CEP (section "Clinical Development Plan" and "Clinical investigation") with full protocols and reports in R-TF-015-006 Series.
15	We will state the limitations found in the PMS Report of the legacy, will be detailed in the R-TF-015-008 Clinical development plan of the subject device, alongside the mitigation measures to reduce the impact of the limitations.	Implemented. The R-TF-007-001 PMS Plan of the subject device includes as one of its inputs the information collected from the legacy device during previous years. As stated in the PMS Plan, the PMS Report of the legacy device will be one of the inputs for the first PSUR of Legit.Health Plus. Limitations and mitigation measures are addressed in R-TF-015-003 Clinical Evaluation Report.
16	We will add an assessment confirming and explaining how the clinical data is sufficient to support safety and performance requirements, summarize how the assessment considers all sources: literature search, clinical investigations, PMS and PMCF activities of the legacy device.	Implemented. Comprehensive assessment in R-TF-015-003 Clinical Evaluation Report covering: literature search (R-TF-015-011 State of the Art), clinical investigations (R-TF-015-006 Series), and planned PMS/PMCF activities (R-TF-007-001 PMS Plan, R-TF-007-002 PMCF Plan).
17	We will include how PMS and PMCF activities for the subject device will complement the clinical data in the future.	Implemented. Future PMS/PMCF activities documented in R-TF-007-001 PMS Plan (data collection activities and integration with risk management) and R-TF-007-002 PMCF Plan (specific PMCF methods addressing CER gaps).

Evidence:

• R-TF-015-001 Clinical Evaluation Plan (includes Clinical Development Plan)
• R-TF-015-003 Clinical Evaluation Report
• R-TF-015-011 State of the Art
• Clinical Investigation Reports
• R-TF-013-002 Risk Management Record

NCR 2650005-202501-M4 (Major) - PMCF Plan

Non-Conformity Statement (BSI)

Assessment of whether and how PMCF activities meet the requirements of PMCF in Annex XIV 5, 6.1 and 6.2 is absent.

Evidence is not provided that the PMCF activities will allow for sufficient collection of clinical data on safety and performance of the device throughout its lifetime.

Sufficient justification of the methodology including justification of endpoints and how a sufficient sample size will be ensured in order to power results, statistical power calculation is not found.

• Annex XIV: 5, 6

Root Cause (Original)

The PMCF documents were not properly prepared in accordance with the MDR. Limited experience in the preparation of these documents and the absence of expert guidance on the appropriate presentation of clinical evidence.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	Modify the R-TF-007-002 Post-Market Clinical Follow-up (PMCF) Plan so that the activities will be: Review of the clinical literature, PMCF Investigations (we will update the list of studies, because we have improved the plan since the last submission), Planned Real-world Evidence (RWE) analysis: concordance between the output of the device and HCP confirmation, specifically for the ICD Multiclass Classifier processor, when accessing the severity assessment feature, Data of similar devices.	Implemented. R-TF-007-002 PMCF Plan section "PMCF activities" includes: General PMCF Methods (clinical literature review, user feedback, PMS data analysis) and Specific PMCF Methods with 8 targeted activities (A.1-A.3 for triage/malignancy, B.1-B.4 for severity assessment, C.1-C.2 for diagnostic performance monitoring).
2	All of them, as well as the description of the activity, will include: a detailed explanation on the methodology, a detailed explanation on the purpose of the activity and the rationale, justification on sample size and how it offers sufficient data to power the results, and, among other things: description of the endpoints, comparison method (gold standard), etc.	Implemented. R-TF-007-002 PMCF Plan includes for each activity: study code, type (observational/prospective/retrospective), intended start date, primary endpoints with quantified acceptance criteria, and gold standard methodology.
3	The expected lifetime will specify a number, instead of saying that it's unlimited. This will be seen inside the section "PMCF activities".	Implemented. R-TF-007-002 PMCF Plan specifies expected device lifetime. PMCF Evaluation Report estimated completion date: January 2027.
4	Regarding the R-TF-007-001 Post-Market Surveillance Plan (PMS) of the subject device, one of the inputs will be the PMS Report of the legacy device, that will be attached in the submission, and mentioned in the R-TF-007-001 Post-Market Surveillance Plan (PMS).	Implemented. R-TF-007-001 PMS Plan section "Legacy Device Data" specifies that historical data from the legacy device is one of the inputs. The PMS Report of the legacy device will be an input for the first PSUR of Legit.Health Plus.

Evidence:

• R-TF-007-002 Post-Market Clinical Follow-up (PMCF) Plan
• R-TF-007-001 Post-Market Surveillance (PMS) Plan

NCR 2650005-202501-M5 (Major) - AI/ML Documentation

Non-Conformity Statement (BSI)

Within the AI Expert Review the following gaps against GSPR 17.2 and Annex II, 6.1 (a) and (b) were identified:

Rationales provided to justify alignment with the state-of-the-art were not appropriately documented in the technical file. This applies to multiple instances: model description; dataset description and corresponding V&V evidence on provenance, quality and representativity; model robustness; bias analysis and corresponding requirements and V&V evidence that all the models within the device are free of unwanted biases; change management plan, where the appropriate classification of changes – and the corresponding processes – has not been correctly provided.

• GSPRs Affected: GSPR 17.2
• Annex II: 6.1 (a), (b)

Root Cause (Original)

The issue relates primarily to the presentation of documentation, the organization of available data, and the review of processes to ensure that the required records are properly generated and can adequately be reviewed. Also, the explanation on version changes was not comprehensive enough to explain the impact of different cases of re-training.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	Modify the R-TF-012-006 Lifecycle plan and report, specifically under section "List of processors", so that each processor includes, as well as a short description, the following information: model description, including all the architectures and the rationale for choosing them for the target task, dataset description and corresponding verification and validation evidence on provenance.	Implemented with dedicated AI/ML documentation suite. Created R-TF-028-001 AI/ML Description with comprehensive model documentation instead of modifying R-TF-012-006.
2	We will modify all the Software Requirement Specification (SRS-*)s that stem from URS-001, URS-002, URS-003 and URS-004, to include quality and representativity of the data used for training, test model robustness, using techniques like removing the lesion from the image or adding problem-specific artefacts to the images, among other commonly used techniques, bias analysis and corresponding requirements.	Implemented with dedicated documents. R-TF-028-011 AI/ML Risk Matrix provides comprehensive bias analysis. R-TF-028-009 AI/ML Design Checks covers robustness testing.
3	We will describe in further detail the multiple bias prevention techniques that we have used for the processor. This will happen in the following way: the clinical URSs (URS-001, URS-002, URS-003 and URS-004) will each result in a respective SRS that refers specifically to bias prevention.	Implemented. Bias prevention centralized in R-TF-028-011 AI/ML Risk Matrix providing better auditability.
4	We will generally improve the GP-012 so that, as well as being compliant with 62304, it also references AI/ML models-related MDR requirements, as well as guidance MDCG 2020-3 Rev.1 and other relevant regulatory requirements AI Act.	Implemented. GP-012 updated with AI/ML and AI Act references.
5	We will modify the section "Software versioning" from GP-012, to explain which AI/ML model changes constitute major or minor changes, in accordance with MDCG 2020-3 Rev.1. Patch fix change (X.X.X.↑): Never. All re-training of AI/ML models are minor or major.	Implemented as proposed.
6	Minor change (X.X.↑.X): Retraining when same architecture, modified training configuration, training dataset modifications (adding images, removing images, fixing labels) with validation and performance evaluation using same fixed test dataset.	Implemented with clarification per BSI feedback. Dataset modifications require evaluation and may be minor if validation pipeline unchanged.
7	Major change (X.↑.X.X): Retraining when using a different architecture that does not maintain the neuron number, different architecture with consistent neuron count and output task, different inputs, multitask learning, output normalization, increasing of the number of visible signs.	Clarified per BSI feedback. Architecture changes are now classified as major regardless of maintaining same output neurons. All changes undergo individual impact assessment per MDCG 2020-3 Rev.1.

Evidence:

• R-TF-028-001 AI/ML Description
• R-TF-028-002 AI/ML Development Plan
• R-TF-028-009 AI/ML Design Checks
• R-TF-028-011 AI/ML Risk Matrix

NCR 2650005-202501-N1 (Minor) - IFU and Labeling

Non-Conformity Statement (BSI)

The information supplied by the manufacturer is not fully compliant to GSPR 23.

Evidence that the medium, format and location of the label and instructions for use are appropriate to the device, its intended purpose and its intended users per GSPR 23.1 (a) is not fully demonstrated for clinical users.

A clear specification of the intended users is not observed in the IFU per GSPR 23.4 (b). Training and qualification requirements for IT professionals are not observed in the IFU per GSPR 23.4(j).

• GSPRs Affected: GSPR 23.1 (a), (f); 23.4 (b), (j)

Root Cause (Original)

Insufficient training in the preparation of these documents and confusion about the right method to deliver the IFU and label information, due to the API nature of the device.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	We will provide the IFU in two ways: An online, easily-accessible, user friendly URL: https://apidocs-draft.legit.health/ (the URL will change once the device is CE marked and transferred to the market). This will be accessible to the clinical users. A link to the above mentioned URL in the JSON output of the device.	Implemented. (1) Electronic IFU accessible at https://apidocs-draft.legit.health/docs/1.1.0.0/. (2) JSON output includes eIFU field with URL to the IFU.
2	We will provide the labels in two ways: Inside the user-friendly URL, Embedded in the JSON output of the device.	Implemented. (1) Label in IFU section "Label" at https://apidocs-draft.legit.health/docs/1.1.0.0/general-information/label. (2) JSON output includes label data (UDI, manufacturer, version, etc.).
3	We will add to the IFU a form to request the IFU in paper format at no cost, including the time within they can expect to receive it.	Implemented. IFU section "Request paper IFU" at https://apidocs-draft.legit.health/docs/1.1.0.0/request-paper-ifu with form and delivery timeframe (7 calendar days).
4	We will add to the IFU a section called "User interface" explaining the UI elements that an ITP may develop for their users to interact. In this section, we will include a recommended way of displaying to users the information of the manufacturer and the intended use, according to Annex I, 17.2.	Implemented. IFU section "User interface" at https://apidocs-draft.legit.health/docs/1.1.0.0/installation-manual/user-interface with UI guidelines for ITPs and recommended display of manufacturer info per MDR Annex I, 17.1.
5	A clear specification of the intended users (ITP and HCP), as well as specific instructions for each of the intended users. This will include the qualifications and training by the intended users.	Implemented. IFU section "Intended use or purpose" at https://apidocs-draft.legit.health/docs/1.1.0.0/general-information/intended-use-purpose/eu-intended-purpose with headings "Intended user", "User qualifications and competencies", "Healthcare professionals", and "IT professionals".
6	We will change the document Description and specifications to explain that the intended users are HCP and ITP. This will be found inside the heading "Intended user", specifically inside the heading "User qualification and competencies". This will include the qualifications and training by the intended users.	Implemented. IFU section "Intended use or purpose" at https://apidocs-draft.legit.health/docs/1.1.0.0/general-information/intended-use-purpose/eu-intended-purpose contains heading "User qualifications and competencies" with subsections "Healthcare professionals" and "IT professionals".

Evidence:

• Electronic IFU
• IFU - Label
• IFU - Intended Purpose
• IFU - User Interface
• IFU - Request Paper IFU
• R-TF-001-008 Label

NCR 2650005-202501-N2 (Minor) - Usability Engineering

Non-Conformity Statement (BSI)

Evidence that the device has been developed and manufactured in accordance with state of the art taking into account the principles of development life cycle, risk management and validation per GSPRs 1, 5 and 17.2 is incomplete.

Evidence of usability engineering under EN 62366-1 to which compliance has been claimed to demonstrate conformity to the relevant GSPRs is incomplete.

Documentation of intended users per Annex II, 1.1 (a) is inconsistent. Documentation of product validation per Annex II, 6 is incomplete.

• GSPRs Affected: GSPR 1, 5, 17.2
• Annex II: 1.1 (a); 6.1 (a), (b)

Root Cause (Original)

Insufficient knowledge of ISO 62366 and excessive focus on the ITP user and the interaction with the device via API.

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	We will create a new major version of all usability records (according EN 62366). It will include a Use-Related Risk Analysis (URRA). The risks related to usability will mention the scenario by which they are addressed by.	Implemented. Complete usability engineering file restructured with URRA per GP-025 Usability and Human Factors Engineering procedure.
2	Usability will be conducted separately for the two intended users: HCP - It will be clear that they will have access to the IFU, It will provide evidence for the usability of the IFU and its contents when used in the clinical use environment. ITP - The method for testing will not consist of providing steps for completion. Instead, it will provide scenarios, per EN 62366 5.4 and 5.5, using the IFU alone.	Implemented. A unified protocol (R-TF-025-004) covers both user groups with distinct use scenarios per IEC 62366-1 §5.4 and §5.5. HCP testing completed (n=18); ITP testing completed (n=18).
3	We will refrain from referring to intended users with any other term.	Implemented. Consistent terminology (HCP and ITP) used throughout all usability documentation.
4	As stated in a previous non-conformity, we will modify the record Description and specifications to also clarify the intended users: HCP and IPT.	Implemented as proposed.
5	The scenarios will be representative of normal use conditions.	Implemented as proposed.
6	We will include all necessary information about testers, including their affiliations.	Implemented. HCP participants include affiliation to Hospital Universitario La Paz and Hospital Universitario 12 de Octubre. ITP participants include Software Engineers, DevOps Engineers, Backend Developers, Full Stack Developers, API Integration Specialists, and Systems Integrators.

Evidence:

• GP-025 Usability and Human Factors Engineering
• R-TF-025-001 Usability Plan
• R-TF-025-002 Identification of Characteristics for Safety and Possible Use Errors
• R-TF-025-004 Summative Evaluation Protocol
• R-TF-025-007 Summative Evaluation Report
• R-TF Device Description and Specification
• IFU - Intended Purpose

NCR 2650005-202501-N3 (Minor) - Risk Control Effectiveness Verification

Non-Conformity Statement (BSI)

Evidence of conformity to GSPRs 1 and 4 is incomplete because evidence of risk management under EN ISO 14971 to which compliance has been claimed to demonstrate conformity to the relevant GSPRs is incomplete. Specifically, documentation of verification of effectiveness of risk controls (e.g., per EN ISO 14971, 7.2) is incomplete.

The CER and summative evaluation report have been cited as evidence of effectiveness of risk control measures for several risks in R-TF-013-002 Risk management record. These references alone are insufficient without specific references to the data/test results that demonstrate the effectiveness of the specific control(s).

• GSPRs Affected: GSPR 1, 4, 17.2
• Annex II: 5 (b); 6.1 (a), (b)

Root Cause (Original)

Failed to provide traceability between the risks stated in the R-TF-013-002 Risk management record and the Test plans and Test runs (PLAN-, TEST-) that provide evidence of verification of effectiveness of risk controls. Also, as explained in the previous non-conformity, a deficient implementation of usability testing, specially for the clinical user (HCP).

Corrective Actions

#	Original CAPA Plan Proposal	Actual Implementation
1	We will include in the column "Verification of implementation of risk control measures (Risk control)" of the R-TF-013-002 Risk Management records concrete references to the specific Test plans and Test runs (PLAN-, TEST-) and/or other documents that verify the effectiveness of the relevant risk controls. Effectiveness of risk controls may be demonstrated through different types of evidence depending on the nature of the control.	Implemented with enhanced structure. Added both verificationOfImplementation and verificationOfEffectiveness fields to risk matrix.
2	We will make sure that every risk has at least one objective test reference.	Implemented. Each risk now has specific test case references (e.g., "C106, C454, C455...").
3	As explained in the previous non-conformity, we will also re-do the usability testing, and it will include both intended users (HCP and IPT). It may be cited as evidence of verification of effectiveness of risk controls for some risks, especially those related to usability.	Implemented. See NCR N2 above.
4	We will create an in-depth risk assessment specific to usability (URRA) that we will conduct alongside the new usability engineering documentation. These risks will be added to the R-TF-013-002 Risk management record. This means that we will expand on the risks, including those mentioned in GSPR 5 regarding risks related to ergonomic features.	Implemented. Use-Related Risk Analysis integrated into risk management.
5	Likewise, the CER may be cited as evidence of verification of effectiveness of risk control. We will modify the CER so that, in those cases, it is better explained how it serves as verification.	Implemented. CER updated with specific sections demonstrating risk control effectiveness.
6	When we relink risks to tests, we will include pass/fail results, IDs and dates so that it is easier to trace and audit.	Implemented. Test references include IDs and pass/fail status.
7	We will clearly identify in the Risk file (new column in the risk matrix) and in the clinical file (CER) the Verification of Effectiveness of Risk Controls. We will confirm that the risk control actually reduces the identified risk to an acceptable level in the validation phase.	Implemented. Risk matrix now clearly separates verificationOfImplementation (Phase 4) from verificationOfEffectiveness (Phase 5 and post-market).

Clarification per BSI feedback

Verification of effectiveness may be demonstrated through different types of evidence depending on the control nature:

• Verification tests: For technical controls (e.g., software verification)
• Validation tests: For commissioning controls
• Usability testing: For user interaction controls
• Clinical evaluation: For clinical decision-making controls
• Visual inspection: For labeling controls

Evidence:

• R-TF-013-002 Risk Management Record
• R-TF-013-003 Risk Management Report

Summary of Status

NCR Number	Category	Area	CAP Status	Final Status
2650005-202501-M1	Major	Technical - V&V	✅ ACCEPTED	✅ Implemented
2650005-202501-M2	Major	Technical - 62304	✅ ACCEPTED	✅ Implemented
2650005-202501-M3	Major	Clinical	✅ ACCEPTED	✅ Implemented
2650005-202501-M4	Major	Clinical - PMCF	✅ ACCEPTED	✅ Implemented
2650005-202501-M5	Major	AI	✅ ACCEPTED	✅ Implemented
2650005-202501-N1	Minor	Technical - IFU	✅ ACCEPTED	✅ Implemented
2650005-202501-N2	Minor	Technical - Usability	✅ ACCEPTED	✅ Implemented
2650005-202501-N3	Minor	Technical - Risk	✅ ACCEPTED	✅ Implemented

Key Changes Summary

Naming Convention Changes

CAPA Plan Original	Technical File Implementation	Reason
User Requirement (URS-*)	R-TF-012-031 Product Requirement Specification (PRS)	Better alignment with MDR terminology
PLAN-, TEST-	R-TF-012-033 Software Test Plan, R-TF-012-034 Software Test Description, R-TF-012-035 Software Test Report	Consolidated into three structured documents per IEC 62304
REL-* (Verified Version Release)	R-TF-012-038 Verified Version Release	Consistent document numbering
TRA-* (Validated Version Transfer)	R-TF-012-039 Validated Version Transfer	Consistent document numbering
R-TF-012-009 (ML Validation)	R-TF-028-001 AI/ML Description through R-TF-028-011 AI/ML Risk Matrix	Expanded to comprehensive AI/ML suite

New Documents Created

Document	Purpose
R-TF-015-011 State of the Art	Establish clinical benchmarks and acceptance criteria
R-TF-028-001 AI/ML Description	Comprehensive model documentation
R-TF-028-002 AI/ML Development Plan	AI development lifecycle
R-TF-028-003 Data Collection Instructions	Dataset provenance and collection methodology
R-TF-028-004 Data Annotation Instructions	Annotation protocols and quality
R-TF-028-005 AI/ML Development Report	Development results and metrics
R-TF-028-006 AI/ML Release	Release validation
R-TF-028-009 AI/ML Design Checks	Robustness testing
R-TF-028-011 AI/ML Risk Matrix	Bias analysis and fairness metrics

Training Completed

As specified in "Actions to Prevent Recurrence":

• ✅ IEC 62304 (Software lifecycle)
• ✅ EN 82304-1 (Health software)
• ✅ ISO 13485 (QMS)
• ✅ ISO 14971 (Risk management)
• ✅ IEC 62366-1 (Usability engineering)
• ✅ MDR requirements
• ✅ AI Act requirements

Responsible Persons

Area	Responsible
Technical (M1, M2)	Alfonso Medela, Taig Mac Carthy, Gerardo Fernández, Alejandro Carmena
Clinical (M3, M4)	Alfonso Medela, Jordi Barrachina
AI (M5)	Alfonso Medela, Taig Mac Carthy
IFU/Usability (N1, N2, N3)	Taig Mac Carthy, Saray Ugidos, Alejandro Carmena

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003, JD-004
• Approver: JD-001