GP-028 AI Development
Table of contents
Purpose
This procedure describes the phases of development, validation, and maintenance in the lifecycle of the Artificial Intelligence (AI) and Machine Learning (ML) models developed by AI Labs Group S.L. It details how we design, develop, test, release, and update the algorithms that power the Legit.Health Plus device, ensuring they are robust, safe, and effective for their intended purpose.
Scope
All AI/ML activities performed by AI Labs Group S.L. for Legit.Health Plus, including new development, maintenance, retraining, and reevaluation of existing models, and any third-party AI components used within the device.
Reference Documents
- ISO 13485:2016 - Medical devices - Quality management systems
- ISO/IEC 42001:2023 - Information technology — Artificial intelligence — Management system
- MDR 2017/745 - Medical Device Regulation
- Guidance MDCG 2019-11 - Qualification and Classification of Software in MDR 2017/745
- Good Machine Learning Practice for Medical Device Development: Guiding Principles
- R-TF-012-009 - Validation and testing of machine learning models
- GP-012 - Design, Redesign and Development
- GP-023 - Change control management
Terms and Definitons
| Term / Abbreviation | Definition |
|---|---|
| Algorithm | A set of mathematical/logical operations coded by the AI team to achieve a specific task, such as image recognition, object detection, or semantic segmentation. |
| AI/ML | Artificial Intelligence and Machine Learning. |
| Data Annotation | The action of associating a label to data (e.g., an ICD category to an image, a bounding box to a lesion) by a qualified person following a given protocol. |
| Data Collection | The action of acquiring data (e.g., clinical images) from various sources, including clinical partners and public datasets, following a given protocol. |
| Training Set | The subset of data used to fit or train the parameters of an AI/ML model. |
| Validation Set | The subset of data used to provide an unbiased evaluation of a model fit on the training set while tuning model hyperparameters. |
| Test Set | A fixed subset of data used to provide an unbiased evaluation of a final model's performance after training is complete. |
| Vision Transformer (ViT) | A specific neural network architecture used for image recognition tasks, inspired by the Transformer architecture. |
| Test-Time Augmentation (TTA) | A technique used to improve model performance by creating multiple augmented versions of a test image and averaging the predictions. |
Process and Responsibilities
Responsibilities
| Role | Responsibilities |
|---|---|
| Technical Manager | - Application of the present procedure. - Overall management of team planning and resources. - Ensuring alignment with other QMS procedures. |
| AI Team | - Data Management (Collection, Annotation, Partitioning). - Design of AI/ML projects (Description, Plan, Risk Assessment). - Development of algorithms (Training, Calibration, Experimentation). - Verification and Validation of algorithms (Reporting, Testing). - Release and subsequent updates of algorithms. |
AI/ML Development Cycle
The AI/ML development cycle is structured into three primary phases, ensuring a systematic and controlled progression from conception to deployment.
Design Phase
Description and Plan
The AI/ML Description (R-028-001) is prepared by the AI/ML team to outline the algorithms to be developed, including their specifications.
The development and deployment of these algorithms require defined resources and structured methodologies. These are documented in the AI/ML Development Plan (R-028-002), which provides details on data management, algorithm training, evaluation processes, release procedures, and AI/ML risk management.
Data Collection and Annotation
The development of algorithms necessitates the use of labeled datasets. A single algorithm may require multiple data collections, each potentially annotated using different methods. Conversely, a single dataset may support the development of multiple algorithms and undergo various annotation processes.
The AI/ML team is responsible for planning data collection and annotation activities in accordance with predefined instructions. All collected data and corresponding annotations are securely stored and fully traceable.
For each data collection, the AI/ML team, in collaboration with the Clinical Operations team, prepares Data Collection Instructions (R-028-003), specifying the methodology for data acquisition.
For each data annotation activity, the AI/ML team—supported by the Medical team—prepares Data Annotation Instructions (R-028-004), detailing the annotation procedures.
Project-specific data management requirements are defined within the AI/ML Development Plan (R-028-002).
AI/ML Risk Management
During the design phase, the AI/ML team develops the AI/ML Risk Management Plan as part of the AI/ML Development Plan (R-028-002). The team also conducts risk identification, analysis, and evaluation, documented in the AI/ML Risk Matrix (R-028-011).
Additionally, the AI/ML team collaborates with the product and software development teams to assess AI/ML-related safety risks, which are recorded in the Safety Risk Matrix (???).
Development Phase
The development phase is a structured, agile, and iterative process in which the AI/ML team designs, trains, evaluates, and refines models to meet the defined performance, safety, and compliance requirements. This stage is critical to ensuring that the resulting algorithms are robust, reliable, and suitable for integration into Legit.Health Plus.
Key activities in this phase include:
-
Model Architecture Design: Evaluating and implementing advanced architectures, such as Vision Transformers (ViT) for image recognition, based on literature reviews and internal experimentation. This may involve exploring novel configurations, adapting pre-existing models, or adjusting hyperparameters to optimize performance.
-
Data Management and Integrity: Preparing datasets through rigorous splitting into training, validation, and test sets, with subject-level separation applied where possible to mitigate data leakage. A fixed test set is maintained for consistent benchmarking. All dataset versions, transformations, and annotations are tracked for full traceability.
-
Model Training and Optimization: Training models using supervised learning, transfer learning from pre-trained weights (e.g., ImageNet), and applying data augmentation strategies to enhance sample diversity and improve generalization. Training pipelines are monitored with tools that record experiments, parameter settings, and results for reproducibility.
-
Model Calibration and Post-Processing: Applying post-processing techniques, such as temperature scaling, to calibrate model output probabilities, ensuring they are well-calibrated, interpretable, and aligned with clinical decision-making requirements. Where necessary, additional pre-processing or output transformations are implemented to meet performance and safety criteria.
-
Evaluation and Performance Monitoring: Defining and refining key performance metrics and success criteria, visualizing results for stakeholder review, and ensuring rigorous separation between training and evaluation data. Performance is assessed not only on accuracy but also on robustness, fairness, and clinical applicability.
Throughout the development phase, interim results and experimental findings are communicated to relevant stakeholders. Any discoveries that impact prior assumptions or specifications may trigger revisions to the AI/ML Description (R-028-001), AI/ML Development Plan (R-028-002), and AI/ML Risk Matrix (R-028-011).
Verification and Validation Phase
Once each algorithm within a package is deemed ready for integration, the AI/ML team prepares an AI/ML Development Report (R-028-005). This report provides a comprehensive account of the algorithm development process, including design, training, tuning, selection, verification, evaluation, and validation, demonstrating compliance with all applicable regulatory requirements. The report also incorporates the AI/ML Risk Assessment, detailing identified risks and mitigation measures.
Upon successful verification and validation of all algorithms within the package, the package is formally released. Implementation guidelines for deploying the package within an SDK by the Software Development team are documented in the AI/ML Release (R-028-006), ensuring proper and consistent integration into the target software environment.
AI/ML Description
The AI team describes the algorithm package with sufficient specificity to guide data collection, annotation, and development activities. This includes the specifications of the required algorithms (e.g., image recognition for ICD categories, object detection for lesion counting) and their intended integration into the Legit.Health Plus device.
AI/ML Development Plan
The AI Team plans the resources and methodologies required to develop the algorithm package described in the AI/ML Description (R-028-001). The plan is a comprehensive document that must detail the entire scope of the development project.
Notably, the plan shall include:
-
Project Objectives: Clear goals for the algorithm package, aligned with the device's intended use.
-
Project Management & Resources: Definition of the human resources, roles, and project management approach.
-
Development Environment: Specification of the hardware, software, and tools (including SOUP as per GP-012) to be used.
-
Data Management Plan: A detailed strategy for data acquisition, curation, annotation, and partitioning. This plan may lead to the creation of specific Data Collection Instructions and Data Annotation Instructions.
-
Training and Evaluation Plan: The methodology for training, tuning, and evaluating the models, following good machine learning practices and the procedures outlined in R-TF-012-009. This includes defining metrics, acceptance criteria, and traceability measures.
-
Release Plan: Details on the deliverables for the Software Development team, including the list of algorithms, documentation, and dependencies.
-
Risk Management Plan: The strategy for managing AI/ML-specific risks, which includes the creation of the first version of the AI/ML Risk Matrix (F-GP-028-009).
After the plan is drafted, it is formally reviewed. Checks are performed using the AI/ML Design Checks (R-028-007) checklist to ensure that the defined specifications and requirements conform with good practices for AI/ML development.
The AI/ML Development Plan is a living document. It may be revised during the development phase based on the AI Team's exploratory work and findings. Consequently, Data Collection Instructions and Data Annotation Instructions may also be updated to meet the evolving data requirements of the project.
Data Collection Instructions
To ensure the acquisition of high-quality, relevant data for algorithm development, the AI Team, in collaboration with the Clinical Operations team, shall define and document formal Data Collection Instructions.
These instructions must provide clear specifications for:
-
Dataset Composition: The required characteristics of the data to be collected, including demographics and clinical presentation, to ensure it is representative of the intended patient population.
-
Dataset Size: An estimation of the required volume of data, with a clear rationale supporting its statistical significance for the intendedmodeling task.
-
Acquisition Protocol: Detailed clinical and technical requirements for data acquisition to ensure consistency and quality across all sources.
The acquisition instructions must be sufficiently specific to enable consistent data collection by qualified personnel and to serve as a verifiable record that the collected data meets the predefined requirements. If a formal clinical investigation is necessary to collect data, it shall be initiated and conducted in accordance with the company's established procedure for clinical investigations.
Data Annotation Instructions
The AI Team, with support from the clinical team, is responsible for providing detailed Data Annotation Instructions to all personnel tasked with labeling datasets. These instructions shall specify precisely how medical experts or other qualified annotators are to label the data required for algorithm development (e.g., applying ICD category labels, drawing bounding boxes for lesions).
Annotation instructions must be unambiguous and serve two primary functions:
-
To act as a clear guide for annotators to ensure consistency and accuracy.
-
To establish a baseline against which the quality of the resulting annotations can be formally evaluated.
All annotators must be formally trained on these instructions before commencing work. A record of this training shall be maintained to ensure traceability and document the competence of the personnel involved.
AI/ML Risk Matrix
As an integral part of the design phase, the AI Team shall conduct an initial risk assessment focused on hazards unique to the AI/ML development lifecycle. The team will identify, analyze, and evaluate potential AI/ML risks related to data management, algorithm training, model evaluation, and deployment.
All identified risks shall be recorded in the AI/ML Risk Matrix (R-028-009). This initial risk analysis forms the basis for ongoing risk management activities throughout the project.
It is critical to recognize that these AI/ML-specific risks can contribute to or result in broader safety risks for the medical device. Therefore, any identified "safety risks related to AI/ML" must be formally communicated to the product and software development teams for inclusion in the overall safety risk management file.
AI/ML Development Report
The development report summarizes how the algorithms were created and validated. It provides evidence that the models meet their predefined acceptance criteria and are fit for their intended purpose. The report includes detailed sections on data management, algorithm training, and a comprehensive evaluation of performance using the metrics specified in R-TF-012-009.
Data Management PENDING
Algorithm Training PENDING
Algorithm Evaluation PENDING
AI/ML Release
The formal transfer of a new or updated algorithm package from the AI Team to the Software Development team is managed through an AI/ML Release.
To facilitate a smooth integration, design transfer occurs proactively through collaborative sprint reviews attended by both teams. This allows the Software Development team to begin integration work even before the final algorithms are formally released.
The complete release package provided to the Software Development team shall include:
-
Algorithm Package: All required algorithms, models, and their associated configuration files necessary for deployment.
-
Release Report (R-028-004): Comprehensive documentation detailing all instructions the Software Development team must follow to correctly integrate the algorithm package into the target software.
-
Technical Support: Ongoing support from the AI Team to assist the Software Development team with the integration process.
Prior to release, all deliverables are verified to ensure they were developed and packaged as expected. These checks are formally recorded using the AI/ML V&V Checks (R-028-008) checklist.
A new AI/ML Release is not required for algorithm updates that do not alter the implementation or integration interface of the package.
AI/ML Updates
The lifecycle of AI/ML algorithms extends beyond their initial release. This section outlines the controlled processes for managing post-release updates to algorithm packages, which are categorized as either Retraining or Reevaluation.
Any update to an AI/ML model requires a formal risk assessment. The AI/ML risks associated with the update must be analyzed, and a benefit/risk impact assessment shall be conducted. Any potential new or modified safety risks related to AI/ML resulting from the update must be assessed and communicated to the relevant teams.
Retraining
Retraining is performed when an algorithm's core logic or data foundation is modified. This includes training on new or updated data, implementing a new model architecture, or changing key parameters/hyperparameters.
When an algorithm is retrained, the process must follow the appropriate AI/ML Development Plan. Upon completion, an AI/ML Retraining Report (R-028-005) shall be produced. This report must describe:
-
Modifications and Rationale: A clear description of the changes made to the predicate algorithm and the goals for the retraining effort.
-
Data Management: Details of any new data collection or annotation activities.
-
Training Process: A summary of the training methodology if it differs from the predicate.
-
Performance Testing: A comprehensive evaluation of the retrained algorithm's performance.
Performance testing must demonstrate, using the same test data and metrics as the predicate algorithm, that the update shows non-regression in performance and meets the success criteria defined in the original AI/ML Development Report or the updated AI/ML Development Plan.
A new revision of the AI/ML Release (R-028-004) shall be issued to the Software Development team, and the release must be verified using the AI/ML V&V Checks (R-028-008).
A retraining must not involve modifications to the algorithm's fundamental input or output specifications. If such changes are required, the work is considered a new project, which necessitates the creation of a new AI/ML Description, Development Plan, Development Report, and Release.
Reevaluation PENDING
Reevaluation is performed when an existing, unchanged algorithm is evaluated against new test data or new performance metrics, typically driven by a change in product requirements.
If the reevaluation is driven by changes to product requirements, the AI/ML Description (R-028-001) and AI/ML Development Plan (R-028-002) may need to be revised first. The reevaluation process itself is governed by the appropriate AI/ML Development Plan.
Upon completion, an AI/ML Reevaluation Report (R-028-006) shall be produced. This report must describe:
-
Modifications and Rationale: A description of the changes to the evaluation plan from the original Development Report and the rationale for the reevaluation.
-
Data Management: Details of any new data collection or annotation activities used for the new test set.
-
Performance Testing: The results of the performance testing on the new test data, using both the original metrics and any new metrics.
The performance testing should demonstrate that the algorithm shows similar performance on the new testing data and/or meets the success criteria defined in the original AI/ML Development Report or the updated AI/ML Development Plan.
Related QMS Documents
- T-028-001 AI/ML Description
- T-028-002 AI/ML Development Plan
- T-028-003 Data Collection Instructions
- T-028-004 Data Annotation Instructions
- T-028-005 AI/ML Development Report
- T-028-006 AI/ML Release
- T-028-007 AI/ML Retraining Report
- T-028-008 AI/ML Reevaluation Report
- T-028-009 AI/ML Design Checks
- T-028-010 AI/ML V&V Checks
- T-028-011 AI/ML Risk Matrix
Signature meaning
The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:
- Author: Team members involved
- Reviewer: JD-003, JD-004
- Approver: JD-001