Skip to main content
QMSQMS
QMS
  • Welcome to your QMS
  • Quality Manual
  • Procedures
    • GP-001 Control of documents
    • GP-002 Quality planning
    • GP-003 Audits
    • GP-004 Vigilance system
    • GP-005 Human Resources and Training
    • GP-006 Non-conformity, Corrective and Preventive actions
    • GP-007 Post-market surveillance
    • GP-008 Product requirements
    • GP-009 Sales
    • GP-010 Purchases and suppliers evaluation
    • GP-011 Provision of service
    • GP-012 Design, Redesign and Development
      • Deprecated
      • Templates
        • T-012-001 Requirements
        • T-012-003 Test run
        • T-012-004 Software version release
        • T-012-005 Design change control
        • T-012-006 _Product name_ life cycle plan and report_YYYY_nnn
        • T-012-007 Formative evaluation plan_YYYY_nnn
        • T-012-008 Formative evaluation report_YYYY_nnn
        • T-012-009 Validation and testing of machine learning models_YYYY_nnn
        • T-012-010 Device backup verification_YYYY_nnn
        • T-012-012 Customers product version control_YYYY_nnn
        • T-012-013 Design stage review
        • T-012-014 Summative evaluation plan_YYYY_nnn
        • T-012-015 Summative evaluation report YYYY_nnn
        • T-012-016 Software usability test guide
        • T-012-017 Integration test review
        • T-012-018 Test plan
        • T-012-019 SOUP
        • T-012-020 Predetermined Change Control Plan
      • Specific procedures
    • GP-013 Risk management
    • GP-014 Feedback and complaints
    • GP-015 Clinical evaluation
    • GP-016 Traceability and identification
    • GP-017 Technical assistance service
    • GP-018 Infrastructure and facilities
    • GP-019 Software validation plan
    • GP-020 QMS Data analysis
    • GP-021 Communications
    • GP-022 Document translation
    • GP-023 Change control management
    • GP-024 Cybersecurity
    • GP-025 Usability and Human Factors Engineering
    • GP-027 Corporate Governance
    • GP-050 Data Protection
    • GP-051 Security violations
    • GP-052 Data Privacy Impact Assessment (DPIA)
    • GP-100 Business Continuity (BCP) and Disaster Recovery plans (DRP)
    • GP-101 Information security
    • GP-200 Remote Data Acquisition in Clinical Investigations
    • GP-026 Market-specific product requirements
  • Records
  • Legit.Health Plus Version 1.1.0.0
  • Legit.Health Plus Version 1.1.0.1
  • Licenses and accreditations
  • External documentation
  • Procedures
  • GP-012 Design, Redesign and Development
  • Templates
  • T-012-015 Summative evaluation report YYYY_nnn

T-012-015 Summative evaluation report YYYY_nnn

Objective​

This study aims to validate through objective and empirical evidence that the intended users of the medical device can use the device easily and safely.

Abbreviations and definitions​

  • Instruction for use: following the Medical Device Regulation, the term `instructions for use refers to the information provided by the manufacturer to inform the user of a device's intended purpose and proper use and of any precautions to be taken
  • Intended user: Users are defined in IEC 62366-1:2015 as persons interacting with (i.e. operating or handling) the medical device.
  • Summative user testing: It (quantitative testing) provides an indirect assessment of the usability of the design. A group of testers are given a task, and based on their performance, the usability quotients of the design elements are measured.
  • Tester: a person who has been asked to perform the tasks described in the present protocol.
  • eIFU : Electronic instruction for use
  • IFU: Instruction for use
  • URRA: Use related risk analysis

Device description​

Device Identification​

Information
Device nameLegit.Health Plus (hereinafter, the device)
Model and typeNA
Version1.1.0.0
Basic UDI-DI8437025550LegitCADx6X
Certificate number (if available)MDR 792790
EMDN code(s)Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code65975
EU MDR 2017/745Class IIb
EU MDR Classification ruleRule 11
Novel product (True/False)FALSE
Novel related clinical procedure (True/False)FALSE
SRNES-MF-000025345

Intended use

The device is a computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures, enhancing efficiency and accuracy of care delivery, by providing:

  • an interpretative distribution representation of possible International Classification of Diseases (ICD) categories that might be represented in the pixels content of the image
  • quantifiable data on the intensity, count and extent of clinical signs such as erythema, desquamation, and induration, among others

Quantification of intensity, count and extent of visible clinical signs

The device provides quantifiable data on the intensity, count and extent of clinical signs such as erythema, desquamation, and induration, among others; including, but not limited to:

  • erythema,
  • desquamation,
  • induration,
  • crusting,
  • xerosis (dryness),
  • swelling (oedema),
  • oozing,
  • excoriation,
  • lichenification,
  • exudation,
  • wound depth,
  • wound border,
  • undermining,
  • hair loss,
  • necrotic tissue,
  • granulation tissue,
  • epithelialization,
  • nodule,
  • papule
  • pustule,
  • cyst,
  • comedone,
  • abscess,
  • draining tunnel,
  • inflammatory lesion,
  • exposed wound, bone and/or adjacent tissues,
  • slough or biofilm,
  • maceration,
  • external material over the lesion,
  • hypopigmentation or depigmentation,
  • hyperpigmentation,
  • scar,
  • ictericia

Image-based recognition of visible ICD categories

The device is intended to provide an interpretative distribution representation of possible International Classification of Diseases (ICD) categories that might be represented in the pixels content of the image.

Device description

The device is a computational software-only medical device leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures. Its principal function is to provide a wide range of clinical data from the analyzed images to assist healthcare practitioners in their clinical evaluations and allow healthcare provider organisations to gather data and improve their workflows.

The generated data is intended to aid healthcare practitioners and organizations in their clinical decision-making process, thus enhancing the efficiency and accuracy of care delivery.

The device should never be used to confirm a clinical diagnosis. On the contrary, its result is one element of the overall clinical assessment. Indeed, the device is designed to be used when a healthcare practitioner chooses to obtain additional information to consider a decision.

Intended medical indication

The device is indicated for use on images of visible skin structure abnormalities to support the assessment of all diseases of the skin incorporating conditions affecting the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails) and associated mucous membranes (conjunctival, oral and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).

Intended patient population

The device is intended for use on images of skin from patients presenting visible skin structure abnormalities, across all age groups, skin types, and demographics.

Intended user

The medical device is intended for use by healthcare providers to aid in the assessment of skin structures.

User qualifications and competencies

This section outlines the qualifications and competencies required for users of the device to ensure its safe and effective use. It is assumed that all users already possess the baseline qualifications and competencies associated with their respective professional roles.

Healthcare professionals

No additional official qualifications are required for healthcare professionals (HCPs) to use the device. However, it is recommended that HCPs possess the following competencies to optimize device utilization:

  • Proficiency in capturing high-quality clinical images using smartphones or equivalent digital devices.
  • Basic understanding of the clinical context in which the device is applied.
  • Familiarity with interpreting digital health data as part of the clinical decision-making process.

The device may be used by any healthcare professional who, by virtue of their academic degree, professional license, or recognized qualification, is authorized to provide healthcare services. This includes, but is not limited to:

  • Medical Doctors (MD, MBBS, DO, Dr. med., or equivalent)
  • Registered Nurses (RN, BScN, MScN, Dipl. Pflegefachfrau/-mann, or equivalent)
  • Nurse Practitioners (NP, Advanced Nurse Practitioner, or equivalent)
  • Physician Assistants (PA, or equivalent roles such as Physician Associate in the UK/EU)
  • Dermatologists (board-certified, Facharzt für Dermatologie, or equivalent)
  • Other licensed or registered healthcare professionals as recognized by local, national, or European regulatory authorities

Each HCP must hold the academic title, degree, or professional registration that confers their status as a healthcare professional in their jurisdiction, whether in the United States, Europe, or other regions where the device is provided.

IT professionals

IT professionals are responsible for the technical integration, configuration, and maintenance of the medical device within the healthcare organization's information systems.

No specific official qualifications are mandated. Nevertheless, it is advisable that IT professionals involved in the deployment and support of the device have the following competencies:

  • Foundational knowledge of the HL7 FHIR (Fast Healthcare Interoperability Resources) standard and its application in healthcare data exchange.
  • Ability to interpret and manage the device's data outputs, including integration with electronic health record (EHR) systems.
  • Understanding of healthcare data privacy and security requirements relevant to medical device integration, including GDPR (Europe), HIPAA (US), and other applicable local regulations.
  • Experience with troubleshooting and supporting clinical software in a healthcare environment.
  • Familiarity with IT standards and best practices for healthcare, such as ISO/IEC 27001 (Information Security Management) and ISO 27799 (Health Informatics—Information Security Management in Health).

IT professionals may include, but are not limited to:

  • Health Informatics Specialists (MSc Health Informatics, or equivalent)
  • Clinical IT System Administrators
  • Healthcare Integration Engineers
  • IT Managers and Project Managers in healthcare settings
  • Software Engineers and Developers specializing in healthcare IT
  • Other IT professionals with relevant experience in healthcare environments, as recognized by local, national, or European authorities

Each IT professional should possess the relevant academic degree, professional certification, or demonstrable experience that qualifies them for their role in the healthcare organization, in accordance with the requirements of the United States, Europe, or other regions where the device is provided.

Use environment

The device is intended to be used in the setting of healthcare organisations and their IT departments, which commonly are situated inside hospitals or other clinical facilities.

The device is intended to be integrated into the healthcare organisation's system by IT professionals.

Operating principle

The device is computational medical tool leveraging computer vision algorithms to process images of the epidermis, the dermis and its appendages, among other skin structures.

Body structures

The device is intended to use on the epidermis, its appendages (hair, hair follicle, sebaceous glands, apocrine sweat gland apparatus, eccrine sweat gland apparatus and nails) and associated mucous membranes (conjunctival, oral and genital), the dermis, the cutaneous vasculature and the subcutaneous tissue (subcutis).

In fact, the device is intended to use on visible skin structures. As such, it can only quantify clinical signs that are visible, and distribute the probabilities across ICD categories that are visible.

Explainability

For visual signs that can be quantified in terms of count and extent, the underlying models not only calculate a final value, such as the number of lesions, but also determine their locations within the image. Consequently, the output for these visual signs is accompanied by additional data, which varies depending on whether the quantification involves count or extent.

  • Count. When a visual sign is quantifyed by counting, the device generates bounding boxes for each detected entity. These bounding boxes are defined by their x and y coordinates, as well as their height and width in pixels.
  • Extent. When a visual sign is quantifyed by its extent, the device outputs a mask. This mask, which is the same size as the image, consists of 0's for pixels where the visual sign is absent and 1's for pixels where it is present.

The explainability output can be found with the explainabilityMedia key. Here is an example:

{
  "explainabilityMedia": {
    "explainabilityMedia": {
      "content": "base 64 image",
      "detections": [
        {
          "confidence": 98,
          "label": "nodule",
          "p1": {
            "x": 202,
            "y": 101
          },
          "p2": {
            "x": 252,
            "y": 154
          }
        },
        {
          "confidence": 92,
          "label": "pustule",
          "p1": {
            "x": 130,
            "y": 194
          },
          "p2": {
            "x": 179,
            "y": 245
          }
        }
      ]
    }
  }
}

Risk management​

We conducted a task analysis on the use of the device which led to a comprehensive use-related risk analysis and list of potential use errors. This process involved analyzing known use problems with similar devices and/or previous versions of the device, identifying the user interface characteristics related to safety, identifying potential use errors, and identifying known and foreseeable hazards and hazardous situations.

note

Include a table coming from the Use-related risk analysis with the hazards related to usability.

Summative evaluation testing​

Method​

Instructions

Insert a general description of the study design: timing, methods, description of the tasks to be performed by the testers, version of the software used in the test, version of the IFU.

Participants or testers​

The study was conducted with insert number of participants participants who are representative of the intended users.

The user profile is the following: insert user profile.

Instructions

Insert details about the participants, such as full name, role, healthcare organization, and statement that participants met the user profile described above.

Test environment​

Instructions

Insert a description of the test environment.

Test results​

Data analysis​

Instructions

Explain the methods used to analyze the collected data, including any metrics.

Findings​

Instructions

Summarize the results of the test and their analysis with the established success criteria, and key findings from the summative evaluation test, including both positive outcomes and areas for improvement.

Use errors​

Instructions

List any use errors identified during testing, along with their potential impact and frequency of occurrence.

Information about safety​

Instructions

Describe whether information for safety could correctly be perceived and understood by the users and whether it supported the correct use of the medical device.

User feedback​

Instructions

Provide a summary of qualitative feedback received from test participants, highlighting any common themes or significant insights.

Conclusions​

Instructions

The final discussion about the results should contain the following: whether all tests were successfully passed, whether all risks related to usability are acceptable, and whether additional data about usability shall be collected as part of the post-market surveillance activities.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

  • Author: Team members involved
  • Reviewer: JD-003, JD-004
  • Approver: JD-001
Delete this

Delete this section when you create a new record from this template.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

  • Author: Team members involved
  • Reviewer: JD-003, JD-004
  • Approver: JD-001
Previous
T-012-014 Summative evaluation plan_YYYY_nnn
Next
T-012-016 Software usability test guide
  • Objective
  • Abbreviations and definitions
  • Device description
    • Device Identification
  • Risk management
  • Summative evaluation testing
    • Method
    • Participants or testers
    • Test environment
    • Test results
      • Data analysis
      • Findings
      • Use errors
      • Information about safety
      • User feedback
  • Conclusions
All the information contained in this QMS is confidential. The recipient agrees not to transmit or reproduce the information, neither by himself nor by third parties, through whichever means, without obtaining the prior written permission of Legit.Health (AI LABS GROUP S.L.)