Skip to main content
QMSQMS
QMS
  • Welcome to your QMS
  • Quality Manual
  • Procedures
    • GP-001 Control of documents
    • GP-002 Quality planning
    • GP-003 Audits
    • GP-004 Vigilance system
    • GP-005 Human Resources and Training
    • GP-006 Non-conformity, Corrective and Preventive actions
    • GP-007 Post-market surveillance
    • GP-008 Product requirements
    • GP-009 Sales
    • GP-010 Purchases and suppliers evaluation
    • GP-011 Provision of service
    • GP-012 Design, Redesign and Development
    • GP-013 Risk management
    • GP-014 Feedback and complaints
    • GP-015 Clinical evaluation
      • Templates
        • T-015-001 Clinical Evaluation Plan_YYYY_nnn
        • T-015-002 Preclinical and clinical evaluation record_YYYY_nnn
        • T-015-003 Clinical Evaluation Report_YYYY_nnn
        • T-015-004 Clinical Investigation Plan_Study code
        • T-015-005 Investigator's Brochure_Study code_es
        • T-015-006 Clinical investigation report_Study code_en
        • T-015-006 Clinical investigation report_Study code_es
        • T-015-007 Delaration of interest Clinical evaluation team
        • T-015-008 Clinical development plan_YYYY_nnn
        • T-015-009 Clinical Investigation Plan_Study code_en
    • GP-016 Traceability and identification
    • GP-017 Technical assistance service
    • GP-018 Infrastructure and facilities
    • GP-019 Software validation plan
    • GP-020 QMS Data analysis
    • GP-021 Communications
    • GP-022 Document translation
    • GP-023 Change control management
    • GP-024 Cybersecurity
    • GP-025 Corporate Governance
    • GP-026 Product requirements for US market
    • GP-027 Product requirements for UK market
    • GP-028 Product requirements for the Brazilian market
    • GP-050 Data Protection
    • GP-051 Security violations
    • GP-052 Data Privacy Impact Assessment (DPIA)
    • GP-100 Business Continuity (BCP) and Disaster Recovery plans (DRP)
    • GP-101 Information security
    • GP-200 Remote Data Acquisition in Clinical Investigations
  • Records
  • TF_Legit.Health_Plus
  • Licenses and accreditations
  • External documentation
  • Procedures
  • GP-015 Clinical evaluation
  • Templates
  • T-015-002 Preclinical and clinical evaluation record_YYYY_nnn

T-015-002 Preclinical and clinical evaluation record_YYYY_nnn

Scope​

Medical device information​

Manufacturer contact details​

Medical device characterization​

Intended purpose​

Variants and models​

Expected lifetime​

List of any accessories​

Explanation of any novel features​

Literature search methodology​

The literature search methodology establishes the strategy used to search and compile scientific articles, as well as the criteria used to include/exclude and evaluate the doctuments compiled.

Review team undertaking the literature search​

The literature search protocol has been developed and executed by professionals with expertise in information retrieval and understanding of the scope of the clinical evaluation set out by the manufacturer, who have been compensated for the time taken to perform the review.

The involvement of the literature review team will help to optimize literature retrieval to identify all relevant published literature. The review team has prepared the protocol to describe an objective, non-biased systematic search and review methods, using information found in the IFU and risk management process to develop the clinical research questions.

Period covered by the search​

Goals of the literature search​

This literature search was conducted to include the relevant scientific articles that are related to:

info

Examples

  • Clinical trials that probe the safety and usefulness of Computer-Aided Systems for obtaining...
  • State-of-the-art algorithms and methods in the fields of ..., to obtain an ...
  • Datasets that are useful for the development of technology to obtain an ...., in .....
  • Smartphone applications that implement or evaluate ... to obtain an .....
  • Reviews and surveys of the available literature related to obtain an ...., in ....

Data sources​

For the scientific article compilation we have used different sources of information:

Examples

Medical Literature Databases​

  • PubMed (MEDLINE): comprises more than 36 million citations for biomedical literature from MEDLINE, life science journals, and online books.
  • Cochrane Library: collection of databases that contain high-quality, independent evidence to inform healthcare decision-making. The Cochrane Library is owned by Cochrane and published by Wiley.

Google Scholar​

Google Scholar provides a simple way to broadly search for scholarly literature across a wide variety of disciplines and sources: articles, theses, books, abstracts and court opinions. Different from the Medical Literature Databases, Google Scholar also includes medical articles that have been published in non-medical journals and conferences and that might not have been indexed by the previous Medical Literature Databases.

Article Citations​

Scientific articles are also a meaningful source of information when describing their related work. Relevant articles that are cited within a given published manuscript are also considered for their evaluation.

The PICO format​

The Literature Search methodology is based on the PICO format.

This format has been designed for high-quality clinical research evidence by defining a set of relevant keywords in four different categories (Population, Intervention, Comparison, and Outcome), that are then used to construct the search queries.

To include a description or clarification

Example: Since our research is not limited to clinical research, by also to articles related to state-of-the-art Deep Learning algorithms, paper reviews, and the deployment of Deep Learning solutions, we have modified these categories to also include keywords related to non-clinical terms.

CategoryDescription
Population Patient Problem (P)Diseases that the software is able to identify (skin cancer, melanoma, chronic skin conditions).
Intervention (I)Intervention, prognostic factor or exposure.
Comparator (C)Known software, algorithm, or method with the same characteristics as the device.
Outcome (O)Diagnosis, identification, analysis, or assessment of certain skin pathologies.

Screening, selection, and organizing literature​

The articles retrieved from each data source go through an initial set of filters to ensure their quality and adequacy to the goals of this literature review:

Metadata check​

Duplicate articles, non-English articles, and articles published before YYYY are not considered for the Data Appraisal.

Scope analysis​

We read the abstract, motivation, and conclusions of each article to discern if they match the scope of topics pursued in the Literature Review. Articles that are our of the scope of this literature review are filtered out. After this initial look, we save a brief summary of the content and contribution of the articles.

Publication source quality analysis​

To update with the analysis performed

For each article, we analyze the impact factor of the journal, conference, or book where they have been published. The impact factor is a metric that reflects the ratio between the number of citations received in a year and the number of citable publications that were published in the same source during the two preceding years. In our case, we retrieve the Impact Factor from Academic Accelerator that has been calculated for the year YYYY. Finally, we filter out the articles that have an Impact Factor under X, which stands for a low-quality publication source. Since the Impact Factor measures the quality of the publication source and not the quality of the publication itself, we also included for the Data Appraisal the papers that we considered relevant during the Scope Analysis step.

The articles that have passed the screening quality evaluation are then forwarded to their appraisal.

Appraisal of selected literature​

The Data Appraisal evaluates the suitability of the selected articles for establishing the safety and performance of the device. To this end, we take into consideration three parameters: the methodological quality and scientific validity of the information (Q), the relevance to the device and its intended use (R), and the contribution to the overall evaluation (C). Articles are finally accepted or rejected after applying the following weight-based system. In this process, we also compile some notes related to the content and contribution of the articles.

Data characteristicsMethodological quality QRelevance, RContribution, CWeighted value, W (W=Q+R+C)Appraisal dataNotes
Very relevant information in relation to the product and its intended useUp to 40Up to 30Up to 30W ≥ 70AcceptedPivotal data
Relevant information in relation to the product and its intended useUp to 40Up to 30Up to 3030 < W < 70AcceptedOther data
Little relevant information in relation to the product and its intended useUp to 40Up to 30Up to 30W ≤ 30RejectedNo contribution, rejected

Methodological quality and scientific validity of the information (Q):

  • The scientific article properly describes the relevant related work.
  • The scientific article compares its results and conclusions with the relevant related work.
  • Clinical trials use data extracted from RCTs (Randomized Clinical Trials), meta-analysis and clinical trials, from publications based on scientific evidence or from previous experience on the market.
  • The study design is adequate in terms of type, sample size, relevance of endpoints, randomization of patients, inclusion and exclusion criteria, stratification of patients, prognostic factors, follow-up, recording and reporting of serious adverse events, reliability of the methods used for quantifying outcomes, or procedures for retrieving complete information.
  • Data consists of an adequate number of observations that can confirm or refute the claims of the device under evaluation.
  • The methods for data processing and statistics are suitable.
  • It complies with Good Clinical Practice (GCP) according to EN ISO 14155 or equivalent standards.
  • The information is disclosed adequately, including a summary, introduction, methods, results, discussion and conclusions.

Relevance to the device and its intended use (R):

  • It is representative of the device under evaluation, including data concerning the medical conditions that are managed with the device, other devices and medical alternatives, or equivalent devices.
  • The aspects covered are related to clinical safety and performance, claims, hazards and hazardous situations, management of risks, establishment of current knowledge and state of the art, determination and justification of the benefit/risk ratio and the acceptability of undesirable side-effects, or the determination of equivalence.
  • It is representative to the intended purpose or claims of the device under evaluation.
  • It is representative to the user group, patient population, medical indication, age group, gender, type and severity of the medical condition, or range of time.
  • It develops algorithms or techniques to improve the performance of skin-related image analysis.

Contribution to the overall evaluation (C):

  • Device and/or related techniques intended use. The contribution of the analyzed data in relation to the device or technique features and in relation to the intended use must be strong and clear.
  • Device and/or related techniques risk analysis. The contribution of the analyzed data in relation to the device or techniques of risk analysis must be strong and clear, based on keywords definition.

Literature Review results​

Screening​

For each one of the PICO components we have defined the set of keywords as:

Keywords
Population, Patient, Problem (P)
Intervention (I)
Comparator (C)
Outcome (O)

In the following, we describe how we use these keywords to form queries for each source of information, the retrieved articles, and the outcome of the Data Screening.

PubMed​

Following the PICO format and using the PubMed advanced search tool, we have defined a global query to retrieve the articles whose title or abstract relate to the query:

# 1 AND # 2 AND # 3 AND # 4

Example
(("dermatosis") OR ("skin cancer") OR ("chronic skin conditions") OR ("inflammatory skin diseases") OR ("malignant skin lesions") OR ("pigmented skin lesions") OR ("melanoma") OR ("basal cell carcinoma") OR ("squamous cell carcinoma") OR ("atypical nevus") OR ("acne") OR ("psoriasis") OR ("urticaria") OR ("atopic dermatitis") OR ("onychomycosis") OR ("melasma") OR ("solar lentigo") OR ("dermatofibroma") OR ("skin diseases") OR ("skin lesions")) AND (("clinical image") OR ("digital imaging") OR ("web application") OR ("smartphone") OR ("dermatoscopy") OR ("camera") OR ("CAD")) AND (("artificial intelligence") OR ("machine learning") OR ("deep learning") OR ("computer vision") OR ("deep neural networks") OR ("convolutional neural networks") OR ("metaoptima") OR ("automated")) AND (("diagnosis") OR ("diagnosis support") OR ("followup") OR ("segmentation") OR ("detection") OR ("estimation") OR ("classification") OR ("assessment") OR ("severity assessment") OR ("improving"))

As a result, we have compiled nnn articles of which nnn articles have passed the Data Screening:

The editable table is here: To include the URL
IDNameJournal/ConferenceImpact factorIn scope? Not duplicated?Appraisal included
nnn

Cochrane​

Following the PICO format and using the Cochrane advanced search tool, we have defined a global query to retrieve the articles whose title, abstract, or keywords relate to the query:

Example
("dermatosis" OR "skin cancer" OR "chronic skin conditions" OR "inflammatory skin diseases" OR "malignant skin lesions" OR "pigmented skin lesions" OR "melanoma" OR "basal cell carcinoma" OR "squamous cell carcinoma" OR "atypical nevus" OR "acne" OR "psoriasis" OR "urticaria" OR "atopic dermatitis" OR "onychomycosis" OR "melasma" OR "solar lentigo" OR "dermatofibroma" OR "skin diseases" OR "skin lesions")  AND ("clinical image" OR "digital imaging" OR "web application" OR "smartphone" OR "dermatoscopy" OR "camera" OR "CAD")  AND ("artificial intelligence" OR "machine learning" OR "deep learning" OR "computer vision" OR "deep neural networks" OR "convolutional neural networks" OR "metaoptima" OR "automated") AND ("diagnosis" OR "diagnosis support" OR "followup" OR "segmentation" OR "detection" OR "estimation" OR "classification" OR "assessment" OR "severity assessment" OR "improving")

As a result, we have compiled nnn articles of which nn articles have passed the Data Screening:

The editable table is here: To include the URL
IDNameJournal/ConferenceImpact factorIn scope? Not duplicated?Appraisal included
nnn

Google Scholar​

Since the Google Scholar search tool does not allow as much detail as PubMed and Cochrane, we have defined X different queries to retrieve the articles. From the retrieved list of articles, sorted by relevance, we compile the first 20 results (2 pages):

Query X:

As a result, nnn articles have passed the Data Screening:

IDNameJournal/ConferenceImpact factorIn scope? Not duplicated?Appraisal included
nnn

Cited articles​

nn articles, cited by the article with ID nn, have passed the Data Screening:

IDNameJournal/ConferenceImpact factorIn scope? Not duplicated?Appraisal included
nnn

Data appraisal​

All the articles that passed the Data Screening have been reviewed thoroughly to assess their Quality, Relevance, and Contribution. As a result:

  • XX articles have been evaluated as highly relevant.
  • XX articles have been evaluated as relevant.
  • XX articles have been evaluated as non-relevant.
The editable table is here: To include the URL
IDNameJournal/ConferenceImpact factorIn scope? Not duplicated?Appraisal included
nnn

Responsible evaluators​

The qualifications and experience of the responsible evaluators are provided in the T-015-003 Clinical Evaluation Report (CER) to demonstrate that the responsible persons fulfill the requirements to evaluate the product clinical evaluation.

NamePositionCompanyRole in the evaluationDeclaration of interestCV

Record signature meaning​

  • Author: JD-009 Author name
  • Review: JD-003 or JD-004 Reviewer name
  • Approval: JD-005 Approver name

Template signature meaning​

info

Delete this section when you create a new record from this template.

  • Author: JD-004 María Diez
  • Review: JD-003 Taig Mac Carthy
  • Approval: JD-001 Andy Aguilar
Previous
T-015-001 Clinical Evaluation Plan_YYYY_nnn
Next
T-015-003 Clinical Evaluation Report_YYYY_nnn
  • Scope
  • Medical device information
    • Manufacturer contact details
  • Medical device characterization
  • Intended purpose
    • Variants and models
    • Expected lifetime
    • List of any accessories
    • Explanation of any novel features
  • Literature search methodology
    • Review team undertaking the literature search
    • Period covered by the search
    • Goals of the literature search
    • Data sources
      • Medical Literature Databases
      • Google Scholar
      • Article Citations
    • The PICO format
    • Screening, selection, and organizing literature
      • Metadata check
      • Scope analysis
      • Publication source quality analysis
    • Appraisal of selected literature
  • Literature Review results
    • Screening
      • PubMed
      • Cochrane
      • Google Scholar
      • Cited articles
    • Data appraisal
  • Responsible evaluators
  • Record signature meaning
  • Template signature meaning
All the information contained in this QMS is confidential. The recipient agrees not to transmit or reproduce the information, neither by himself nor by third parties, through whichever means, without obtaining the prior written permission of Legit.Health (AI LABS GROUP S.L.)