Skip to main content
QMSQMS
QMS
  • Welcome to your QMS
  • Quality Manual
  • Procedures
    • GP-001 Control of documents
    • GP-002 Quality planning
    • GP-003 Audits
    • GP-004 Vigilance system
    • GP-005 Human Resources and Training
    • GP-006 Non-conformity, Corrective and Preventive actions
    • GP-007 Post-market surveillance
    • GP-008 Product requirements
    • GP-009 Sales
    • GP-010 Purchases and suppliers evaluation
    • GP-011 Provision of service
    • GP-012 Design, redesign and development
    • GP-013 Risk management
    • GP-014 Feedback and complaints
    • GP-015 Clinical evaluation
    • GP-016 Traceability and identification
    • GP-017 Technical assistance service
    • GP-018 Infrastructure and facilities
    • GP-019 Software validation plan
    • GP-020 QMS Data analysis
    • GP-021 Communications
    • GP-022 Document translation
    • GP-023 Change control management
    • GP-024 Predetermined Change Control Plan
    • GP-025 Usability and Human Factors Engineering
    • GP-027 Corporate Governance
    • GP-028 AI Development
      • Specific procedures
        • T-TF-028-001 AI Description
        • T-TF-028-002 AI Development Plan
        • T-TF-028-003 Data Collection Instructions
        • T-TF-028-004 Data Annotation Instructions
        • T-TF-028-005 AI Development Report
        • T-TF-028-006 AI Release Report
        • T-TF-028-007 AI Retraining Report
        • T-TF-028-008 AI Reevaluation Report
        • T-TF-028-009 AI Design Checks
        • T-TF-028-010 AI V&V Checks
        • T-TF-028-011 AI Risk Assessment
    • GP-029 Software Delivery and Commissioning
    • GP-030 Cyber Security Management
    • GP-050 Data Protection
    • GP-051 Security violations
    • GP-052 Data Privacy Impact Assessment (DPIA)
    • GP-100 Business Continuity (BCP) and Disaster Recovery plans (DRP)
    • GP-101 Information security
    • GP-200 Remote Data Acquisition in Clinical Investigations
    • GP-026 Market-specific product requirements
    • GP-110 Esquema Nacional de Seguridad
  • Records
  • Legit.Health Plus Version 1.1.0.0
  • Legit.Health Plus Version 1.1.0.1
  • Legit.Health Utilities
  • Licenses and accreditations
  • Applicable Standards and Regulations
  • Pricing
  • Public tenders
  • Procedures
  • GP-028 AI Development
  • Specific procedures
  • T-TF-028-004 Data Annotation Instructions

T-TF-028-004 Data Annotation Instructions

Table of contents
  • Context
  • Objectives
  • Annotation Personnel
    • Primary Annotation Role
      • Qualifications
      • Responsibilities
    • Supporting/Clinical Role (if applicable)
      • Qualifications
      • Responsibilities
  • Annotation Protocol
    • Data Source Processing
    • Annotation Workflow
    • Annotation Guidelines
      • General Guidelines
      • Specific Guidelines by Task Type
    • Handling Ambiguous Cases
  • Annotation Tools
  • Quality Control
    • Multi-Annotator Process
    • Consensus Ground Truth
    • Senior Review
    • Quality Metrics
  • Validation and Finalization
  • Related Documents

Context​

Describe the context for this annotation task. Explain what data is being annotated, why the annotation is needed, and how it relates to the AI development process.

Objectives​

List the objectives of this annotation procedure.

  • Objective 1
  • Objective 2

Annotation Personnel​

Primary Annotation Role​

Qualifications​

  • Required: Qualification 1
  • Recommended: Qualification 2
  • Required Knowledge: Knowledge requirement

Responsibilities​

List the responsibilities of the primary annotator.

  • Responsibility 1
  • Responsibility 2

Supporting/Clinical Role (if applicable)​

Qualifications​

  • Required: Qualification 1
  • Recommended: Qualification 2

Responsibilities​

List the responsibilities of the supporting role.

  • Responsibility 1
  • Responsibility 2

Annotation Protocol​

Describe the annotation protocol in detail.

Data Source Processing​

Describe how source data is processed before annotation.

  1. Step 1
  2. Step 2

Annotation Workflow​

Describe the annotation workflow.

  1. Step 1
  2. Step 2

Annotation Guidelines​

Provide detailed guidelines for annotators.

General Guidelines​

List general annotation guidelines.

  • Guideline 1
  • Guideline 2

Specific Guidelines by Task Type​

Provide specific guidelines for different annotation tasks.

Task Type 1: [Name]​
  • Signs/Items to Annotate: List items
  • Instruction: Describe the annotation instruction
  • Guidelines: Provide specific guidelines
Score/CategoryDescription
Task Type 2: [Name]​
  • Signs/Items to Annotate: List items
  • Instruction: Describe the annotation instruction
  • Guidelines: Provide specific guidelines

Handling Ambiguous Cases​

Describe how to handle ambiguous or complex cases.

Annotation Tools​

Describe the annotation tools to be used.

ToolPurpose

Quality Control​

Multi-Annotator Process​

Describe the multi-annotator process (if applicable).

Consensus Ground Truth​

Describe how consensus is established.

Senior Review​

Describe the senior review process for disagreements.

Quality Metrics​

Define quality metrics to be tracked.

MetricTarget

Validation and Finalization​

Describe the validation and finalization process.

  1. Step 1
  2. Step 2

Related Documents​

Document IDTitle
R-TF-028-001
R-TF-028-002
R-TF-028-003

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

  • Author: JD-009
  • Reviewer: JD-009
  • Approver: JD-005
Previous
T-TF-028-003 Data Collection Instructions
Next
T-TF-028-005 AI Development Report
  • Context
  • Objectives
  • Annotation Personnel
    • Primary Annotation Role
      • Qualifications
      • Responsibilities
    • Supporting/Clinical Role (if applicable)
      • Qualifications
      • Responsibilities
  • Annotation Protocol
    • Data Source Processing
    • Annotation Workflow
    • Annotation Guidelines
      • General Guidelines
      • Specific Guidelines by Task Type
        • Task Type 1: [Name]
        • Task Type 2: [Name]
    • Handling Ambiguous Cases
  • Annotation Tools
  • Quality Control
    • Multi-Annotator Process
    • Consensus Ground Truth
    • Senior Review
    • Quality Metrics
  • Validation and Finalization
  • Related Documents
All the information contained in this QMS is confidential. The recipient agrees not to transmit or reproduce the information, neither by himself nor by third parties, through whichever means, without obtaining the prior written permission of Legit.Health (AI Labs Group S.L.)