R-TF-028-004 Data Annotation Instructions

Table of contents

• Context
• Objectives
• Annotation Personnel
  • Role
  • Qualifications
  • Responsibilities
• General Annotation Workflow
• Specific Annotation Instructions by Task Type
  • Task: Intensity Annotation (Ordinal)
  • Task: Skin Tone Identification
  • Task: Extent Annotation (Polygons)
  • Task: Body Site Identification (Categorical)
• Quality Control

Context

This document provides instructions for annotating images for non-clinical data. The annotation process focuses exclusively on non-clinical attributes, such as visual image quality scoring, image type, skin tone and body site identification, and drawing polygons for affected areas. These tasks are essential for training, validating, and testing non-clinical supervised learning models that support the device's performance.

Objectives

• To create a comprehensive, multi-modal ground-truth dataset for the quantification and description of non-clinical attributes of dermatological images.
• To provide clear, unambiguous instructions to data annotation professionals to ensure consistency and accuracy across all annotation tasks.
• To establish a robust foundation for training, validating, and testing AI/ML models responsible for non-clinical data analysis (e.g., image quality, skin tone, body site, and extent annotations).

Annotation Personnel

Role

Data annotation professional.

Qualifications

• Required: Normal visual acuity and average computer skills.
• Experience: A minimum of three years of experience in data annotation services.

Responsibilities

• To review and understand these annotation instructions thoroughly.
• To perform annotations on the provided images using the designated annotation platform.
• To complete each annotation task accurately and consistently according to the specific guidelines below.

General Annotation Workflow

All annotations will be performed using a web-based medical image annotation platform. For each image assigned, the annotator will:

1. Examine the image to identify all relevant non-clinical features related to the task at hand.
2. Perform the annotation as per the specific instructions for that task type (visual quality assessment, image domain, skin tone and body site classification, and extent annotation using polygons).
3. Repeat the process until all non-clinical labels have been annotated.

Specific Annotation Instructions by Task Type

This section details the distinct annotation methodologies for each non-clinical annotation task. No clinical visual sign annotation is performed as part of this process.

Task: Intensity Annotation (Ordinal)

This task is for signs where severity is assessed on a scale.

• Ordinal Scale (e.g., 0-4): Acneiform Inflammatory Pattern Identification.
• Instruction: For each sign present in the image, select the numerical value or descriptive category from the provided list that best represents its intensity or type.
• Guidelines & Example: The platform will present a scale. Detailed clinical scoring guides based on established literature will be provided as a reference.

Task: Skin Tone Identification

This task is for assigning two labels to an entire image based on the pictured Fitzpatrick skin tone (FST) and Monk skin tone (MST) scales.

• Instruction (FST): Observe the skin content of the image and label the skin tone as one of the following: I, II, III, IV, V, VI.
• Instruction (Monk): Observe the skin content of the image and label the skin tone as one of the categories in the Monk scale (1 to 10)

Task: Extent Annotation (Polygons)

This task is to annotate the boundaries that cover a surface area.

• Instruction: Using the polygon tool, carefully trace the complete boundary of the area affected by the specified sign.
• Guidelines:
  • The outline should be as precise as possible, closely following the visible edge of the sign.
  • Place polygon points close enough together to accurately capture curves.
  • For conditions with multiple, non-contiguous patches, draw a separate polygon for each distinct area.

Task: Body Site Identification (Categorical)

• Instruction: Inspect the image and identify which body parts are visible, and then label the image as one or more of the following categories:
  • Scalp
  • Top of the head
  • Back of the head
  • Face
  • Mouth
  • Tongue
  • Ear
  • Eye
  • Nose
  • Neck
  • Trunk, chest and abdomen
  • Back
  • Armpit
  • Arm
  • Hand
  • Finger
  • Hand nail
  • Leg
  • Knee
  • Foot
  • Toe
  • Foot nail
  • Genitals (groin, penis, vulva, anus)
  • Buttock
  • Close-up image (body parts not visible)

Quality Control

To ensure the creation of a high-fidelity ground truth and to measure inter-observer variability, a rigorous quality control process will be followed.

• Multi-Annotator Process: To ensure the highest quality, images will be annotated by one or more qualified annotators. In the case of image quality assessment (IQA), at least 15 observers will be used for each annotation project, following the ITU-T P.910 recommendation from the International Telecommunications Union (ITU).
• Consensus Ground Truth: When multiple experts annotate the same image, the final ground-truth label will be established by pooling their independent assessments. The method for establishing consensus will depend on the task (e.g., mean for image quality scores, voting for categories, or algorithmic fusion for bounding boxes/polygons).

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003 Design & Development Manager, JD-004 Quality Manager & PRRC
• Approver: JD-001 General Manager