T-TF-025-005 Summative evaluation observation form
ITP Use Scenario 1: Simulated Use
| Usability Task ID | Task | Success Criteria | Outcome | Observations |
|---|---|---|---|---|
| ITP-T-01 | Access and read the Instructions for Use (IFU) | The user demonstrates understanding of: 1) Correct endpoint URLs and their purposes, 2) Required request formats and parameters for each endpoint, 3) Expected response structures, 4) Authentication workflow, 5) Error handling procedures |
| |
| ITP-T-02 | Authenticate using valid credentials | User inputs correct username, password, or other required authentication details |
| |
| ITP-T-03 | Receive and store the JSON response | User stores the JSON response correctly, without loss or corruption, and can retrieve it for further processing |
| |
| ITP-T-04 | Send request to /diagnosis-support endpoint | User formats the data correctly, does not include the wrong JSON structure or incorrect field names |
| |
| ITP-T-05 | Receive and store the JSON response | User stores the JSON response correctly, without loss or corruption, and can retrieve it for further processing |
| |
| ITP-T-06 | Confirm JSON contains expected fields as per IFU | User verifies that all expected fields are present and correctly formatted according to the IFU |
| |
| ITP-T-07 | Verify the version of the device API being integrated | The user correctly identifies: 1) API version from response headers, 2) Version from documentation |
| |
| ITP-T-08 | Handle and log a 400/500 error response from the API. | Integrator handles and logs error correctly according to IFU (e.g., retry or user alert). |
|
HCP Use Scenario 1: Simulated Use: No Lesion
| Usability Task ID | Task | Success Criteria | Outcome | Observations |
|---|---|---|---|---|
| HCP-T-01 | Access and read the Instructions for Use (IFU): Clinical User Manual Review | The user demonstrates understanding of How to take the pictures and proper system usage as described in the manual |
| |
| HCP-T-02 | User takes a photo of the patient's lesion. | The photo has good quality, captures the relevant skin structure, is well lit, focused, with adequate distance |
| |
| HCP-T-03 | User logs into the system | The user enters the correct username and password and logs in successfully |
| |
| HCP-T-04 | User uploads photos to the Legit.Health Plus client. | The user uploads a high-quality image |
| |
| HCP-T-05 | User can read the device's output. | The user can read the devices output |
| |
| HCP-T-06 | User correctly identifies the probability of certain conditions based on the ICD categories in the report. | The user identifies the conditions and probability listed in the report |
| |
| HCP-T-07 | User correctly interprets the report output. | The user correctly identifies the quantification of disease intensity, extent, or count of clinical signs. |
| |
| HCP-T-08 | The user understands that the report is not a certain diagnosis. | The user acknowledges that the output report is not a standalone diagnosis. |
| |
| HCP-T-09 | After use the device, answer the questions of Knowledge Assessment. | The user answers correctly at least 80% of the questions in the Knowledge Assessment. |
| |
| HCP-T-10 | User reacts appropriately to an error message (e.g., image upload failed). | User follows UI or IFU instructions to retry or report the issue as described. |
|
HCP Use Scenario 2: Simulated Use: Lesion
| Usability Task ID | Task | Success Criteria | Outcome | Observations |
|---|---|---|---|---|
| HCP-T-01 | Access and read the Instructions for Use (IFU): Clinical User Manual Review | The user demonstrates understanding of How to take the pictures and proper system usage as described in the manual |
| |
| HCP-T-02 | User takes a photo of the patient's lesion. | The photo has good quality, captures the relevant skin structure, is well lit, focused, with adequate distance |
| |
| HCP-T-03 | User logs into the system | The user enters the correct username and password and logs in successfully |
| |
| HCP-T-04 | User uploads photos to the Legit.Health Plus client. | The user uploads a high-quality image |
| |
| HCP-T-05 | User can read the device's output. | The user can read the devices output |
| |
| HCP-T-06 | User correctly identifies the probability of certain conditions based on the ICD categories in the report. | The user identifies the conditions and probability listed in the report |
| |
| HCP-T-07 | User correctly interprets the report output. | The user correctly identifies the quantification of disease intensity, extent, or count of clinical signs. |
| |
| HCP-T-08 | The user understands that the report is not a certain diagnosis. | The user acknowledges that the output report is not a standalone diagnosis. |
| |
| HCP-T-09 | After use the device, answer the questions of Knowledge Assessment. | The user answers correctly at least 80% of the questions in the Knowledge Assessment. |
| |
| HCP-T-10 | User reacts appropriately to an error message (e.g., image upload failed). | User follows UI or IFU instructions to retry or report the issue as described. |
|