TEST_016 Ensure API compliance with Base64 image format and FHIR standard
Test type​
System
Linked activities​
- MDS-449
Result​
- Passed
- Failed
Description​
This test run record documents the results of executing test plans designed to verify two main areas: confirming that the API only accepts images in Base64 format and validating that the data schemas of the request and response content adhere to the FHIR (Fast Healthcare Interoperability Resources) standard. The record includes detailed outcomes of the tests, highlighting any deviations from expected behavior, along with corresponding error messages and potential impacts on system functionality.
These test cases are being conducted for the first time on this version of the medical device.
Run environment​
Here are the technical specifications of the runtime environment in which the tests were conducted:
- Operating system: Ubuntu 22.04.3 LTS (Linux kernel version: 5.10.0-051000-generic)
- Hardware specifications:
- CPU:
- Model name: AMD Ryzen Threadripper PRO 5995WX (x86_64)
- Number of cores: 64
- Thread(s) per core: 2
- GPU:
- CUDA version: 12.0
- Devices:
- NVIDIA RTX 6000 (49140 MiB)
- NVIDIA RTX 6000 (49140 MiB)
- RAM: 252 GB
- Storage: 6.5 TB
- Network:
- Mean speed: 380 Mbps
- Mean latency: 5 ms
- CPU:
- Other relevant software: No particular software was used.
Test case runs​
The following test cases have been executed in this batch:
TEST_016_001​
Outcome​
- Passed
- Failed
Expected results​
- The request with the well-formed payload is accepted, and the response is validated against the FHIR schema without errors. The response aims to closely follow the structure of the FHIR
DiagnosticReportresource. While we adapt this resource to fit our specific use case, we ensure that the mandatory and most common keys are implemented. - The API rejects the invalid JSON payload and responds with a
422 Unprocessable Contentstatus code. The error message informs you that the request body is incorrectly formatted, and no report is returned.
Actual results​
- The request with the well-formed payload is accepted, and the response is validated against the FHIR schema without errors. The response aims to closely follow the structure of the FHIR
DiagnosticReportresource. While we adapt this resource to fit our specific use case, we ensure that the mandatory and most common keys are implemented:
Status code: 200
{
"resourceType": "DiagnosticReport",
"status": "preliminary",
"category": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0074",
"code": "IMG",
"display": "Diagnostic Imaging"
}
],
"text": "Diagnostic Imaging"
},
"code": {
"coding": [
{
"system": "http://loinc.org/",
"code": "100063-7",
"display": "Primary skin concern"
},
{
"system": "http://loinc.org/",
"code": "10206-1",
"display": "Physical findings of Skin Narrative"
}
],
"text": "Skin Condition Analysis"
},
"device": {
"name": "Legit.Health Plus",
"manufacturer": "AI LABS GROUP SL",
"category": {
"system": "http://hl7.org/fhir/device-category",
"code": "software",
"display": "Software"
},
"type": {
"system": "https://www.snomed.org/",
"code": "468878001",
"display": "Dermatology picture archiving and communication system application software"
},
"url": "https://medical-device.legit.health/"
},
"effectiveDateTime": "2024-07-05T09:27:53.142154",
"conclusions": [
{
"coding": {
"code": "XH5AW4",
"display": "Haemangioma",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 20.61
},
{
"coding": {
"code": "XH4L78",
"display": "Nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 8.72
},
{
"coding": {
"code": "2F20.0Y, XH27A6",
"display": "Compound nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 5.12
},
{
"coding": {
"code": "XH4846",
"display": "Malignant melanoma",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 4.78
},
{
"coding": {
"code": "XH2HG8",
"display": "Spitz nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 4.76
}
],
"metrics": {
"type": {
"coding": [
{
"system": "urn:iso:std:iso:11073:10101",
"code": "2",
"display": "Metrics"
}
],
"text": "Diagnosis Metrics"
},
"category": "calculation",
"sensitivity": 0.91,
"specificity": 0.98
},
"preliminaryFindings": {
"hasConditionSuspicion": 0.99,
"needsSpecialistAttention": 1.0,
"isMalignantSuspicion": 0.05,
"isPreMalignantSuspicion": 0.05
},
"imagingStudySeries": [
{
"media": {
"validity": {
"metrics": {
"quality": {
"valid": true,
"score": 60,
"category": "Acceptable"
},
"domain": {
"valid": true,
"score": 100,
"category": "Dermatological"
}
},
"isValid": true
},
"modality": "Clinical"
},
"metrics": {
"type": {
"coding": [
{
"system": "urn:iso:std:iso:11073:10101",
"code": "2",
"display": "Metrics"
}
],
"text": "Diagnosis Metrics"
},
"category": "calculation",
"sensitivity": 0.91,
"specificity": 0.98
},
"conclusions": [
{
"coding": {
"code": "XH5AW4",
"display": "Haemangioma",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 20.61
},
{
"coding": {
"code": "XH4L78",
"display": "Nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 8.72
},
{
"coding": {
"code": "2F20.0Y, XH27A6",
"display": "Compound nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 5.12
},
{
"coding": {
"code": "XH4846",
"display": "Malignant melanoma",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 4.78
},
{
"coding": {
"code": "XH2HG8",
"display": "Spitz nevus",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 4.76
}
],
"preliminaryFindings": {
"hasConditionSuspicion": 0.99,
"needsSpecialistAttention": 1.0,
"isMalignantSuspicion": 0.05,
"isPreMalignantSuspicion": 0.05
}
}
],
"analysisDuration": "0.6285 secs"
}
-
The response JSON from the API closely follows the structure of the
DiagnosticReportresource as defined by the FHIR standard, sharing keys likestatus,category,code,effectiveDateTime, andconclusion. -
The API rejects the invalid JSON payload and responds with a
422 Unprocessable Contentstatus code. The error message informs you that the request body is incorrectly formatted, and no report is returned:
Status code: 422
{
"detail": [
{
"type": "missing",
"loc": [
"body",
"media"
],
"msg": "Field required",
"input": {
"user": {
"reference": [
"fake-patient-id"
]
},
"images": [
{
"contentType": "image/jpeg",
"data": "/9j/4AAQSkZJRgABAQAAAQABAAD//gAIB..."
}
]
}
},
{
"type": "extra_forbidden",
"loc": [
"body",
"user"
],
"msg": "Extra inputs are not permitted",
"input": {
"reference": [
"fake-patient-id"
]
}
},
{
"type": "extra_forbidden",
"loc": [
"body",
"images"
],
"msg": "Extra inputs are not permitted",
"input": [
{
"contentType": "image/jpeg",
"data": "/9j/4AAQSkZJRgABAQAAAQABAAD//gAIB..."
}
]
}
]
}
- The API response contains only the error message displayed above. It clearly states that the required
mediafield is missing and that the fieldsuserandimagesare unrecognized.
Remarks​
No comments to add. The test was carried out manually without any issues.
TEST_016_002​
Outcome​
- Passed
- Failed
Expected results​
- Both endpoints successfully handle their respective requests containing the Base64 image and generate a well-structured preliminary report.
- The response from the
/severity-assessmentendpoint includes any image data in Base64 format. If everything worked correctly, you can find the image results in the response at the JSON path:patientEvolution.ascorad.media.attachments[0].data.
Actual results​
-
Both endpoints successfully handle their respective requests containing the Base64 image and generate a well-structured preliminary report:
-
Preliminary report from diagnosis support service
Status code: 200
{
"resourceType": "DiagnosticReport",
"status": "preliminary",
"category": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0074",
"code": "IMG",
"display": "Diagnostic Imaging"
}
],
"text": "Diagnostic Imaging"
},
"code": {
"coding": [
{
"system": "http://loinc.org/",
"code": "100063-7",
"display": "Primary skin concern"
},
{
"system": "http://loinc.org/",
"code": "10206-1",
"display": "Physical findings of Skin Narrative"
}
],
"text": "Skin Condition Analysis"
},
"device": {
"name": "Legit.Health Plus",
"manufacturer": "AI LABS GROUP SL",
"category": {
"system": "http://hl7.org/fhir/device-category",
"code": "software",
"display": "Software"
},
"type": {
"system": "https://www.snomed.org/",
"code": "468878001",
"display": "Dermatology picture archiving and communication system application software"
},
"url": "https://medical-device.legit.health/"
},
"effectiveDateTime": "2024-07-05T10:45:23.801254",
"conclusions": [
{
"coding": {
"code": "EA80",
"display": "Atopic dermatitis",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 65.39
},
{
"coding": {
"code": "6B25.1",
"display": "Excoriation disorder",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 3.22
},
{
"coding": {
"code": "EB05",
"display": "Urticaria",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 3.07
},
{
"coding": {
"code": "EA80",
"display": "Eczema",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 2.79
},
{
"coding": {
"code": "1E91.Z",
"display": "Herpes zoster",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 2.49
}
],
"metrics": {
"type": {
"coding": [
{
"system": "urn:iso:std:iso:11073:10101",
"code": "2",
"display": "Metrics"
}
],
"text": "Diagnosis Metrics"
},
"category": "calculation",
"sensitivity": 0.82,
"specificity": 0.98
},
"preliminaryFindings": {
"hasConditionSuspicion": 1.0,
"needsSpecialistAttention": 1.0,
"isMalignantSuspicion": 0.0,
"isPreMalignantSuspicion": 0.0
},
"imagingStudySeries": [
{
"media": {
"validity": {
"metrics": {
"quality": {
"valid": true,
"score": 87,
"category": "Acceptable"
},
"domain": {
"valid": true,
"score": 100,
"category": "Dermatological"
}
},
"isValid": true
},
"modality": "Clinical"
},
"metrics": {
"type": {
"coding": [
{
"system": "urn:iso:std:iso:11073:10101",
"code": "2",
"display": "Metrics"
}
],
"text": "Diagnosis Metrics"
},
"category": "calculation",
"sensitivity": 0.82,
"specificity": 0.98
},
"conclusions": [
{
"coding": {
"code": "EA80",
"display": "Atopic dermatitis",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 65.39
},
{
"coding": {
"code": "6B25.1",
"display": "Excoriation disorder",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 3.22
},
{
"coding": {
"code": "EB05",
"display": "Urticaria",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 3.07
},
{
"coding": {
"code": "EA80",
"display": "Eczema",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 2.79
},
{
"coding": {
"code": "1E91.Z",
"display": "Herpes zoster",
"system": "https://icd.who.int/en",
"systemAlias": "ICD-11"
},
"probability": 2.49
}
],
"preliminaryFindings": {
"hasConditionSuspicion": 1.0,
"needsSpecialistAttention": 1.0,
"isMalignantSuspicion": 0.0,
"isPreMalignantSuspicion": 0.0
}
}
],
"analysisDuration": "0.6216 secs"
}
- Preliminary report from severity assessment service
Status code: 200
{
"resourceType": "DiagnosticReport",
"status": "preliminary",
"category": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0074",
"code": "IMG",
"display": "Diagnostic Imaging"
}
],
"text": "Diagnostic Imaging"
},
"code": {
"coding": [
{
"system": "http://loinc.org/",
"code": "100063-7",
"display": "Primary skin concern"
},
{
"system": "http://loinc.org/",
"code": "10206-1",
"display": "Physical findings of Skin Narrative"
}
],
"text": "Skin Condition Analysis"
},
"device": {
"name": "Legit.Health Plus",
"manufacturer": "AI LABS GROUP SL",
"category": {
"system": "http://hl7.org/fhir/device-category",
"code": "software",
"display": "Software"
},
"type": {
"system": "https://www.snomed.org/",
"code": "468878001",
"display": "Dermatology picture archiving and communication system application software"
},
"url": "https://medical-device.legit.health/"
},
"effectiveDateTime": "2024-07-05T10:52:26.374451",
"media": {
"validity": {
"metrics": {
"quality": {
"valid": true,
"score": 87,
"category": "Acceptable"
},
"domain": {
"valid": true,
"score": 100,
"category": "Dermatological"
}
},
"isValid": true
},
"modality": "Clinical"
},
"patientEvolution": {
"ascorad": {
"name": "ascorad",
"score": {
"text": "score",
"value": 40.8,
"interpretation": "Severe"
},
"items": {
"surface": {
"type": "visual-sign",
"code": {
"text": "surface-area"
},
"coverage": 40,
"localCoverage": 4
},
"erythema": {
"type": "visual-sign",
"code": {
"text": "erythema"
},
"intensity": 2,
"intensityRaw": 59
},
"crusting": {
"type": "visual-sign",
"code": {
"text": "crusting"
},
"intensity": 2,
"intensityRaw": 82
},
"swelling": {
"type": "visual-sign",
"code": {
"text": "swelling"
},
"intensity": 2,
"intensityRaw": 56
},
"excoriation": {
"type": "visual-sign",
"code": {
"text": "excoriation"
},
"intensity": 2,
"intensityRaw": 73
},
"lichenification": {
"type": "visual-sign",
"code": {
"text": "lichenification"
},
"intensity": 1,
"intensityRaw": 29
},
"dryness": {
"type": "visual-sign",
"code": {
"text": "dryness"
},
"intensity": 1,
"intensityRaw": 47
},
"itchiness": {
"type": "subjective-symptom",
"code": {
"text": "itchiness"
},
"intensity": 3
},
"sleeplessness": {
"type": "subjective-symptom",
"code": {
"text": "sleeplessness"
},
"intensity": 2
}
},
"media": {
"attachments": [
{
"title": "segmentation",
"contentType": "image/jpeg",
"data": "/9j/4AAQSk...",
"height": 527,
"width": 817
}
]
}
}
},
"analysisDuration": "0.3725 secs"
}
-
The response from the
/severity-assessmentendpoint includes any image data in Base64 format. If everything worked correctly, you can find the image results in the response at the JSON path:patientEvolution.ascorad.media.attachments[0].data: -
If we look at the preliminary report generated by the severity assessment service, we can find a Base64 string for a clinical image processed by AI at the JSON path
patientEvolution.ascorad.media.attachments[0].data. The string is truncated to keep the report readable and prevent overwhelming the readers with lengthy data.
Remarks​
No comments to add. The test was carried out manually without any issues.
TEST_016_003​
Outcome​
- Passed
- Failed
Expected results​
- Both endpoints are unable to process the request and return an error message indicating that the image must be encoded in Base64 format.
Actual results​
-
Both endpoints are unable to process the request and return an error message indicating that the image must be encoded in Base64 format:
-
Response body from the
/diagnosis-supportendpoint:
Status code: 422
{
"detail": [
{
"type": "value_error",
"loc": [
"body",
"media",
0,
"data"
],
"msg": "Value error, Only base64 data is allowed",
"input": "b'\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\...'",
"ctx": {
"error": {}
}
}
]
}
- Response body from the
/severity-assessmentendpoint:
Status code: 422
{
"detail": [
{
"type": "value_error",
"loc": [
"body",
"media",
"data"
],
"msg": "Value error, Only base64 data is allowed",
"input": "b'\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\...'",
"ctx": {
"error": {}
}
}
]
}
Remarks​
No comments to add. The test was carried out manually without any issues.
Summary of results​
- Total cases: 3
- Passed: 3
- Failed: 0
- Pass rate: 100 %
Defects and issues​
| Defect ID | Description | Severity | Status | Reported by | Assigned to | Activities generated | Remarks |
|---|---|---|---|---|---|---|---|
Observations and recommendations​
All planned test cases were executed successfully, with a 100% pass rate. This indicates that the system performed as expected under the defined scenarios.
Additionally, no defects were found during this testing cycle. The absence of defects suggests a high level of system stability and reliability.
Signature meaning
The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:
- Tester: JD-017, JD-009, JD-004
- Approver: JD-005