R-TF-012-035: Software Test Report

Document Information

Field	Value
Product / System	Legit.Health Plus
Release version	1.1.0.0
Commit SHA	5f8549e02f3f362db8930906cf6dfdedf232119a
Test management system	TestRail — Project “Medical Device”, Suite “Master”
Prepared by	Gerardo Fernández Moreno
Date	2026-01-19
Reviewed/Approved by	Alejandro Carmena
Review/Approval date	2026-01-19

Executive Summary

This report summarizes the verification activities performed for Legit.Health Plus version 1.1.0.0. The verification campaign confirmed that all software requirements have been met and the system is ready for clinical production.

• Verification Status: 100% Passed. All test cases in the verification and commissioning suites reached "Passed" status.
• Deviations: Zero (0) deviations identified during commissioning.
• Standards Compliance: Verification was conducted according to ISO 62304 and ISO 82304-1.

Test Environment and Levels

Verification was performed across the following levels as defined in the Software Test Plan (R-TF-012-033):

Level	Environment	Focus
Unit Testing	Development	Low-level logic verification of software units.
Integration Testing	Development	Interface verification between internal services.
System Verification	Staging/QA	Requirements-based functional, clinical, and security verification.
Commissioning	Production	Operational readiness and "as-installed" verification.

Preproduction Verification Environment Specification

For the AI Models Integration Verification (Run R18), the testing was conducted using a decoupled Client-Server architecture. The explicit environment specifications used to generate the test results are as follows:

1. System Under Test (Preproduction Cloud Infrastructure)

• Domain: md-prototype.legit.health
• Compute Instance: AWS EC2 g6.8xlarge (NVIDIA L4 Tensor Core GPUs, 32 vCPUs, 130 GiB RAM).
• Storage: 2500 GB EBS Volume.
• Data Persistence: Amazon DynamoDB utilizing 4 dedicated tables (legit_health_plus_api_gateway_users, legit_health_plus_api_gateway_organizations, legit_health_plus_api_gateway_lockouts, legit_health_plus_api_gateway_calls).
• Configuration: Environment variables matching the Production baseline.
• Container Runtime: Docker, orchestrating the api-gateway and 58 specific expert AI models (e.g., condition-classifier:1.0.0, erythema-classifier:1.0.0).
• Test Dataset: Immutable image dataset snapshot stored at s3://legit-health-plus/software-tests/v1.1.0.0/integration-verification-dataset/ to ensure absolute reproducibility.

2. Test Execution Environment (Local Client)

• Hardware: MacBook Pro (Apple Silicon M1).
• Operating System: macOS 26.2
• Runtime: Python 3.10 utilizing the requests library.
• Test Software: Custom integration script (Legit-Health/medical-device-integration-verification-tester, Commit SHA: a9b3209cafa65c0ac176d8c9e141657733786c5a) executing over HTTPS.

Detailed Results: Development Verification

Regulatory objective: Confirm that the software units and internal integrations perform as specified prior to system-level testing (ISO 62304 §5.5, §5.6).

Test artifacts are organized per component, with each test run identified by timestamp and git commit hash for full traceability. Evidence includes JUnit XML test reports, coverage reports (XML, JSON, HTML), and metadata files recording the test environment.

Software Unit Verification (Unit Testing)

• Result: Passed.
• Evidence: s3://legit-health-plus/software-tests/v1.1.0.0/01_development_verification/unit_tests/

Component	Status	Tests	Failed	Line Coverage	Duration
api_gateway	Passed	732	0	94.5%	4.7s
control_plane	Passed	83	0	98.0%	1.8s
report_builder	Passed	845	0	95.7%	2.1s
orchestrator	Passed	255	0	97.5%	1.0s
condition_classifier	Passed	818	0	90.6%	11.6s
essentials	Passed	216	0	86.6%	2.4s
expert_core	Passed	474	0	78.7%	1.3s
Total		3,423	0		24.9s

Each component directory contains:

• test_results/junit.xml — Test case enumeration and pass/fail status
• coverage/ — Line and branch coverage reports (XML, JSON, HTML)
• metadata.json — Version, commit SHA, timestamp, and environment details

Software Integration Verification (Integration Testing)

• Result: Passed.
• Evidence: s3://legit-health-plus/software-tests/v1.1.0.0/01_development_verification/integration_tests/

Component	Status	Tests	Failed	Duration
api_gateway	Passed	358	0	27.4s
control_plane	Passed	83	0	3.5s
report_builder	Passed	138	0	1.7s
orchestrator	Passed	106	0	1.4s
condition_classifier	Passed	211	0	7.1s
expert_core	Passed	226	0	4.1s
Total		1,122	0	45.2s

Each component directory contains:

• test_results/junit.xml — Test case enumeration and pass/fail status
• metadata.json — Version, commit SHA, timestamp, and environment details

Detailed Results: System Verification

This section contains the results of the requirements verification performed in the controlled Staging environment.

Test Case Id	Test Case Title	Test Id	Test URL	Status
C50	Verify the API service accepts incoming HTTP requests on the designated network port	T105	View	✅
C62	Verify API returns 200 HTTP status codes for successful requests	T106	View	✅
C68	Verify API processes JSON requests and returns JSON responses with correct Content-Type headers	T107	View	✅
C77	Verify successful user authentication and token generation via the POST /auth/login endpoint	T109	View	✅
C73	Verify retrieval and filtering of clinical signs data via /clinical/severity-experts endpoint	T108	View	✅
C106	Verify API endpoints are accessible via URL paths prefixed with the major and minor version identifier	T110	View	✅
C110	Verify the API returns HTTP 413 when the request body exceeds the configured maximum size	T111	View	✅
C124	Verify POST /clinical/severity-assessment returns quantified results for valid image and sign list	T112	View	✅
C128	Verify the diagnosis-support endpoint accepts valid images and returns diagnostic analysis	T113	View	✅
C162	Verify simultaneous availability and processing of requests across distinct API versions	T114	View	✅
C330	Verify API rejects malformed inputs with standardized 422 Unprocessable Entity responses	T115	View	✅
C331	Verify API returns sanitized error responses with appropriate HTTP status codes and no internal details	T116	View	✅
C454	Verify API returns 401 HTTP status codes for wrong login requests	T272	View	✅
C455	Verify API returns 422 HTTP status code when invalid data is submitted	T273	View	✅
C456	Verification of controlled 503 response and graceful degradation during downstream service failure.	T300	View	✅
C169	Verify health check endpoint returns unhealthy when some service is unavailable	T121	View	✅
C46	Verify the public health endpoint returns HTTP 200 and status OK when operational	T118	View	✅
C66	Verify retrieval of mandatory legal information, UDI, and regulatory metadata via API	T119	View	✅
C543	Verify that the eIFU is reachable, accessible and navigable via the link provided on the electronic label	T384	View	✅
C159	Verify availability of OpenAPI specification and interactive documentation endpoints	T120	View	✅
C255	Verify API returns aggregated ICD probability distribution with structured code details in studyAggregate array	T122	View	✅
C256	Verify response includes per-image ICD probabilities and heat maps for the top five categories	T123	View	✅
C258	Verify response includes normalized entropy score between 0 and 1 in findings	T125	View	✅
C260	Verify report response includes highPriorityReferral score within riskMetrics object	T127	View	✅
C261	Verify report response includes malignantConditionProbability score within riskMetrics object	T128	View	✅
C262	Verify report response includes pigmentedLesion score within riskMetrics object	T129	View	✅
C263	Verify report response includes anyConditionProbability score within riskMetrics object	T130	View	✅
C264	Verify report response includes urgentReferral score within riskMetrics object	T131	View	✅
C265	Verify diagnosis workflow returns ranked ICD-11 codes, binary indicators, and explainability maps for valid images	T132	View	✅
C266	Verify epithelial tissue classification returns right presence prediction and confidence score	T133	View	✅
C267	Verify API returns Hurley stage and inflammatory status with associated probabilities for valid image input	T134	View	✅
C268	Verify tissue wound bed necrotic classification returns right presence prediction and confidence score	T135	View	✅
C269	Verify pustule classification returns right intensity and confidence	T136	View	✅
C270	Verify inflammatory nodular lesion detector return correct counts and bounding boxes for drainning tunnels	T137	View	✅
C271	Verify wound borders delimited classification returns right presence prediction and confidence score	T138	View	✅
C272	Verify wound exudation serous classification returns right presence prediction and confidence score	T139	View	✅
C274	Verify wound exudation purulent classification returns right presence prediction and confidence score	T141	View	✅
C275	Verify wound maceration segmentation analysis returns segmentation masks and the right percentage of surface affected	T142	View	✅
C276	Verify erythema classification returns right intensity and confidence	T143	View	✅
C277	Verify crusting classification returns right intensity and confidence	T144	View	✅
C278	Verify thickened wound borders classification returns right presence prediction and confidence score	T145	View	✅
C279	Verify induration classification returns right intensity and confidence	T146	View	✅
C280	Verify hair loss segmentation analysis returns segmentation masks and the right percentage of surface affected	T147	View	✅
C281	Verify wound perilesional erythema classification returns right presence prediction and confidence score	T148	View	✅
C282	Verify wound stage classification returns right score and confidence metrics from a valid wound image	T149	View	✅
C283	Verify erythema segmentation analysis returns segmentation masks and the right percentage of surface affected	T150	View	✅
C284	Verify lichenification classification returns right intensity and confidence	T151	View	✅
C285	Verify wound affected tissues intact classification returns right presence prediction and confidence score	T152	View	✅
C286	Verify API returns follicle count, bounding boxes, and confidence scores for a valid scalp image	T153	View	✅
C287	Verify wound borders indistinguishable classification returns right presence prediction and confidence score	T154	View	✅
C288	Verify wound affected tissues subcutaneous classification returns right presence prediction and confidence score	T155	View	✅
C289	Verify wound orthopedic material segmentation analysis returns segmentation masks and the right percentage of surface affected	T156	View	✅
C290	Verify wound borders damaged classification returns right presence prediction and confidence score	T157	View	✅
C291	Verify wound biofilm material segmentation analysis returns segmentation masks and the right percentage of surface affected	T158	View	✅
C292	Verify xerosis classification returns right intensity and confidence	T159	View	✅
C293	Verify wound granulation segmentation analysis returns segmentation masks and the right percentage of surface affected	T160	View	✅
C294	Verify wound bone segmentation analysis returns segmentation masks and the right percentage of surface affected	T161	View	✅
C295	Verify swelling classification returns right intensity and confidence	T162	View	✅
C296	Verify wound exudation serous classification returns right presence prediction and confidence score	T163	View	✅
C297	Verify wound affected tissues muscle classification returns right presence prediction and confidence score	T164	View	✅
C298	Verify skin segmentation analysis returns segmentation masks and the right percentage of surface affected	T165	View	✅
C299	Verify hive detector return correct counts and bounding boxes for hives	T166	View	✅
C300	Verify wound biofilm tissue classification returns right presence prediction and confidence score	T167	View	✅
C302	Verify tissue wound bed slough classification returns right presence prediction and confidence score	T169	View	✅
C303	Verify desquamation classification returns right intensity and confidence	T170	View	✅
C304	Verify hypopigmentation segmentation analysis returns segmentation masks and the right percentage of surface affected	T171	View	✅
C305	Verify wound borders diffused classification returns right presence prediction and confidence score	T172	View	✅
C306	Verify wound bed segmentation analysis returns segmentation masks and the right percentage of surface affected	T173	View	✅
C307	Verify oozing classification returns right intensity and confidence	T174	View	✅
C308	Verify wound affected tissues bone classification returns right presence prediction and confidence score	T175	View	✅
C309	Verify acneiform detector return correct counts and bounding boxes for papules, pustules, spots	T176	View	✅
C310	Verify AWOSI classification returns right score and confidence metrics from a valid wound image	T177	View	✅
C311	Verify tissue wound bed closed classification returns right presence prediction and confidence score	T178	View	✅
C312	Verify wound perilesional maceration classification returns right presence prediction and confidence score	T179	View	✅
C313	Verify nail lesion segmentation analysis returns segmentation masks and the right percentage of surface affected	T180	View	✅
C314	Verify excoriation classification returns right intensity and confidence	T181	View	✅
C315	Verify wound necrosis segmentation analysis returns segmentation masks and the right percentage of surface affected	T182	View	✅
C316	Verify hyperpigmentation segmentation analysis returns segmentation masks and the right percentage of surface affected	T183	View	✅
C317	Verify wound bloody exudation classification returns right presence prediction and confidence score	T184	View	✅
C318	Verify wound exudation fibrinous classification returns right presence prediction and confidence score	T185	View	✅
C319	Verify follicular and inflammatory pattern identification returns right result	T186	View	✅
C320	Verify wound tissue wound bed granulation classification returns right presence prediction and confidence score	T187	View	✅
C321	Verify generation of structured clinical assessment report with quantified results for requested signs via API	T188	View	✅
C325	Verify body surface segmentation analysis returns segmentation masks and the right percentage of surface affected	T193	View	✅
C446	Verify acneiform detector return correct counts and bounding boxes for nodules, pustules and scabs	T264	View	✅
C447	Verify acneiform detector return correct counts and bounding boxes for scabs, comedones, papules and pustules	T265	View	✅
C448	Verify hive detector return correct counts and bounding boxes for hives (second image)	T266	View	✅
C449	Verify inflammatory nodular lesion detector return correct counts and bounding boxes for non drainning tunnels	T267	View	✅
C450	Verify inflammatory nodular lesion detector return correct counts and bounding boxes for nodules	T268	View	✅
C323	Verify head detection returns right bounding boxes and heads count inside an image	T191	View	✅
C327	Verify API returns ""clinical"" for image modality category when a skin image is provided	T195	View	✅
C328	Verify API returns image domain category equals to ""dermatological"" and confidence score for a skin image	T196	View	✅
C329	Verify API returns dermatological image quality score, interpretation, and acquisition feedback	T197	View	✅
C451	Verify API returns ""dermoscopic"" for image modality category when a skin image is provided	T269	View	✅
C452	Verify API returns image domain category equals to ""non_dermatological"" and confidence score for a dog image	T270	View	✅
C453	Verify FHIR DiagnosticReport base structure for a segmenter	T271	View	✅
C332	Verify API accepts requests over HTTPS using TLS 1.2 or 1.3	T198	View	✅
C333	Verify API rejects or redirects unencrypted HTTP requests	T199	View	✅
C335	Verify progressive increase in enforced delay across consecutive failed authentication attempts	T201	View	✅
C336	Verify delay resets upon successful authentication	T202	View	✅
C337	Verify successful access to permitted endpoints for an authorized role	T203	View	✅
C338	Verify access denial for endpoints outside the assigned role scope	T204	View	✅
C339	Verify HTTP 403 response when request volume exceeds defined threshold	T205	View	✅
C340	Verify request acceptance after rate limit time window expiration	T206	View	✅
C341	Verify generated authentication tokens include the expiration claim	T207	View	✅
C342	Verify access denial for requests using an expired JWT	T208	View	✅
C343	Verify generation of authentication token using valid credentials	T209	View	✅
C344	Verify rejection of authentication requests with invalid credentials	T210	View	✅
C345	Verify password update functionality and subsequent authentication	T211	View	✅
C346	Verify account lockout enforcement after threshold reached	T212	View	✅
C347	Verify failed attempt counter reset on successful login	T213	View	✅
C348	Verify administrative manual account unlock capability	T214	View	✅
C349	Verify enforcement of password complexity and length constraints	T215	View	✅
C350	Verify authentication behavior for expired passwords	T216	View	✅
C351	Verify protected endpoints allow access with a valid OAuth 2.0 Bearer token	T217	View	✅
C352	Verify protected endpoints reject requests lacking a valid token with 401 Unauthorized	T218	View	✅
C353	Verify public endpoints are accessible without an Authorization header	T219	View	✅
C354	Verify AES-256 encryption configuration for data storage	T220	View	✅
C355	Verify authorized administrator can retrieve current user information for review	T221	View	✅
C356	Verify authorized administrator can revoke permissions during access review	T222	View	✅
C357	Verify successful execution and audit logging of system integrity checks	T223	View	✅
C363	Verify backup generation	T225	View	✅
C364	Verify automated backup generation	T226	View	✅
C366	Verify blocking of anomalous high-frequency request bursts	T228	View	✅
C368	Verify FHIR DiagnosticReport base structure for a detector	T230	View	✅
C369	Verify analysisDuration field population in DiagnosticReport	T231	View	✅
C370	Verify isAssessable is true when domain and quality criteria are met	T232	View	✅
C371	Verify isAssessable is false when image quality is unacceptable	T233	View	✅
C372	Verify isAssessable is false when image is non-dermatological	T234	View	✅
C373	Verify single image analysis maps to structured object in imageAnalyses array	T235	View	✅
C374	Verify multiple image analyses map to distinct objects in imagingAnalysis array	T236	View	✅
C375	Verify response structure compliance with OpenAPI success schema	T237	View	✅
C376	Verify response structure compliance with OpenAPI error schema	T238	View	✅
C377	Verify Assignment of Official Identifier to DiagnosticReport	T239	View	✅
C378	Verify Uniqueness of Generated DiagnosticReport Identifiers	T240	View	✅
C382	Verify System Timestamp Accuracy via API Response Headers	T244	View	✅
C383	Verify System Time Synchronization and Accuracy Status	T245	View	✅
C388	Verify Role-Based Access Control for Audit Trail Interface	T247	View	✅
C389	Verify Audit Trail Search and Export Capabilities	T248	View	✅
C391	Verify audit records cannot be modified or deleted via API	T249	View	✅
C395	Verify audit trail generation for authentication lifecycle and security anomalies	T251	View	✅
C398	Verify audit record completeness for successful API event	T252	View	✅
C399	Verify audit record completeness for failed API event	T253	View	✅
C410	Verify audit trail generation for clinical data creation events	T255	View	✅
C413	Audit record preservation during database unavailability	T258	View	✅
C416	Verify p95 API latency remains under 10 seconds during nominal load	T261	View	✅

Detailed Results: AI Models Integration Verification

Regulatory objective: Confirm that each AI model integrated into the medical device produces the expected outputs when invoked in the preproduction environment, verifying correct model loading, preprocessing, inference, and post-processing (IEC 62304 §5.6, ISO 82304-1).

• Result: Passed.
• Evidence: TestRail Plan R17, Run R18 — AI Models Integration Plan — Preproduction. Per-model evidence stored at s3://legit-health-plus/software-tests/v1.1.0.0/integration-verification-dataset/.

All tests were executed using the Preproduction Verification Environment specified in Section 3, ensuring identical AI compute hardware (AWS g6.8xlarge) as intended for production.

Metric	Value
Test Cases	76
Passed	76
Failed	0
Pass Rate	100%
Tested By	Gerardo Fernández
Execution Date	2026-01-15

All 76 test cases verified that the integrated AI models (classifiers, segmenters, detectors, and pattern identifiers) return the expected probability distributions, scores, segmentation masks, and detection outputs for controlled reference images. Each test case stores both the request payload and the response payload as evidence in S3.

Test case details are provided in the controlled export: ai-models-integration-tests.csv.

Detailed Results: Commissioning (Production)

This section details the operational readiness verification performed on the deployed production instance (AWS eu-west-3).

Test Case Id	Test Case Title	Reviewer	Review Status	Test Id	Status
C420	Deployment Environment Verification	Gerardo Fernández	Approved	T274	✅
C421	Container Image Verification	Gerardo Fernández	Approved	T275	✅
C422	Configuration Baseline Verification	Gerardo Fernández	Approved	T276	✅
C423	Security Configuration Verification	Gerardo Fernández	Approved	T277	✅
C424	External Dependency Availability	Gerardo Fernández	Approved	T278	✅
C425	Deployment Deviations Review	Gerardo Fernández	Approved	T279	✅
C426	Deployment Acceptance	Gerardo Fernández	Approved	T280	✅
C427	Test Data Controls	Gerardo Fernández	Approved	T281	✅
C428	Credentials and Access Controls	Gerardo Fernández	Approved	T282	✅
C429	API Availability and Routing	Gerardo Fernández	Approved	T283	✅
C430	Authentication and Authorization Enforcement	Gerardo Fernández	Approved	T284	✅
C431	Input Validation and Error Handling	Gerardo Fernández	Approved	T285	✅
C432	Internal Integration Sanity Checks	Gerardo Fernández	Approved	T286	✅
C433	Audit Logging Verification	Gerardo Fernández	Approved	T287	✅
C434	Failure Mode Sanity Checks	Gerardo Fernández	Approved	T288	✅
C435	Functional Deviations Review	Gerardo Fernández	Approved	T289	✅
C436	Functional Commissioning Acceptance	Gerardo Fernández	Approved	T290	✅
C439	AI/ML Model Runtime Availability	Gerardo Fernández	Approved	T293	✅
C440	Clinical Output Integrity and Structure	Gerardo Fernández	Approved	T294	✅
C441	Report Generation and Serialization	Gerardo Fernández	Approved	T295	✅
C442	Operational Readiness Checks	Gerardo Fernández	Approved	T296	✅
C443	Clinical Workflow Limitations and Known Constraints	Gerardo Fernández	Approved	T297	✅
C444	Clinical Deviations Review	Gerardo Fernández	Approved	T298	✅
C445	Clinical Commissioning Acceptance	Gerardo Fernández	Approved	T299	✅
C457	Diagnosis Support of Clinical Images	Gerardo Fernández	Approved	T301	✅
C458	Severity Assessment – PASI Workflow	Gerardo Fernández	Approved	T302	✅
C459	Severity Assessment – UAS Workflow	Gerardo Fernández	Approved	T303	✅
C460	Severity Assessment – IHS4 Workflow	Gerardo Fernández	Approved	T304	✅
C461	Severity Assessment – SCORAD Workflow	Gerardo Fernández	Approved	T305	✅
C462	Verification of End-to-End Clinical Workflow Presentation and Scoring Interface.	Gerardo Fernández	Approved	T306	✅

Anomalies and Deviations Review

Verification Anomalies

No anomalies were identified during the system verification phase. All requirements were verified against the specified acceptance criteria.

Commissioning Deviations

The commissioning campaign concluded with no identified deviations.

• Infrastructure: AWS account ID and Region match the declared production environment.
• Security: TLS, JWT Authentication and IAM credentials are active and enforced.
• Clinical: All 58 expert models are resident in memory and producing structured JSON outputs.

Conclusion

The software version 1.1.0.0 is formally verified and commissioned. Based on the 100% pass rate and the absence of deviations, the device is considered safe and effective for its intended clinical use in the production environment.

Additional graphical reports and execution logs can be retrieved from the TestRail PDF Export and the S3 evidence URI identified in the commissioning records.

Signature meaning

The signatures for the approval process of this document can be found in the verified commits at the repository for the QMS. As a reference, the team members who are expected to participate in this document and their roles in the approval process, as defined in Annex I Responsibility Matrix of the GP-001, are:

• Author: Team members involved
• Reviewer: JD-003 Design & Development Manager, JD-004 Quality Manager & Person Responsible for Regulatory Compliance (PRRC)
• Approver: JD-001 General Manager