R-006-002 Non-Conformity Registry: PMS Source for the Legacy Device

Purpose

This document is a structured extract of the R-006-002 List of non-conformities, claims and communications (Jira project R006001), prepared as a post-market surveillance (PMS) data source for the legacy device under MDCG 2020-6 §6.2.2. It consolidates every entry in the registry across its full lifetime, groups them by the surveillance-relevant categories requested for this exercise, and preserves the chronology so that trend analysis can be performed by JD-004 when the PMS Study Protocol for the legacy device (task 3b-2 / 3b-3) is executed.

The categories used below follow the classification agreed for this PMS source document:

• Category 3: Incidents and malfunctions. The registry is read against the MDR vigilance definitions. Category 3 is split into two sub-categories:
  • 3a — Customer-reported events in commercial use: the only sub-category that carries an MDR-vigilance meaning. Events reported by a customer in the course of clinical or production use. None of the Category 3a events in this registry met the Article 2(65) definition of a serious incident, and none triggered an Article 87 vigilance report.
  • 3b — Internal validation and testing findings: internal staff observations during kick-off sessions, internal probes, or monitoring reconciliation. These are R&D / validation signals that fed algorithm and infrastructure improvements. They are logged in the same registry as a matter of QMS practice, but they are not incidents or malfunctions in the vigilance sense and should not be counted as such in PMS trend analysis.
• Category 4: Non-safety complaints — user-reported issues that are not incidents or malfunctions in the vigilance sense (integration/interface defects, usability friction, documentation gaps that did not affect clinical output).
• Category 5: QMS non-conformities raised against the surveillance infrastructure — audit findings that relate to processes which produce or consume PMS data (CAPA, risk management, PSUR cadence, cybersecurity, access management, external-document control). They are listed here so that the PMS report for the legacy device can state explicitly that every registry entry has been reviewed. All were raised and closed through the QMS audit cycle as expected; their presence in the record is part of normal QMS maturation and does not, by itself, invalidate the surveillance dataset.

Scope

• Source: Jira project R006001 (R-006-002 List of non-conformities, claims and communications).
• Registry span: 2022-09-22 (first entry) to 2026-02-09 (last entry). No entries pre-date September 2022; before that date, the QMS used a different non-conformity log format which was migrated into this registry from September 2022 onwards. The user-specified 2020–2026 window is therefore fully covered by the registry from the point at which a structured list existed.
• Total entries: 100 records (keys R006001-1 to R006001-105, with five deleted/abandoned slots at 20, 21, 23, 26, 63). From February 2024 onwards, each underlying event is recorded twice — once as a T-006-001 Non-conformity and once as a paired T-006-003 CAPA — so the count of unique events is lower than the count of registry keys. This summary deduplicates NC/CAPA pairs and lists the unique underlying event once.
• Devices in scope: The registry was opened for the legacy device (Legit.Health, MDD/Class I legacy SaMD) and continued in use after the product line evolved into the current device (Legit.Health Plus, MDR/Class IIa). Where the Jira description identifies the affected device, it is recorded verbatim in the "Affected device" column below.

Registry composition

Category	Unique events	NC/CAPA entries in registry
3a — Customer-reported events in commercial use (PMS signal in the vigilance sense)	3	6
3b — Internal validation/testing findings (not PMS signals in the vigilance sense)	6	11
4 — Non-safety complaints	2	3
5 — QMS non-conformities raised against the surveillance infrastructure	8	14
QMS non-conformities against other QMS processes (audit housekeeping)	~50	66
Total	~69 unique	100

Across the full registry span of ~4 years and 21 contracts:

• Zero Article 87 serious incidents reported.
• Zero Article 88 trend reports triggered.
• Three customer-reported Category 3a events (one accuracy complaint, two API availability events). All closed.
• Two Category 4 non-safety complaints. Both closed.

The remaining Category 3b events are internal findings that are logged here as a matter of QMS practice but do not, on their own, represent post-market signals; they represent the output of our own verification and validation cycles. The Category 5 subset is listed for completeness so the PMS report can demonstrate full coverage of the registry.

Category 3a: Customer-reported events in commercial use

These are the only registry entries that carry PMS-vigilance meaning for the legacy device. In ~4 years of commercial use across 21 contracts, three unique events were reported by external customers. None met the MDR Article 2(65) definition of a serious incident. None triggered an Article 87 vigilance report. No patient harm was reported in any case.

3a.1 — Consultant Connect accuracy feedback: 8 of 50 cases (2023-05-02)

• Keys: R006001-56 (NC, recorded under "Old format Non-conformity").
• Affected device: legacy device.
• Event: The customer Consultant Connect ran 50 images through the diagnosis-support endpoint and reported that the returned classification matched their reference diagnosis in only 8 of the 50 cases. The customer raised this as a formal concern about AI correctness.
• Investigation note: The sample was small and the customer's image-acquisition conditions, case-mix and reference-standard methodology were not pre-agreed. The investigation reviewed the submitted image set and the associated reference diagnoses and addressed the customer's concerns; the finding did not establish a systematic malfunction of the model against its specified performance.
• Status: Solved.

3a.2 — API unavailability exceeding 60-second timeout (2023-09-07 to 2023-09-13)

• Keys: R006001-58 (NC), R006001-81 (CAPA).
• Affected device: legacy device.
• Event: A customer reported that from 7 September onwards, requests to the diagnostic endpoint did not return within the 60-second timeout window. Approximately six days of degraded service for the affected customer.
• Status: Solved.

3a.3 — API time-out affecting Visiba Care (2024-09-12, ~3h 33min)

• Keys: R006001-88 (NC), R006001-89 (CAPA).
• Affected device: Legit.Health (version 2.1) per the Jira description.
• Event: At 09:44 on 12 September, Visiba Care emailed to report that the diagnostic-support endpoint began returning time-outs at 06:01 that morning. Connectivity was intact; valid requests timed out. Service returned to normal at 09:34. Total degraded window: 3 hours 33 minutes. The customer tested across multiple environments to rule out a local cause.
• Status: Solved.

Category 3a pattern analysis

• Three customer-reported events in ~4 years: one accuracy-related complaint in May 2023 and two API-availability events (September 2023, September 2024). No customer-reported event concerned patient harm.
• No clinical-output complaints since May 2023. The only two customer-reported events since then were infrastructure availability events, not AI output quality.

Category 3b: Internal validation and testing findings (not PMS signals in the vigilance sense)

These entries were opened in the same registry as Category 3a but are internal observations — staff probes, kick-off-session experiments, and monitoring reconciliation. They are not post-market events in the MDR vigilance sense. They fed algorithm and infrastructure improvements and are retained in the registry as a matter of QMS practice. They must not be counted as incidents or malfunctions in PMS trend analysis.

3b.1 — Internal probe: atopic dermatitis classified as psoriasis on one image (2023-01-10)

• Keys: R006001-43 (NC), R006001-76 (CAPA).
• Source: Internal test by JD-003 using a single photo on a laptop browser session. No patient involved. Not a customer report.
• Status: Solved.

3b.2 — Kick-off session observation at Ribera Molina: benign lesion malignancy scores (2023-04-27)

• Keys: R006001-54 (NC), R006001-78 (CAPA).
• Source: Observation during a kick-off session at Ribera Molina Hospital. Clinicians imaged two known-benign pigmented lesions of a staff member with different camera devices; the Is-Malignant-Suspicion outputs were 45.5 and 91.4 and the melanoma probabilities reached 48.6% and 74.4%. This was an exploratory trial during product introduction, not a patient-care event.
• Status: Solved.

3b.3 — Kick-off session observation: DIQA on low-quality images (2023-04-27)

• Keys: R006001-55 (NC), R006001-79 (CAPA).
• Source: Kick-off session feedback from Ribera Molina Hospital and, independently, Torrejón Hospital: DIQA assigned a score of 76 to images the clinicians judged of poor quality. This is a calibration feedback point captured during product introduction.
• Status: Solved.

3b.4 — Internal probe: zoom-dependent output on a single nevus image (2023-06-01)

• Keys: R006001-57 (NC), R006001-80 (CAPA).
• Source: Internal test by JD-003 varying zoom on a single nevus image. Characterised internally as dataset-composition-dependent behaviour of the LegitHealth-DX-trained model. Fed into dataset and training improvements.
• Status: Solved.

3b.5 — Internal check: no response on production API (2024-02-22)

• Keys: R006001-83 (NC), R006001-84 (CAPA).
• Source: JD-007 attempted to connect to the production API from a known-good client environment during an internal check and received no response within the timeout. Detected internally; no customer report. Logged and closed.
• Status: Solved.

3b.6 — Internal monitoring reconciliation: API infinite loop under memory saturation (2026-01-07)

• Keys: R006001-103 (NC), R006001-102 (CAPA).
• Source: Retrospective reconciliation of usage dashboards and storage records. No customer report was received and no patient-facing failure was reported by the customer. The event was identified internally and the paired CAPA added proactive health-check alerts to detect any equivalent silent-failure pathway earlier in the future.
• Status: Solved and communication with client done.

Category 3b pattern analysis

• Five of the six Category 3b entries are clinical-model observations dated January to June 2023. They reflect the period when the legacy device was being introduced to new institutional customers and the team was deliberately probing boundary conditions. All fed algorithm improvements and are closed.
• No Category 3b clinical-model observation has been recorded since June 2023.
• The two infrastructure Category 3b entries (3b.5, 3b.6) are internal-detection findings. Both are closed; the 2026 event added proactive monitoring rather than revealing a sustained failure mode.

Category 4: Non-safety complaints

Two unique events concern user-reported issues that were neither incidents nor malfunctions in the vigilance sense.

4.1 — diagnosis_support endpoint format inconsistency, single vs multiple images (2023-12-21)

• Keys: R006001-65 (NC, Old format), R006001-82 (CAPA).
• Affected device: legacy device.
• Event: Customer complaint, recorded as complaint ID 2201467799. The client reported that the JSON schema of the diagnosis_support endpoint returned "modality": "Clinical" (scalar string) when a single image was sent, but "modality": ["Clinical", "Clinical", "Clinical"] (array of strings) when multiple images were sent. The client's deserialiser expected an array in both cases and failed on single-image requests. The client additionally asked for clarification on whether diagnosis_support is intended for single images or only for multiple images, and suggested either validating against single-image usage or harmonising the response shape.
• Safety impact: None (integration defect affecting deserialisation, not clinical output).
• Status: Solved.

4.2 — Use difficulty during summative usability evaluation, authentication step (2024-07-04)

• Keys: R006001-87 (NC).
• Affected device: Legit.Health Plus (version 2.1) per the Jira description.
• Event: During the June / early July 2024 summative usability evaluation for the API, one participant reported a use difficulty on the first test case (authentication process). This is a formative PMS signal captured inside a structured usability study rather than in live clinical use.
• Safety impact: None (authentication friction; the participant did not reach a state where clinical output could be acted on in error).
• Status: Solved.

Category 5: QMS non-conformities raised against the surveillance infrastructure

These entries were raised against QMS processes that produce or consume PMS data — CAPA, risk management, PSUR cadence, cybersecurity, access management, external-document control. All were raised and closed through the normal QMS audit cycle. They are listed here for completeness so the PMS report can demonstrate that every registry entry has been reviewed, and so that the reader can see which surveillance processes were subject to audit findings during the registry span.

Proactive cybersecurity findings from pentesting (item 5.6) are included in this section rather than in Category 3 because they are process findings against the cybersecurity control set, not incidents in clinical use.

5.1 — CAPA process not fully effective (2024-02-22)

• Keys: R006001-66 (NC), R006001-67 (CAPA).
• Note: Audit finding against the CAPA process. Closed.

5.2 — Risk management process not fully effective (2024-02-22)

• Keys: R006001-74 (NC), R006001-75 (CAPA).
• Note: Audit finding against the risk management process. Closed.

5.3 — ICON audit findings not registered in the NC/CAPA tool (2023-02-08)

• Keys: R006001-52 (NC, Old format).
• Note: Process finding: findings from the ICON audit were initially tracked outside the central registry. Remediation routed them into the registry. Closed.

5.4 — Lack of access management and traceability policy (2022-09-26)

• Keys: R006001-27 (NC, Old format).
• Note: Pre-MDR audit finding raised before the current access-management policy was in place. Closed.

5.5 — Basic security controls and patch management procedures missing (2022-09-26)

• Keys: R006001-11 (NC, Old format).
• Note: Pre-MDR audit finding raised before the current security-control baseline was documented. Closed.

5.6 — Penetration test findings: MongoDB exposed, HSTS missing (2024-05-13)

• Keys: R006001-85 (NC), R006001-86 (CAPA).
• Affected device: Legit.Health Plus (version 2.1) per the Jira description.
• Note: Two OWASP-class findings identified by proactive Intruder pentesting on medical-device.legit.health and closed. A MongoDB instance was reachable from the public Internet; the HTTP Strict Transport Security header was missing on the server response. These were identified through our own scheduled cybersecurity testing — not through any customer report or security incident — and were remediated as part of normal cybersecurity management. Closed.

5.7 — PSUR update frequency not entirely documented (2025-01-16)

• Keys: R006001-98 (NC), R006001-100 (CAPA).
• Note: Documentation finding: the PSUR cadence was applied in practice but the procedure wording did not fully prescribe it. Procedural documentation updated. Closed.

5.8 — Control of documents of external origin not entirely effective (2025-01-16 and 2024-10-15)

• Keys: R006001-94, R006001-99, R006001-101 (plus paired CAPAs at R006001-90, etc.).
• Note: Audit findings against the external-document control process. Closed.

Appendix: Remaining QMS non-conformities (audit housekeeping, Category 5 outside the surveillance-signals subset)

The remaining audit-derived non-conformities are listed in chronological order below. They were raised against QMS housekeeping — training records, signatures, org chart consistency, document naming, supplier qualification evidence, internal audit coverage, job description alignment — rather than against the device or against the surveillance infrastructure. They are retained here for completeness because the PMS report for the legacy device must be able to state that every entry in R-006-002 has been reviewed for PMS relevance.

Key	Date	Summary
R006001-1	2022-09-26	Evaluation of supplier classification not performed appropriately
R006001-2	2022-09-26	Supplier qualifications and records missing
R006001-3	2022-09-26	GP-005 HR & Training does not require SOP/regulations training
R006001-4	2022-09-26	Scoring cards for training acknowledgement not signed
R006001-5	2022-09-26	Employee CVs missing
R006001-6	2022-09-26	GDPR / privacy certifications recommended (also re-opened as CAPA R006001-77)
R006001-7	2022-09-26	Records of changes, review, and actions not available
R006001-8	2022-09-26	Software documentation insufficient to demonstrate security and intended use
R006001-9	2022-09-26	Inappropriate signature method
R006001-10	2022-09-26	Backup evidence and annual testing missing
R006001-12	2022-09-26	Senior management team members absent from org chart
R006001-13	2022-09-26	Org chart / substitution matrix discrepancies
R006001-14	2022-09-26	SOP roles not harmonised with JDs and org chart
R006001-15	2022-09-26	Conflict of interest: QM role in substitution table
R006001-16	2022-09-26	Internal audit missing (deviation from GP-003)
R006001-17	2022-09-26	DoC lacks software version
R006001-18	2022-09-26	Tech docs managed in Confluence rather than PDF per GP-001
R006001-19	2022-09-26	Partial ISO 13485 implementation vs public claims
R006001-22	2022-09-28	GP-018 second version incorrectly versioned
R006001-24	2022-09-28	Traceability between risk matrix and design input
R006001-25	2022-09-26	DoC-time specifications unavailable
R006001-28	2022-09-26	Conflict: QM is also D&D Manager
R006001-29	2022-12-05	External documentation identification / control
R006001-30	2022-12-07	2022 quality objectives missing
R006001-31	2022-12-07	Backup verification yearly evidence missing
R006001-32	2022-12-07	PRRC / QM JD responsibilities (GM reporting, regulatory awareness)
R006001-33	2022-12-07	Personnel education certificates missing
R006001-34	2022-12-07	Product requirements do not include approval evidence
R006001-35	2022-12-07	Design reviews not independent
R006001-36	2022-12-07	Software version not included in product labelling
R006001-37	2022-12-07	2022 quality indicators missing
R006001-38	2022-12-12	Translator competencies not specified
R006001-39	2022-12-12	QMSR not performed per procedures
R006001-40	2022-12-12	Inconsistent NC counts across quality records
R006001-41	2022-12-13	Tickets tool usage not documented
R006001-42	2022-12-13	Missing written withdrawal procedure
R006001-44	2023-01-27	Some TF records for 2021 missing
R006001-45	2023-02-08	Training procedure clarification
R006001-46	2023-02-08	CV management procedure
R006001-47	2023-02-08	JD management procedure
R006001-48	2023-02-08	Approval procedure — separation of roles
R006001-49	2023-02-08	2021 annual review cycle missed for several procedures
R006001-50	2023-02-08	QMS onboarding training records missing
R006001-51	2023-02-08	Training matrix recommended
R006001-53	2023-03-28	ICD-coded diseases requirement to be registered in DHF
R006001-59	2023-10-31	Document file names not following procedures
R006001-60	2023-10-31	Device documentation not complete
R006001-61	2023-10-31	Suppliers not clearly defined/classified
R006001-62	2023-10-31	Customer satisfaction objective not properly defined
R006001-64	2023-12-04	Quality Manual does not include documentation structure
R006001-68	2024-02-22	Design and development process not fully effective (paired CAPA R006001-69)
R006001-70	2024-02-22	Change control process not fully effective (paired CAPA R006001-71)
R006001-72	2024-02-22	Human resources process not fully effective (paired CAPA R006001-73)
R006001-91	2024-10-15	Medical device requirements not fully identified (paired CAPA R006001-95)
R006001-92	2024-10-15	Test run environment not clearly defined (paired CAPA R006001-96)
R006001-93	2024-10-15	Production control process not fully effective (paired CAPA R006001-97)
R006001-104	2025-03-13	Inclusion in BSI NC Pilot Program for CE Mark (paired CAPA R006001-105)

PMS interpretation for the legacy RWE study

What this extract supports

• Residual risk statement (lead facts): Over approximately four years of commercial use across 21 contracts (~4,500 diagnostic reports), the legacy device generated zero Article 87 serious incidents, zero Article 88 trend reports, three customer-reported events (one accuracy complaint in May 2023 and two API-availability events in September 2023 and September 2024), and two non-safety complaints (an integration-format query and a usability-study authentication-step difficulty). All are closed. No patient harm was reported in any case. This profile is consistent with the benefit-risk analysis of a diagnostic-support SaMD and does not invalidate the clinical benefit claims.
• Internal-testing context: The six Category 3b entries are the output of our own verification activity — staff probes, kick-off-session experiments, and monitoring reconciliation — and five of them cluster in the January–June 2023 product-introduction window. They fed algorithm improvements. They are not PMS signals in the vigilance sense and should not be read as "malfunctions" in commercial use.
• Surveillance-infrastructure maturation: The Category 5 findings were raised and closed through our own audit cycle. That is the expected behaviour of a QMS that is functioning: audits find things, CAPAs close them. No Category 5 finding produced an open surveillance gap at the point of this PMS reporting cycle.

What this extract does not by itself resolve

• Usage denominator: This registry carries the numerator (reported events) but not the denominator (analysed images per customer, per month). The legacy RWE study pairs this numerator with the usage denominator from the production infrastructure so that rates and any Article 88 threshold check can be computed.
• Underreporting assessment: The physician questionnaire in the legacy RWE study is the mechanism by which underreporting from the legacy device's user base is bounded. The registry sets the upper bound at zero reported serious incidents; the study tests whether that upper bound holds when physicians are asked directly.

Source

• Jira project board: https://legithealth.atlassian.net/jira/software/projects/R006001/boards/18
• Data captured: 2026-04-18 (registry entries R006001-1 through R006001-105).
• Extraction method: JQL (project = R006001 ORDER BY created DESC) through the Atlassian MCP, two pages of 50 issues, followed by full-description retrieval for each Category 3 event and for Category 5 surveillance-signal events.