Per-row audit — which sources tick "Safety confirmation" and why
STATUS (2026-04-20): Complete audit against the fixed definition in
proposed-definition.md. Every row in the<EvidenceRankMatrix>rendering (data fileclinicalEvidenceRankMap.ts) and every row in the two CEP tables that mention "Safety confirmation" has been walked. Flipped ticks are few and explicitly justified. Ambiguous cases keep their existing tick state and are flagged as open questions to be raised post-submission.
Methodology
For each evidence source in the portfolio I asked the two questions below, in order.
- Does the source pre-specify safety-relevant outcome collection? (adverse events; device-related harm; usability-related incidents; residual-risk observations — any of these, pre-declared in a protocol / data-collection plan / SOP, not improvised after the fact)
- Does the source report those outcomes with denominators? (count / rate / upper-bound statement, not just a narrative "no events observed" with no denominator)
A tick on "Safety confirmation" requires both. A clinical study can still contribute to Pillar 3 Clinical Performance via its effectiveness endpoints without ticking Safety confirmation if its protocol had no safety section — the two axes are orthogonal.
Row-by-row — data file clinicalEvidenceRankMap.ts
The data file is the source of truth for the <EvidenceRankMatrix> summary cross-tab rendered in the CEP. One row per (source × assignment) — multi-rank sources contribute multiple rows.
| # | Source | Existing Pillar assignment(s) | Safety-confirmation tick after audit | Justification |
|---|---|---|---|---|
| 1 | MC_EVCDAO_2019 | 3, 3 | No (unchanged) | CIP pre-specified AUC / sensitivity / specificity endpoints only; no Section-F-style safety sub-section. Zero-event safety observation is reported in the CER's Safety Benchmarking section but is narrative — denominator only at the portfolio level, not the study level. Pillar 3 primary stands; no Safety tick. |
| 2 | COVIDX_EVCDAO_2022 | 3 | No (unchanged) | CUS / DUQ / SUS questionnaires are performance and usability perception; CIP had no pre-specified safety endpoints. Portfolio-level zero-event reporting covers this study. Pillar 3 primary unchanged. |
| 3 | DAO_Derivación_O_2022 | 3, 3 | No (unchanged) | Referral-adequacy CIP — performance-only endpoints. No Section-F-style safety questionnaire. Pillar 3 primary unchanged. |
| 4 | IDEI_2023 | 3 (Rank 2), 3 (Rank 4) | No (unchanged) | Pigmented-lesion / androgenic-alopecia CIP — diagnostic-accuracy and Ludwig-severity endpoints only. Pillar 3 primary unchanged across both rank assignments. |
| 5 | DAO_Derivación_PH_2022 | 3, 3 | No (unchanged) | Referral-optimisation CIP — performance-only endpoints; protocol deviation affected primary endpoint. No pre-specified safety section. Pillar 3 primary unchanged. |
| 6 | AIHS4 2025 pivotal (severity) | 3 | No (unchanged) | Severity-assessment proof-of-concept (N=2 patients, 16 assessments). CIP scoped to severity agreement / temporal consistency. No pre-specified safety section. Pillar 3 supporting unchanged. |
| 7 | BI_2024 (MRMC) | 3 (Rank 11), 3 (Rank 11) | No (unchanged) | Simulated-use reader study on retrospective images. MDR Article 2(45) definition of clinical investigation does not apply — no live-patient safety surveillance is possible in this design. Rank 11 Pillar 3 §4.4 supporting unchanged. |
| 8 | PH_2024 (MRMC) | 3 (Rank 11) | No (unchanged) | Same as BI_2024. Rank 11 Pillar 3 §4.4 supporting unchanged. |
| 9 | SAN_2024 (MRMC) | 3 (Rank 11) | No (unchanged) | Same as BI_2024. Rank 11 Pillar 3 §4.4 supporting unchanged. |
| 10 | MAN_2025 (MRMC, Fitzpatrick V–VI) | 3 (Rank 11) | No (unchanged) | Same as BI_2024. Rank 11 Pillar 3 §4.4 supporting unchanged. |
| 11 | R-TF-015-011 — Systematic State of the Art review | 1 | No (unchanged) | VCA anchor literature review; no patient-level safety observation. Pillar 1 primary unchanged. |
| 12 | Legacy predecessor passive PMS corpus (2020–present) | Safety (Rank 7) | Yes — unchanged | Passive surveillance pre-specified in the legacy-device PMS Plan; outcomes (0 MDR Art. 87 incidents, 0 FSCAs, 0 Art. 88 trend reports triggered, 7 non-serious complaints) reported with the ≈ 250 000-report denominator in R-TF-007-003. Canonical Safety-confirmation source. Pillar assignment kept at Safety. |
| 13 | R-TF-015-012 — cross-sectional observational study (legacy device) | 3 (Rank 4), 3 (Rank 8) | Yes — tick added | Change. The protocol pre-specifies Section F safety questions F1–F4 (misleading output observation; usability issues affecting clinical use; overall perceived safety Likert 1–5; formal adverse-event logging) with denominator N = 56 in the analysis set. F1 = 26.8 % (below 30 % follow-up threshold), F3 mean = 4.14 / 5, F4 = 7.1 % cross-referenced against R-006-002. This satisfies both criteria of the Safety-confirmation definition. A third assignment pillar: "Safety" is added to the data file; existing Rank 4 Pillar 3 and Rank 8 Pillar 3 assignments are unchanged. See CLAUDE.md §Strategy alignment check #3. |
| 14 | Equivalence with legacy device (MDCG 2020-5) | 3 (Rank 5) | No (unchanged) | Equivalence assessment is a regulatory-methodological step, not an evidence source with its own safety-data collection. It transports the legacy passive PMS corpus into scope (row 12), which is where the safety-confirmation contribution is accounted for. Pillar 3 supporting unchanged. |
| 15 | APASI_2025 / AUAS_2023 / AIHS4_2023 / ASCORAD_2022 | 2, 2, 2, 2 | No (unchanged) | Peer-reviewed severity-algorithm validation publications. Each is a retrospective non-comparative algorithm-vs-consensus study; MINORS-appraised. No patient-level safety observation — these are algorithm-output-accuracy studies. Pillar 2 primary unchanged across all four. |
| 16 | NMSC_2025 | 3 (Rank 4) | No (unchanged) | Retrospective observational BCC / cSCC detection in a specialist head-and-neck clinic. Performance-only endpoints. No pre-specified safety-data collection within this publication. Portfolio-level zero-event reporting covers it. Pillar 3 supporting unchanged. |
| 17–24 | Planned PMCF investigations (triaje_VH_2025, CVCSD_VC_2402, clinical_VH_2025, AFF_EVCDAO_2021, acne, aEASI_HVN, AGM_2026, PMCF-ICD-DXP-2026) | 3 (Rank 4) each | No (unchanged at this revision) | These are planned investigations whose CIPs are still in preparation. Open question to raise post-submission: the PMCF CIPs should include pre-specified safety sections analogous to R-TF-015-012 Section F — if they do, the Safety-confirmation tick will be added when the CIPs are finalised. At the current CEP revision, none of the eight has a published CIP with a safety section, so the tick is left empty. Action: task-3b9 (legacy-PMS-conclusions-into-plus-PMS-plan) owner to confirm that R-TF-007-002 PMCF Activities A–F each carry a pre-specified safety section, and that each executed PMCF CIP inherits that section. Flag, not fix-today. |
Count of safety-confirmation-ticked rows after audit: 2 (Legacy passive PMS corpus Rank 7, and R-TF-015-012 newly ticked). This is consistent with the CLAUDE.md alignment checks #3 (R-TF-015-012 should be ticked) and #4 (R-TF-007-003 / the legacy PMS corpus should be ticked — covered by row 12 above since R-TF-007-003 is the umbrella report that aggregates the passive corpus).
Row-by-row — CEP §Planned evidence classification per study
The canonical source of truth is the data file above. The CEP markdown table is a narrative mirror; it should be kept consistent. Current state and audit result:
| # | Table row | Current "MDCG 2020-1 Pillar" cell | After fix |
|---|---|---|---|
| a | MC_EVCDAO_2019 | Clinical Performance | Clinical Performance (unchanged) |
| b | COVIDX_EVCDAO_2022 | Clinical Performance | Clinical Performance (unchanged) |
| c | DAO_Derivación_O_2022 | Clinical Performance | Clinical Performance (unchanged) |
| d | IDEI_2023 | Clinical Performance | Clinical Performance (unchanged) |
| e | DAO_Derivación_PH_2022 | Clinical Performance | Clinical Performance (unchanged) |
| f | BI_2024 / PH_2024 / SAN_2024 / MAN_2025 | Clinical Performance (MDCG 2020-1 §4.4; supporting Pillar 3 at Rank 11) | Clinical Performance (MDCG 2020-1 §4.4; supporting Pillar 3 at Rank 11) (unchanged) |
| g | AIHS4_2025 | Clinical Performance (limited) | Clinical Performance (limited) (unchanged) |
| h | NMSC_2025 (both rows) | Clinical Performance (Pillar 3 Tier 1) / Technical Performance (Pillar 2) | (unchanged for both rows) |
| i | Legacy PMS data (passive) | Safety confirmation | Safety confirmation (per MDCG 2020-6 §6.1 and MDR Annex I §§1, 8, 14; see §Definitions) — add parenthetical anchor, keep label |
| j | R-TF-015-012 (legacy device post-market observational) | Clinical Performance | Clinical Performance; also contributes Safety confirmation via Section F (F1–F4) pre-specified safety items (see §Definitions) — add Safety leg |
| k | APASI_2025 / AUAS_2023 / AIHS4_2023 / ASCORAD_2022 | Technical Performance (Pillar 2) | Technical Performance (Pillar 2) (unchanged) |
Row-by-row — CEP §Evidence hierarchy (Rank 1 – 12 narrative table)
| Rank | Current "MDCG 2020-1 Pillar the rank populates" cell | After fix |
|---|---|---|
| 2 | Pillar 3 Clinical Performance (primary) | (unchanged) |
| 4 | Pillar 3 Clinical Performance (primary, with methodological limitations quantifiable) | (unchanged) |
| 5 | Pillar 3 Clinical Performance (equivalence-derived) | (unchanged) |
| 6 | Pillar 1 Valid Clinical Association and Pillar 2 Technical Performance | (unchanged) |
| 7 | Pillar 3 Clinical Performance (safety confirmation via legacy PMS) | Safety confirmation (MDCG 2020-6 §6.1; MDR Annex I §§1, 8, 14), orthogonal to the three MDCG 2020-1 MDSW pillars; see §Definitions "Safety confirmation" — removes the Pillar-3-sub-role conflation |
| 8 | Pillar 3 Clinical Performance (supporting; Likert professional-opinion) | (unchanged) |
| 11 | Pillar 3 Clinical Performance (MDCG 2020-1 §4.4 supporting) | (unchanged) |
| 12 | Pillar 2 Technical Performance | (unchanged) |
Row-by-row — CER
The CER's relevant mentions of "safety confirmation" are two narrative sentences, not ticked table rows. Verifying consistency only:
| CER location | Current sentence | Action |
|---|---|---|
| §Route B | "real-world safety confirmation" | Unchanged — narrative usage is correct once the definition is established. |
| §Assessment of the combined evidence portfolio against MDCG 2020-1 pillars | "The post-market Rank 7 vigilance and curated QMS data from the equivalent legacy device supports Pillar 3 safety conclusions…" | Small edit needed. Replace with "The post-market Rank 7 vigilance and curated QMS data from the equivalent legacy device contributes Safety-confirmation evidence (see §Definitions), complementing the three-pillar conclusions set out above". Removes Pillar-3-sub-role framing. |
| §Assessment … (bullet "Safety confirmation from real-world deployment") | Existing wording | Add a parenthetical cross-reference: "(Safety confirmation — see §Definitions)". No content change. |
Open questions to raise post-submission
- PMCF CIP safety sections. Confirm that
R-TF-007-002Activities A.1–A.3, B.1–B.5, C.1–C.2, D.1–D.2, E._ and F._ each include pre-specified safety-data collection; currently the eight planned PMCF investigations in the data file are not ticked for Safety confirmation because their CIPs are in preparation. Feeds task-3b9. - Cybersecurity / clinical-safety routing. Confirm GP-013 complaint handling + GP-030 cybersecurity incident handling correctly route cybersecurity incidents with clinical-safety consequences into the safety-confirmation stream. Low priority — no known incident to date.
- Terminology alignment with
GP-015 T-015-003template. The GP-015 template mentions "Clinical safety confirmation by clinical data (included PMCF)" — this is consistent with the new definition but the wording slightly predates it. Consider a small template edit at the next GP-015 revision to cite the CEP definition by name.