PH_2024 adequacy-review fixes (2026-04-19)

Internal record of the fixes applied to the PH_2024 investigation folder in response to the quadruple adequacy review (bsi-clinical-auditor, audit-deliverable-reviewer, markdown-style, celine-clinical-consultant) run via /review-clinical-investigation on 2026-04-19. Companion to san-2024-fixes.md and man-2025-fixes.md; this note records only what differed from or was additional to those prior passes.

Scope

• Folder: apps/qms/docs/legit-health-plus-version-1-1-0-0/product-verification-and-validation/clinical/Investigation/ph-2024/
• Files edited:
  • r-tf-015-004.mdx (CIP)
  • r-tf-015-006.mdx (CIR)
  • r-tf-015-010.mdx (Annex E — full rewrite)
• Other files edited: None outside the investigation folder in this pass. The clinicalStudiesData.ts protocolDate value (2024-07-04) was not modified; the CIP was signed in mid-2024 and the date is consistent with the investigation-start window.
• Build: npx turbo run build --filter=qms — passes.

Reused from SAN_2024 / MAN_2025 without adaptation

The following session-level decisions were applied verbatim to PH_2024; refer to san-2024-fixes.md and man-2025-fixes.md for the rationale.

• Q1 (MRMC Rank-11 reframe, full). CIP Scope §Nature and positioning, CIR Research Title, CIR Summary Title, CIR Summary Introduction, CIR main Introduction, CIR Discussion §Regulatory positioning, CIR Implications for Future Research and CIR Limitations all rewritten as simulated-use Rank-11 supporting Pillar 3 §4.4 evidence. Real-world patient-outcome, triage, teledermatology, time-to-treatment and healthcare-economics framing removed from CIR Conclusions and relocated to a bounded "Potential clinical-system implications (inferential)" paragraph with explicit external-literature anchoring.
• Q2 (threshold justification, cite R-TF-015-011). New §Justification of acceptance thresholds subsection added to the CIP immediately after <AcceptanceCriteriaTable studyCode="PH_2024" showUserGroup={true} />, explaining derivation of the absolute-value and relative-change thresholds from R-TF-015-011 State of the Art and naming the primary confirmatory endpoint (≥ 10 pp absolute improvement in pooled top-1 accuracy). Secondary thresholds pre-specified as supporting; stratum-level comparisons pre-specified as exploratory / hypothesis-generating.
• Q3 (per-pathology / per-specialty multiplicity, exploratory). Applied across CIP §Statistical analysis and CIR §Analysis. P-values in the per-pathology table in the CIR are now labelled "p-value (exploratory)"; the per-pathology table header explicitly states these analyses are exploratory / hypothesis-generating.
• Q4 (device version, identity bridge). v1.1.0.0, only version released, identity bridge. New §"Device version under investigation and bridging to the CE-marked release" subsection added to both CIP §Product Identification and Description and CIR §Product Identification, with PRRC sign-off.
• Q6 (ethics committee, sponsor determination-of-non-applicability). Annex E rewritten to use the <EthicsCommitteeNonApplicability /> reusable component introduced in SAN_2024. New section ## Ethics Committee Non-Applicability Determination added at the end of Annex E. CIP §Ethical considerations and CIP §Informed Consent Process cross-reference this section; CIR §Ethical Aspects of Clinical Research rewritten to cross-reference it.
• Q7 (Investigator's Brochure, IFU v1.1.0.0). Annex E IB row flipped from FALSE to TRUE, with the IFU cited as the investigational-device orientation document.
• Q8 (product-name anonymisation, introduce-once convention). CIR title retains the brand name once ("Legit.Health Plus (hereinafter 'the device')"); CIP and CIR both add "Throughout this document, references to 'the device' refer to the investigational product identified above" immediately after <DeviceCharacterisation />. Brand-name uses elsewhere in CIP and CIR body prose replaced with "the device".
• Q9 (pilot → investigation). Every occurrence of "pilot study", "pilot subjects", "pilot findings" across CIP and CIR replaced with "investigation", "reader-participants", "investigation findings". PH_2024 is an MRMC simulated-use supporting Pillar 3 §4.4 investigation at Rank 11; it is not a pilot and is not confirmatory Pillar 3 primary evidence (the Pillar 3 primary evidence is supplied by Rank 2–4 prospective real-patient investigations).

Decisions specific to PH_2024 (not in SAN_2024 / MAN_2025)

A — Headline accuracy reconciliation (critical, file-internal contradiction)

The CIR prior to this pass reported two incompatible headline figures in parallel: 63.70% → 81.85% (+18.15 pp) in §Summary and §Conclusions, and 72.96% → 82.22% (+9.26 pp, framed as "a 13% relative increase") in §Analysis and in the Discussion Conclusions. The supporting JSON (sens_spec_performance_PH.json) confirms the pooled per-observation figures 63.70% → 81.85%. The 72.96% / 82.22% figures appear to originate from a per-reader-averaged pooling that was not the pre-specified primary analysis.

Decision: reconcile to the JSON-backed pooled per-observation figures 63.70% → 81.85% (+18.15 pp; Wilson 95% CIs; McNemar p < 0.001) as the primary endpoint. The per-reader-averaged figures were removed from the CIR; §Primary Analysis — confirmatory now reports a single coherent set of headline numbers, with Wilson CIs declared and Newcombe CIs for the paired difference.

B — Per-pathology narrowing and honest "no confirmatory malignancy benefit" statement

The prior CIR Conclusions stated "Legit.Health significantly enhanced primary care physicians' diagnostic accuracy" and cited hidradenitis suppurativa, urticaria and actinic keratosis as "significant improvements", without stating that per-pathology p-values did not reach statistical significance for melanoma (p=0.69), basal cell carcinoma (accuracy decreased 91.67% → 88.89%, p=1.00), plaque psoriasis (no change, p=1.00), nevus (p=1.00), actinic keratosis (p=0.125) or pustular psoriasis (p=0.25).

Decision: rewrite §Conclusions and §Discussion and Overall Conclusions → Conclusions to state explicitly that (i) overall aggregate improvement was statistically significant on the pre-specified primary endpoint, (ii) per-pathology analyses were exploratory and under-powered, (iii) per-pathology p-values did not reach statistical significance for melanoma, basal cell carcinoma, plaque psoriasis, nevus, actinic keratosis or pustular psoriasis, and (iv) no malignancy-specific benefit claim is made on the basis of this investigation alone. Per-pathology table recast with p-value (exploratory) label and the exploratory framing restated above the table. Pustular-psoriasis "5.56% → 22.22%" callout removed from §Results summary.

C — Image-Case Demographics / Reader-Participant Characteristics split

The prior CIR §Subject Demographics conflated the 30 image cases (age, gender, Fitzpatrick phototype) with the 9 reader-participants under a single "Subject Demographics" heading, which was both factually wrong (ISO 14155 "subject" = study subject, which in this MRMC is the reader-participant) and regulatorily misleading (the Fitzpatrick and age distributions describe the image set, not the readers).

Decision: split into two explicit subsections. §Image-Case Demographics (n = 30) describes the depicted image subjects (patient cases sourced from public atlases) with age, gender and Fitzpatrick tables; an explicit in-section statement flags Fitzpatrick V and VI absence and paediatric 2–12y / 12–21y absence, and cross-references R-TF-015-003 and R-TF-007-002 for the coverage-gap mitigation. §Reader-Participant Characteristics (n = 9) describes the nine primary-care physicians by qualification and affiliation, under pseudonymised codes 202407-LR-001 to 202407-LR-009.

D — Paediatric subgroup reformulation (n = 1 non-evaluable for confirmatory purposes)

The prior CIR paediatric subgroup analysis reported sensitivity and specificity for single-positive-case subgroups; for n = 1 there is no negative class and "specificity" does not carry independent statistical meaning.

Decision: rewrite §Paediatric age-band subgroup to explicitly flag the subgroup as exploratory and non-evaluable for confirmatory purposes; remove the sensitivity/specificity columns for single-class subgroups; retain per-reader correct-identification counts with an exact-CI framing; state that paediatric coverage is addressed at Clinical Evaluation level (R-TF-015-003) and by the PMCF Plan (R-TF-007-002). The prior discrepancy between the demographics table (0 cases in the child band 2–12y) and the analysis text (claiming child-band urticaria data) is resolved by the subgroup reformulation — no confirmatory data is reported for bands the demographics table indicates are absent.

E — 3.70% "switch to incorrect" proportion traced to risk file

The prior CIR decomposition reported a 3.70% proportion of device-aided reads that switched a correct unaided decision to an incorrect one, without any trace to the risk management record.

Decision: re-label the decomposition as exploratory / hypothesis-generating (not a pre-specified endpoint); state explicitly that the 3.70% proportion is a known automation-bias direction; trace it to R-TF-013-002 Risk Management Record, to the existing risk controls (IFU warnings on the non-binding nature of the device output, requirement that the clinician remains the decision-maker, Top-5 prioritised differential rather than single binding answer), and to PMCF monitoring under R-TF-007-002.

F — Pseudonymised reader-participants and un-pseudonymised mapping

The prior CIP and CIR named only 4 of the 9 reader-participants (Principal Investigator + 3 collaborators); participants 5–9 were not identified anywhere in audit-visible prose.

Decision: retain the named four in both CIP and CIR, and add an explicit line identifying reader-participants 5–9 under their pseudonymised codes 202407-LR-004 through 202407-LR-009; state that the un-pseudonymised mapping is retained in the investigator identification-code list under the custody of the Principal Investigator and is available on audit request. This follows ISO 14155 §8.2.3 investigator identification without breaking the pseudonymisation used in the locked analysis dataset.

G — Duration deviation and CIP compliance log

The prior CIR §CIP Compliance asserted compliance in a single sentence with no deviation log; §Duration reported 3 months while CIP §Duration and §Calendar stated 4 months.

Decision: expand §Clinical Investigation Plan (CIP) Compliance in the CIR to include a deviations table documenting (i) the approximately one-month shorter actual duration (4 June – 13 September 2024) vs the four-month plan, and (ii) the reader cohort size of 9 vs the "minimum of 8" early-CIP language (9 is the pre-specified target used for powering). Both deviations are documented as minor with no impact on analysis population, endpoints or conclusions; no CAPA is triggered.

H — <StudyMetricsByTaskTable /> duplicate-key fix

The prior CIR passed a taskLabels prop containing two identical keys "Multiple conditions" mapped to two different values, silently overriding one. The renderer lost one mapping and the acceptance-criteria comparison table was incomplete.

Decision: remove the malformed prop entirely and render <StudyMetricsByTaskTable /> bare, letting the component fall back to its defaults (same pattern applied in SAN_2024 Batch 2). The explicit acceptance-criteria pass/fail call in the CIR is carried by <AcceptanceCriteriaTable displayMode="table" showAchievedValues={true} showStateOfArt={true} showUserGroup={true} showValueType={true} /> under §Clinical Performance, Efficacy, and Safety → Acceptance-criteria pass/fail summary.

I — Annex E full rewrite vs surgical-edit pattern

For SAN_2024 and MAN_2025 the Annex E pass was row-by-row surgical edits. For PH_2024 the prior Annex E had so many provenance contradictions (seven rows justified non-applicability as "retrospective analysis of routine care" while the CIP says prospective MRMC on public atlases, plus the IB/EC/RA / ICF internal contradiction) that the cleaner path was a full rewrite. The rewrite keeps the same four-column structure (Document Title, Purpose, Included, Justification) and populates every cell with regulatory-level justifications consistent with CIP and CIR. Header updated to the harmonised edition "UNE-EN ISO 14155:2021".

What was applied — by batch

Batch 1 — Formatting sweep

• CIP §Completion (typo Completition → Completion).
• CIP §Suspension or early termination of clinical research: # → ## (heading-level skip).
• CIP §Data management subheadings (Data Collection, Access Control and Traceability, Data Anonymity, Data Export and Analysis, Data Security): bold-as-heading + colon-inside-bold + hard line break → proper #### headings with blank lines.
• CIP §Informed Consent process subheadings (For Patients/Image Subjects, For Healthcare Practitioners): bold-as-heading + colon-inside-bold → proper ### headings.
• CIP Sample size subheadings (Justification for 270 Observations, Selection of 8-9 Primary Care Practitioners, Selection of 30 Images per Practitioner): bold-as-heading → proper #### headings; colon-inside-bold in the four bullet labels under Justification rewritten with colon outside bold and sentence-case.
• CIP §Statistical analysis: hard line break mid-paragraph → blank line.
• CIR duplicate ### Sample size heading removed.
• CIR typo critieria → criteria.
• CIR hard line breaks between paragraphs in §Design, §Potential clinical-system implications, §Image-Case Demographics, §Analysis repaired.
• CIR **Derived from performanceClaims.ts (for comparison):** removed; replaced with #### Performance claims cross-reference heading over a bare <StudyMetricsByTaskTable />.
• CIR initiation/completion dates l.259-260 converted from two dangling date lines to a labelled list.
• CIR **Summary of Performance Claims:** bold-as-heading → #### Acceptance-criteria pass/fail summary with explanatory introduction.
• CIP / CIR / Annex E trailing Hangul filler ㅤ after <Signature /> removed.

Batch 2 — Leak rewording (engineering nouns + anonymisation)

• CIP T-015-006 / T-013-002 → R-TF-015-006 / R-TF-013-002 (3 locations).
• CIP / CIR "AI Labs Group S.L." → "Technical Support (Manufacturer)" / "the manufacturer" (4 locations).
• CIP / CIR brand-name uses in body prose ("Legit.Health Plus", "Legit.Health") → "the device" (with single introduce-once exception in CIR title per Q8).
• CIP Collaborating Investigators spelling "Dra. Esther Minguela" standardised to "Dr. Esther Minguela" in English document.
• CIR :::info sponsor admonition → plain prose block ("Sponsor institution: …").
• CIR "We developed a website for conducting the experiment" → "A centralised, access-controlled web-based platform served as the electronic Case Report Form (eCRF) for image presentation and data capture."
• CIR "The analysis was conducted using the Python programming language and statistical libraries such as numpy and pandas" → "The pre-specified statistical analysis was implemented in a deterministic, version-controlled analytics environment maintained by the manufacturer; statistical tests (including McNemar's test for the primary endpoint) and confidence intervals were computed as specified in the CIP §Statistical analysis."
• CIR §Duration "creation of the website" → removed; duration narrative rewritten regulatorily with a documented variance against the CIP.
• CIR §Design — Type of research "data collection through websites or photograph analysis" → "captured through a secure, access-controlled web-based eCRF platform".
• CIR ICD-11 inconsistency: "XH4L78 Pigmented Nevus" repositioned as an explicit ICD-11 extension-code reference alongside the 2F20 / 2F20.1 stem codes used elsewhere.
• CIR "The investigational products were stored and handled following strict protocols. This included proper storage conditions, handling procedures, and documentation of product usage" → SaMD-appropriate prose describing version provisioning, access control and session logging.
• CIR "The original signed informed consent form has been securely stored …" boilerplate paragraph removed from §Ethical Aspects (direct contradiction of CIP and Annex E ICF rows).
• CIR / CIP §Center / §Centre: clarified the relationship between the sponsor institution (IIS Puerta de Hierro, Majadahonda) and the reader-participant affiliation (Las Rozas Healthcare Centre).
• Annex E row Investigational Device Accountability Records: "Legit.Health annotation platform" → "validated, access-controlled web-based platform provided by the manufacturer".
• Annex E edition reference: "ISO 14155:2020" header and justifications updated to "UNE-EN ISO 14155:2021".

Batch 3 — Clinical Minor

• CIP / CIR "true accuracy" → "top-1 diagnostic accuracy" (5 locations).
• CIP / CIR "pilot" language → "investigation" / "reader-participants" (9 locations total across CIP and CIR — Q9 sweep).
• CIP Inclusion / Exclusion criteria: "high-quality images" subjectivity tightened with "meeting pre-specified technical quality criteria (sufficient resolution, adequate focus, lesion clearly framed)".
• CIR §Reader and Device Management rewritten as SaMD-appropriate prose.
• CIR §Adverse Events and §Product Deficiencies: added explicit detection mechanism (platform-side error logging, session monitoring, per-session feedback field) and cross-reference to GP-009 Post-Market Surveillance.
• CIP §DMC: explicit "No DMC established" statement with justification.
• CIP §Foreseeable adverse events: cross-reference "R-TF-013-002" (previously "T-013-002").
• CIR §Report Annexes: IFU v1.1.0.0 cross-reference added, consistent with Annex E IB row flip.

Batch 4 — Clinical Major

• Item 10 — CI methodology and MRMC clustering. CIR §Results and §Primary Analysis — confirmatory now declare Wilson 95% CIs for each proportion and Newcombe 95% CIs for the paired difference; statistical-methodology prose added in CIP §Statistical analysis and CIR §Statistical Analysis.
• Item 11 — sample-size statistical justification. CIP §Sample size rewritten with formal effect size (≥ 10 pp absolute), expected baseline (~65%), McNemar test basis, α = 0.05, power ≥ 80%. The justification now ties to the pre-specified primary analysis population (per-observation paired set).
• Item 12 — investigator 5–9 pseudonymisation. Both CIP §Collaborating Investigators and CIR §Research Team name the pseudonymised reader-participants 202407-LR-004 through 202407-LR-009 with reference to the investigator identification-code list held by the Principal Investigator.
• Item 13 — paediatric single-case subgroup methodology. CIR §Paediatric subgroup rewritten to report per-reader counts only (no spurious specificity), flag non-evaluable for confirmatory purposes, and defer paediatric coverage to the CER / PMCF Plan.
• Item 14 — pre-specified statistical-analysis plan. CIP §Statistical analysis rewritten to pre-specify primary test (McNemar), primary analysis population (per-observation paired), CI method (Wilson + Newcombe), missing-data rule (complete-case), zero-cell handling, multiplicity policy (no correction, exploratory analyses labelled as such), inter-reader agreement method (Cohen's kappa).
• Item 15 — bias minimisation. CIP §Bias minimisation rewritten to state the self-controlled design, explicit acknowledgement of within-subject recall bias (same image set presented sequentially without washout), acknowledgement of atlas-image curation bias; pre-specified outcomes reaffirmed.
• Item 16 — structural limitations. CIP §Limitations of clinical research and CIR §Limitations of Clinical Research both rewritten as a numbered list of eleven structural limitations (Rank 11 framing, atlas-curation bias, Fitzpatrick V/VI absence, paediatric scope, within-subject recall bias, per-pathology under-power, per-pathology non-significance for malignancies, 3.70% automation-bias proportion, no real-world validation in this investigation, Hawthorne effect, reader-behaviour variability).
• Item 17 — PMCF trace. CIR §Implications for Future Research rewritten to enumerate the questions that are explicitly outside the scope of this Rank-11 simulated-use investigation (real-world primary-care populations, Fitzpatrick V/VI, 2–12y and 12–21y age bands, malignancy-specific performance, real-world referral-pathway outcomes, long-term post-deployment performance); cross-reference to R-TF-015-003 and R-TF-007-002 for the evidence that closes each gap.
• Item 18 — Annex E provenance contradictions. All rows previously justified as "retrospective analysis of routine care" rewritten to match the actual design: prospective, simulated-use MRMC on fully anonymised images from public dermatological atlases.
• Item 19 — Annex E IDD row. Justification rewritten to reference the MDR Annex II Technical File constituents (device description, R-TF-012 series design-and-development records, R-TF-013-002 risk management record, V&V records, cybersecurity documentation, IFU v1.1.0.0); the "training records served the same purpose" category error removed.
• Item 20 — Annex E "available on request" gating. CRF, Signed Agreements, Training Records, Audit Certificates rows rewritten to describe what is retained in the study essential documents and under whose custody; the "available on request" gating phrasing retained only as a statement of custody, not as evidence substitution.
• Item 21 — CIP compliance deviations table. CIR §Clinical Investigation Plan (CIP) Compliance now includes a two-row static deviations table (duration variance, cohort-size wording) with impact assessment and action columns; no CAPA triggered.

Batch 5 — Clinical Critical

• Item 1 — headline accuracy reconciliation (Decision A): headline figures across CIR §Summary, §Results, §Conclusions, §Analysis and §Discussion and Overall Conclusions → Conclusions reconciled to the pooled per-observation figures 63.70% → 81.85% (+18.15 pp; McNemar p < 0.001). The 72.96% / 82.22% / "13% relative increase" framing removed.
• Item 2 — per-pathology narrowing (Decision B): Conclusions now explicitly state that per-pathology p-values did not reach statistical significance for melanoma, basal cell carcinoma, plaque psoriasis, nevus, actinic keratosis or pustular psoriasis; no malignancy-specific benefit claim is made on the basis of this investigation alone.
• Item 3 — Image-Case Demographics / Reader-Participant Characteristics split (Decision C): explicit split into two subsections with explicit Fitzpatrick V/VI and paediatric coverage-gap call-outs and PMCF cross-reference.
• Item 4 — paediatric age-band reconciliation (Decision D): single-case subgroups reframed as non-evaluable for confirmatory purposes; sensitivity/specificity for single-positive-case subgroups removed; internal contradiction between demographics table and analysis text resolved by the reframing.
• Item 5 — device version bridging: CIP and CIR both declare the investigation version as v1.1.0.0 with identity bridge to the CE-marked release, PRRC sign-off.
• Item 6 — static acceptance-criteria pass/fail table: CIR §Clinical Performance, Efficacy, and Safety → Acceptance-criteria pass/fail summary renders <AcceptanceCriteriaTable displayMode="table" showAchievedValues={true} showStateOfArt={true} showUserGroup={true} showValueType={true} /> which calls pass/fail for each pre-specified acceptance criterion.
• Item 7 — document framing: CIP Scope §Nature and positioning of the evidence and CIR Research Title §Nature and positioning of the evidence both state the MRMC Rank 11 + Pillar 3 §4.4 + surrogate-endpoint + VCA-anchoring-via-R-TF-015-011 posture explicitly.
• Item 8 — Annex E / ISO 14155 framing: Annex E preamble added stating applicability and scope (applies only to ph-2024); Rank 11 + Pillar 3 §4.4 positioning; under MDR Article 2(48) not clinical data on real patients; conducted outside the material scope of the Spanish biomedical-research framework. Ethics Committee Non-Applicability Determination section added at the end of Annex E using <EthicsCommitteeNonApplicability />.
• Item 9 — EC / ICF contradiction: CIR §Ethical Aspects boilerplate claiming "original signed informed consent form … stored" removed; CIP §Ethical considerations and §Informed Consent Process both cross-reference the Annex E non-applicability determination; Annex E ICF row retains FALSE with regulatory-level justification.

Follow-ups to track

1. BI_2024 CIP/CIR and MAN_2025 CIP/CIR cross-check. With PH_2024 now reframed to Rank 11 Pillar 3 §4.4 and using the <EthicsCommitteeNonApplicability /> snippet, the only remaining MRMC investigation that may still contain drift is BI_2024. A consistency check pass on BI_2024 (or a fresh /review-clinical-investigation bi-2024 run) is recommended.
2. <StudyMetricsByTaskTable /> defaults. The CIR now renders the component bare, relying on its defaults to display the PH_2024 performance claims. The component defaults should be audited to confirm they select studyId === "PH_2024" and present the expected task-level metric breakdown; if the default fallback shows all studies or a different study, a studyCode="PH_2024" prop should be added.
3. Per-pathology table Wilson CIs. The per-pathology table in CIR §Analysis reports absolute differences but not per-pathology CIs; the CIP §Statistical analysis declares Wilson CIs as the method. Adding per-pathology Wilson CI columns is desirable but was deferred in this pass because the per-pathology analyses are already labelled exploratory; a future pass can emit the CIs from the locked analytics environment and append them to the table.
4. Registry descriptions (NCT07428941, EUPAS1000000644). CIR retains the registry references at face value. The public-facing registry descriptions should be updated to reflect the reframed MRMC simulated-use nature of the investigation (consistent with the CIR now in force) rather than the prior framing; this sits outside the repository and is tracked separately.
5. Ethics Committee Non-Applicability snippet — Spanish / Portuguese translation parity. Unchanged from SAN_2024 and MAN_2025 follow-ups; English-only is currently acceptable for MDR CIP/CIR usage.
6. clinicalStudiesData.ts investigator names. The PH_2024 entry in packages/ui/src/components/ClinicalValidation/clinicalStudiesData.ts may carry investigator-name list content that renders into these documents via <ClinicalStudyMetadataTable />; if so, reconcile the named investigators there to match CIP and CIR (canonical: Dr. Gastón Roustán Gullón; Dr. Esther Minguela; Dr. Fernando León; Dr. Ángel Rossell Díaz; 202407-LR-004 to 202407-LR-009 pseudonymised).

Build verification

npx turbo run build --filter=qms completed successfully after all edits (2026-04-19).