Research and planning

Internal working document

This document is for internal use only. It contains analysis, gap identification, and response strategy for Item 6b of the BSI Clinical Review Round 1. It will not be included in the final response to BSI.

1. What BSI is asking

BSI requests:

"Please provide a justification that the planned activities will collect sufficient data on safety and performance of the device throughout its lifetime."

The index-level observation adds:

"Overall justification that the PMCF plan is expected to collect sufficient data in quality and quantity to confirm the benefit, safety, and performance of the device over its lifetime based on all activities is not found."

This is a deficiency finding. BSI wants a consolidated argument — not just a list of activities, but an overarching justification that the totality of PMCF activities (general + specific) will produce enough data, in sufficient quality, to cover the device's entire expected lifetime.

2. What "data sufficiency throughout its lifetime" means

Annex XIV 5 requires PMCF to be a "continuous process" that confirms safety and performance "throughout the expected lifetime of the device." For software, the device's lifetime is typically defined by the period during which the manufacturer supports it — this should be stated in the technical documentation.

The data sufficiency argument must demonstrate:

1. Quality: Activities use appropriate methodologies (validated study designs, relevant endpoints, recognised standards)
2. Quantity: Sample sizes are statistically adequate; the total evidence base covers all claims
3. Coverage: Activities cover all intended uses, all user populations, all identified risks, and all performance claims
4. Continuity: The plan does not just collect data once — it describes how data collection continues throughout the device's lifetime (annual cycles, periodic monitoring)
5. Integration: PMCF data feeds back into the clinical evaluation (CER updates) and risk management

3. Gap analysis: why the current plan fails this requirement

The PMCF Plan describes 9 specific activities and 4 general methods. However:

Sufficiency criterion	Status in current PMCF Plan
Quality of methodology	Partial — study types are stated but methodology is not detailed enough to assess quality
Quantity (sample sizes)	Insufficient — only A.1 states a sample size (30,000 images); 8 of 9 activities have no sample size
Coverage of claims	Partial — CER gaps 1–3 are addressed but not all performance claims are explicitly mapped to activities
Coverage of risks	Not demonstrated — no explicit mapping of identified risks to PMCF activities that will monitor them
Continuity	Partial — the plan says PMCF is "continuous" and reports annually, but specific activities are one-off studies with start dates in 2026; no description of what happens after initial studies complete
Integration with CER	Partial — the plan states results will be in the PSUR, but the mechanism for CER update is not explicit
Overall sufficiency statement	Missing — no consolidated argument that collectively, these activities produce sufficient evidence

The "lifetime" problem

All 9 specific activities have start dates in 2026 and no end dates. Once these studies complete (presumably 2026–2027), the PMCF Plan does not describe what happens next:

• Are the studies repeated?
• Are new studies designed based on results?
• How is ongoing performance monitoring maintained beyond the initial studies?

The general methods (routine PMS data collection) provide continuity, but the PMCF Plan does not explicitly state that these general methods, combined with periodic specific studies, constitute the lifetime monitoring strategy.

The coverage problem

The PMCF Plan maps activities to three CER gaps, but does not map to:

• All identified risks in the risk management record (R-TF-013-002)
• All clinical performance claims in the IFU/CER
• All user populations (HCPs and ITPs — all specific activities focus on HCPs)
• Safety specifically (activities focus on performance endpoints; safety monitoring relies on general PMS methods, but this is not stated)

4. Equivalent/similar device evaluation

BSI notes: "An evaluation of the clinical data relating to the equivalent or similar devices is not found."

Annex XIV 6.2 requires the PMCF Plan to include an evaluation of clinical data relating to equivalent or similar devices. The current PMCF Plan has no section addressing this.

This section needs:

1. Identification of equivalent or similar devices (AI-based dermatology diagnostic support tools)
2. Summary of available clinical data for those devices (published studies, vigilance data)
3. Assessment of how that data informs the PMCF strategy (e.g., what risks have been identified in similar devices that our PMCF should monitor?)

For a novel AI medical device, equivalent device data may be limited. But the section must exist and explain the search methodology and findings.

5. Cross-NC connections

Clinical Review Item 6a — Activity descriptions

Item 6a research addresses the detail of each individual activity. Item 6b builds on top of 6a — even if each activity is described in detail, the sufficiency argument requires showing how the activities work collectively. The response to 6b should reference the expanded activity descriptions from 6a.

Clinical Review Item 3b — Data sufficiency justification

Item 3b research raises the same "data sufficiency" concern at the CER level. The PMCF Plan's data sufficiency justification should be consistent with and complementary to the CER's data sufficiency argument. Where 3b identifies evidence gaps, the PMCF Plan should show how those gaps will be filled by PMCF activities.

Clinical Review Item 3a — Clinical data analysis

Item 3a research identified that the CER does not integrate legacy PMS data. The PMCF data sufficiency argument should reference the existing PMS evidence base (7 non-serious incidents over 4+ years of market experience with the legacy device) as part of the baseline safety profile that PMCF will continue to monitor.

Clinical Review Item 5 — PMS Plan

The general PMCF methods draw data from the PMS system. The PMS Plan (R-TF-007-001) describes the data collection infrastructure. The data sufficiency argument should reference the PMS Plan to show how routine data collection supplements specific PMCF studies.

6. Response strategy

The response should:

1. Provide the overall data sufficiency statement that BSI could not find — a consolidated argument that collectively, the general methods plus 9 specific activities produce sufficient evidence in quality and quantity
2. Map PMCF activities to the complete evidence needs — not just CER gaps, but all performance claims, identified risks, and user populations
3. Address continuity — explain the lifecycle of PMCF: initial targeted studies → results integration → updated gap analysis → next cycle of studies
4. Address quality — reference methodological standards (e.g., STARD for diagnostic accuracy, ISO 14155 for clinical investigations)
5. Provide the equivalent/similar device evaluation
6. Confirm that PMCF results feed into CER and risk management updates

Fixes required (in R-TF-007-002)

Fix 1: Add a "Data Sufficiency Justification" section

Add a section early in the PMCF Plan (after Rationale and Objectives) that consolidates:

• How the general + specific methods collectively cover all evidence needs
• How quality is ensured (methodological standards)
• How quantity is ensured (sample size adequacy across the portfolio)
• How coverage is achieved (mapping to claims, risks, populations)
• How continuity is maintained throughout the device lifetime

Fix 2: Add a PMCF lifecycle / continuity section

Explain that PMCF operates in annual cycles:

1. Execute planned activities
2. Analyse results and integrate into CER/PSUR
3. Reassess CER gaps and risks based on new data
4. Design next cycle of activities if gaps remain
5. Repeat

This demonstrates that the plan covers the device's lifetime, not just the initial post-market period.

Fix 3: Add an "Equivalent/Similar Device Evaluation" section

Per Annex XIV 6.2, include:

• Identification of comparable AI dermatology devices
• Summary of their clinical data
• How this data informs the PMCF strategy

Fix 4: Add a coverage mapping

Create an explicit mapping showing:

• Each performance claim → which PMCF activity monitors it
• Each identified risk → which PMCF activity monitors it
• Each user population → which PMCF activity covers it

This directly addresses BSI's concern that there's no overall justification the plan is comprehensive.

7. Risk assessment

Risk	Impact	Mitigation
BSI may find the sufficiency argument unconvincing without sample size data	High — sample sizes are critical for the "quantity" argument	Obtain sample size calculations for all 9 activities (depends on 6a open items)
BSI may question "lifetime" coverage since all activities are one-off studies in 2026	Medium — BSI may ask what happens after initial studies complete	Add lifecycle/continuity section describing annual PMCF cycles
BSI may note that safety monitoring is under-represented in specific activities	Medium — most activities focus on performance endpoints, not safety	Emphasise that general PMCF methods (vigilance, PMS data) provide continuous safety monitoring, and that specific activities include safety endpoints where relevant
Equivalent device evaluation may be thin for a novel AI device	Low — BSI understands novel devices may have limited comparators	Conduct the search, document methodology, and explain the limited findings
ITP user population not covered by any specific PMCF activity	Low — ITP users interact via API only; clinical risk is indirect	Explain in coverage mapping that ITP usability is monitored via PMS (complaints, feedback) and C.1/C.2 activities evaluate API output quality

8. Open items

#	Item	Owner	Status
1	All open items from 6a (sample sizes, acceptance criteria, completion dates) are prerequisites for the data sufficiency argument	Jordi	Required
2	Define the device's expected lifetime (years) — needed for the "throughout its lifetime" argument	Jordi	Required
3	Map all identified risks from R-TF-013-002 to PMCF activities that monitor them	Jordi	Required
4	Map all performance claims from IFU/CER to PMCF activities that confirm them	Jordi	Required
5	Search for equivalent/similar AI dermatology devices and their published clinical data	Jordi	Required
6	Confirm the PMCF annual review cycle is documented in the QMS (e.g., in GP-007 or the PMS Plan)	Jordi	Required