Research and planning

Internal working document

This page is for internal planning only. It will not be included in the final response to BSI.

Analysis

On Q4 of the HCP Knowledge Assessment ("Can the report act as a diagnosis?"), BSI observed that 33% of participants received non-OK scores. After scoring verification during root cause analysis (see sub-item a — HCP-013 reclassified from CC to OK), Q4 non-OK responses total 5/18 (27.8%). BSI questions why this was considered acceptable and whether risks R-TBN and R-CGQ are effectively mitigated.

Detailed breakdown of the 6 non-OK responses:

Participant	Profession	Score	Verbatim answer (Spanish)	Analysis
HCP-003 (Alba Llorens)	Dermatologist	CC	"Es más que un diagnóstico concreto, una serie de diagnósticos diferenciales..."	Initially framed as diagnostic, but describes differential diagnosis support — demonstrates nuanced understanding. Self-corrected by noting clinical information must be considered.
HCP-005 (Mª Carmen Martínez)	Nurse	UD	"Depende de la lesión, la calidad de la foto, la zona a analizar..."	Did not directly answer yes/no. Shows uncertainty about the concept rather than a belief the device diagnoses.
HCP-008 (Maria Pechuan)	Nurse	UE	"Sí"	Only true use error. Unqualified "yes" with no caveats. Did not demonstrate understanding that the device is a support tool.
HCP-013 (Laura Yuste)	GP	CC	"PUEDE AYUDAR MUCHO AL DIAGNÓSTICO, INCLUSO SI SER UN DIAGNÓSTICO POR SI MISMO, UNA ALTA SOSPECHA"	Handwriting is ambiguous between "si" (if) and "sin" (without). Grammatically only "sin" is valid ("incluso sin ser un diagnóstico por sí mismo" = "even without being a diagnosis itself"), suggesting the participant likely understood the device is not diagnostic. However, we should not claim this as definitive evidence of correct understanding — the transcription reads "si" and the answer was scored CC by the evaluator. Treat as a supportive but ambiguous data point.
HCP-015 (Mª Carmen Galindo)	GP	CC	"Sí QUE PUEDE ACTUAR COMO DIAGNÓSTICO, PERO LO MEJOR ORIENTARNOS CON GRAN FIABILIDAD"	Initially says yes, but qualifies it as "orientación" (guidance) with reliability. Mixed understanding.
HCP-017 (Sheila Flores)	Nurse	CC	"Si DEPENDIENDO DE LA CALIDAD Y FORMA DE FOTOGRAFÍA"	Conditional yes focused on image quality rather than the device's role. Does not demonstrate understanding of non-diagnostic intended purpose.

Score summary (after scoring verification, see sub-item a):

• OK: 13/18 (72.2%) — clear correct understanding (HCP-013 reclassified from CC to OK)
• CC: 3/18 (16.7%) — initially incorrect/ambiguous but with qualifications that show partial or full understanding
• UD: 1/18 (5.6%) — uncertain, didn't directly answer
• UE: 1/18 (5.6%) — genuinely incorrect

Key insight: The "33% failure" framing (now 27.8% after scoring verification) conflates fundamentally different categories. By EN 62366-1 definitions:

• Close calls indicate the participant eventually arrived at or demonstrated correct understanding (self-corrected)
• Use difficulties indicate the participant struggled but did not make an error
• Only 1 true use error occurred (5.6%)

However, BSI's concern remains valid: even if self-corrected, the initial confusion suggests the information for safety could be more effective. We must address both the documentation gap AND demonstrate why residual risk is acceptable.

Relevant documents

Document	Path	Key content
R-TF-013-002 Risk Management Record	legit-health-plus-version-1-1-0-0/risk-management/R-TF-013-002.json	Full risk data for R-TBN and R-CGQ (see below)
R-TF-025-007 Summative Evaluation Report	R-TF-025-007-Summative-Evaluation-Report/R-TF-025-007-Summative-Evaluation-Report.mdx	HCP results table, Detailed Scenario Notes, Conclusion
HCP test data (raw)	R-TF-025-007-Summative-Evaluation-Report/2025-10-22-hcp-results/usability_test_responses.json	All 18 verbatim Q4 answers
IFU (EU MDR)	apps/eu-ifu-mdr/versioned_docs/version-1.1.0.0/	Intended purpose statement, precautions, clinical user manual

Risk data:

R-CGQ ("Whole device is wrongly used or is not used as intended"):

• Severity: 3 (initial and controlled); Likelihood: 4 → 1 (after controls)
• Risk control options: A (inherently safe design) + C (information for safety)
• Implemented mitigation: "We specify the product intended purpose in the IFU and label"
• Mitigation requirements: SRS-BYJ, SRS-H3J, SRS-AQM, SRS-BA6, LR-7MN
• Verification of effectiveness: "T-TF-025-007 Summative Evaluation Report"
• User group: ITP → ITP, HCP ✅ UPDATED

R-TBN ("Insufficient label information to understand the device intended use, version"):

• Severity: 2 (initial and controlled); Likelihood: 4 → 1 (after controls)
• Risk control option: C (information for safety)
• Implemented mitigation: "Label design performed according to applicable regulations"
• Mitigation requirements: SRS-HUG, SRS-D6W, SRS-PU2, SRS-SI2, SRS-T5P, SRS-T95, LR-9KT, LR-4KV, LR-2GT
• Verification of effectiveness: "Internal/external audits" → "Internal/external audits" + "T-TF-025-007 Summative Evaluation Report" ✅ UPDATED
• User group: ITP → ITP, HCP ✅ UPDATED

Resolved — R-CGQ and R-TBN updates ✅

Issue 1: Missing analysis. R-TF-025-007 now includes "Assessment of Related Risks" section evaluating R-CGQ and R-TBN mitigation effectiveness using summative evaluation findings (done via sub-item a).

Issue 2: User group scope. Both R-CGQ and R-TBN had user group "ITP" only. Both have been expanded to "ITP, HCP" in R-TF-013-002.json.

Issue 3: R-TBN verification gap. R-TBN verification referenced only "Internal/external audits." T-TF-025-007 Summative Evaluation Report has been added as additional verification of effectiveness. The same rationale that applied to R-CGQ applies to R-TBN: both risks concern users not understanding intended use, both user groups were tested in the summative evaluation, and leaving either risk scoped only to ITPs while BSI is asking about HCPs would invite a follow-up NC.

Changes made to R-TF-013-002.json:

• R-CGQ: userGroup "ITP" → "ITP, HCP"
• R-TBN: userGroup "ITP" → "ITP, HCP"
• R-TBN: verificationOfEffectiveness added "T-TF-025-007 Summative Evaluation Report"
• UserGroup type in types.ts updated to include "ITP, HCP" as valid value

Gap analysis

Partially a gap, partially defensible.

The gap is in the documentation: the summative evaluation report does not include the analysis explaining why the Q4 results are acceptable and how they relate to R-TBN and R-CGQ mitigation effectiveness.

The defense: the data itself shows the controls ARE working (only 1 true use error at 5.6%), but this conclusion was never drawn in the report.

BSI statement	Our position
"33% did not understand the device is not intended for diagnosis"	Reframe: only 5.6% (1/18) made a true use error. 22.2% (4/18) were close calls that self-corrected. 5.6% (1/18) was a use difficulty (uncertainty, not error). The 33% conflates fundamentally different categories of use problems.
"Risks R-TBN and R-CGQ are not clearly effectively mitigated"	R-CGQ: verification references summative evaluation, and testing did occur. The gap is in documenting the analysis, not in the underlying evidence. Fix: update report with RCA + residual risk, and expand R-CGQ to cover HCPs. R-TBN: verification currently references "internal/external audits" only. Fix: update R-TBN in R-TF-013-002 to add the summative evaluation as an additional verification method alongside audits. This is justified because the summative evaluation's Knowledge Assessment (especially Q4) directly tests whether label information is sufficient for users to understand intended use — which is precisely what R-TBN covers. The response should explain that R-TBN is now verified by both audits (label compliance) and summative evaluation (label comprehension).

Response strategy

Lead with concrete actions taken, then provide the supporting analysis. ✅ COMPLETED.

Actions taken:

1. ✅ Updated R-TF-025-007 with RCA, residual risk assessment, and "Assessment of Related Risks" section (done via sub-item a)
2. ✅ Updated R-TF-013-002: R-CGQ user group "ITP" → "ITP, HCP"; R-TBN user group "ITP" → "ITP, HCP"; R-TBN verification of effectiveness expanded
3. ✅ Updated UserGroup type in types.ts to include "ITP, HCP"
4. ✅ Voluntary IFU enhancement implemented (see sub-item d)
5. ✅ Response written referencing all changes, using category-level Q4 breakdown (not participant-level detail)

Response approach used:

• Referenced RCA and residual risk from sub-item a (no duplication)
• Provided Q4 breakdown at category level: 1 UE (5.6%), 3 CC (16.7%), 1 UD (5.6%); 94.4% correct understanding including self-corrections
• Described each R-TF-013-002 change concretely (what field changed, from what to what)
• Referenced voluntary IFU enhancement via sub-item d cross-reference
• Used IEC 62366-1 terminology precisely (close calls, use error, residual risk)

Priority: Highest (addressed jointly with sub-item a). ✅ COMPLETED.

Post-review note

The voluntary IFU enhancement referenced at action 4 above is now concretely implemented. A dedicated "Important Safety Information" page has been added to the Clinical User Manual as the first page (sidebar_position 0). See sub-item d research-and-planning for details. This strengthens the R-CGQ and R-TBN mitigation narrative: both risks now have an additional concrete control measure (the prominent safety callout) beyond the existing intended purpose statement and labeling.