Bug Report: Anonymization and Segmentation — US J&J Psoriasis Clinical Trial

• Date: 2026-04-16
• Reported by (external): Kevin Wilby [JRDUS] <KWilby@its.jnj.com> — email received 2026-04-15
• Reported by (internal): Jon (Customer Success Manager)
• Project: US Johnson & Johnson Clinical Trial Psoriasis Phase 3 (JNJ-77242113)
• UAT phase: UAT 2.0
• Deal: 17976035487

---

Context

During UAT 2.0 (April 2026), the J&J data team reviewed a sample data transfer uploaded to the test S3 bucket (jnj-77242113pso3006-test-data-transfer) for subject CN100150001 (site CV5-CN10015), report ID 019d4b0c-1531-7725-b7bf-36accb988d82.

On 2026-04-15, Kevin Wilby from J&J raised concerns about the anonymization and segmentation of the transferred images. The issues are tracked internally in the project board and are detailed below.

---

Issue 1 — Segmentation: False Positives on Healthy Skin (Head Images)

Description

The erythema_segmenter model, served via the Legit.Health Plus API at {MEDICAL_DEVICE_URL}/clinical/severity-assessment, produces false positives on healthy heads. Specifically:

• The model hallucinates massive lesion areas on healthy skin where no pathology exists.
• This results in inflated area scores for head images that show little or no erythema.
• The model name itself is incorrect: it is labelled erythema_segmenter but should correspond to the inflammatory lesion segmenter.

Evidence

• In the UAT 2.0 test data transfer, the package contains 21 images per report: 11 originals and 10 segmented. The discrepancy (11 vs. 10) may be directly related to the head image being excluded from segmentation due to model unreliability.
• Internal experiments confirmed the model produces outputs inconsistent with clinical expectations for head images with low severity scores (erythema/desquamation/induration near 0–0.03) while simultaneously reporting large affected surface areas.

Root Cause (identified)

The erythema_segmenter was not trained with sufficient healthy head (bald scalp) negative examples, causing it to activate on non-lesional skin texture.

Proposed Solutions (from internal analysis)

Priority	Solution
Short-term	Raise mask binarization threshold from > 125 to > 180 or > 200
Short-term	Correct the model name to match the correct segmenter (inflammatory lesion, not erythema)
Medium-term	Retrain erythema_segmenter with healthy bald head negative examples
Optional	Add severity gating: if all severity scores are near 0, force area to 0 regardless of mask output

Related: Pose Validation False Positives (Head Front/Back)

A parallel bug in the pose validation layer is also relevant to this issue. The internal pose validator (ValidatePose.php) was found to incorrectly approve non-head uploads when ear-like signals are detected in trunk images:

• A trunk-back image without a full head returned head = 1.0 in internal pose validation.
• Batch testing showed ~10% false positive rate in the tested sample.
• Root cause: back-head validation only uses ['left_ear', 'right_ear'] keypoints (ValidatePose.php:46), which can match ear-like regions in shoulder/upper back images. The final threshold of >= 0.5 (ValidatePose.php:30, ValidatePose.php:146) then passes the image.

The decision to ship an updated pose model (scoring-system-global v1.0.0 / v1.0.0-dev via ECR tags) was approved, to be led by Alfonso (MDS) with Alberto as backup.

Status

A combination of intensity output and thresholding was applied and tested as a short-term fix for the segmentation issue. The fix is deployed by pushing the latest version of compute_pathology_area_score.py to the corresponding AWS Lambda. The pose model update is in progress separately.

---

Issue 2 — Anonymization: S3 Bucket Access Blocked (403 Forbidden)

Description

During UAT 2.0, J&J's data team (reported by Uma) encountered a 403 Forbidden error when attempting to access anonymized images from the test S3 bucket.

• Bucket: jnj-77242113pso3006-test-data-transfer
• Example path: s3://jnj-77242113pso3006-test-data-transfer/media/anonymized/0167fee3-6bf9-5966-a84e-9c8f3810c9c6.jpg
• Error on operation: HeadObject

This blocked the J&J data team from validating the data transfer pipeline during UAT 2.0.

Root Cause

IAM permissions for the J&J data team's AWS credentials were incorrectly configured, preventing read access to the test bucket.

Resolution

IAM permissions and bucket policy were reviewed and corrected. Uma (J&J data team) re-ran the aws s3 cp command to confirm access was restored.

Status: Resolved (tracked in project board, moved to Reviewing).

---

Issue 3 — Anonymization: SOW Amendment Pending Signature

Description

The SOW amendment covering anonymized image delivery to J&J is not yet signed as of 2026-04-16. Until the amendment is signed, the anonymization scope and included images scope have not been formally contracted. This means:

• Anonymization deliverables (AN-AUT-BLUR-bsa-bg, AN-AUT-BBOX-EM, AN-MAN-BBOX-TF, AN-MAN-BBOX-O) are being implemented in anticipation of the amendment but are not yet billable or formally obligated.
• The SOW amendment is worth approximately 7,000 anonymized images.

Anonymization Types Being Implemented

Code	Description
AN-AUT-BLUR-bsa-bg	Automatic: background around segmented body surface area blurred
AN-AUT-BBOX-EM	Automatic black box: eyes and mouth
AN-MAN-BBOX-TF	Manual black box: tattoos
AN-MAN-BBOX-O	Manual black box: other identifiable features (jewelry, birthmarks, etc.)

Anonymization Scope

• AN-SCOP-ALL: Full scope — all images anonymized before data transfer.

Implementation

Anonymization is processed via an AWS Step Functions orchestrator (jnj-data-transfer-orchestrator) before upload to the S3 delivery bucket.

Open Questions (Anonymization Follow-up)

The following scope questions were raised internally and remain open:

• Should extra anonymization be applied to leg images beyond background blur?
• Is the foot-back perspective included in the anonymization scope?

These need to be resolved before the SOW amendment is finalized and before production data transfers begin.

Status

Pending contract signature. Expected to be signed before active study data collection (study start: 2026-04-01 — already underway).

---

Summary Table

Issue	Severity	Status
Segmentation false positives (healthy heads)	High — affects data quality delivered to J&J	Short-term fix applied; medium-term retraining recommended
S3 bucket 403 Forbidden for J&J team	High — blocked UAT validation	Resolved
SOW amendment for anonymization not signed	Medium — contractual gap	Pending J&J signature

---

Next Steps

1. Confirm with J&J (Kevin Wilby) that the segmentation fix addresses the reported output issues.
2. Validate that the 21-picture anonymized package for subject CN100150001 (report 019d4b0c-1531-7725-b7bf-36accb988d82) is correctly structured and accessible in the test S3 bucket.
3. Follow up on the SOW amendment signature with the J&J contracting team.
4. If the medium-term segmentation retraining is approved, coordinate with the MDS team (Alfonso / Alberto as backup) to create a new model version before active data collection scales up.

---

Responsible: Jon (Customer Success Manager), Alba (Project Manager), MDS team

---

References (Planka Board)

Card	Title
1708102106029754252	Internal UAT — Model outputs: segmentation errors (erythema_segmenter false positives on healthy heads)
1727582569270084821	Review the pose bug for head front (false positives on trunk images)
1728404720424322535	Update Pose model to improve head pose validation (decision: new version approved)
1729017250540685022	scoring-system-global versioning strategy (ECR tags: 1.0.0, 1.0.0-dev)
1719724737095009679	Ensure Model output and Data transfer output are correct and aligned
1727671498304390466	Follow-up anonymization (open questions: legs, foot-back scope)