Jain 2021 — AI tool for skin-condition diagnosis by PCPs and NPs in teledermatology
Citation
Jain A, Way D, Gupta V, Gao Y, de Oliveira Marinho G, Hartford J, et al. Development and assessment of an artificial intelligence–based tool for skin condition diagnosis by primary care physicians and nurse practitioners in teledermatology practices. JAMA Netw Open. 2021 Apr 1;4(4):e217249. DOI: 10.1001/jamanetworkopen.2021.7249. PMID 33909055.
Study design and population
Retrospective diagnostic study with randomised case-assignment. 20 primary-care physicians and 20 nurse practitioners reviewed 1,048 teledermatology cases covering 120 skin conditions, with and without AI decision support.
Reported metrics
- Agreement with dermatologist reference, PCPs: 48 % → 58 % (+10 pp); OR 2.0 (95 % CI 1.7–2.4)
- Agreement with dermatologist reference, NPs: 46 % → 58 % (+12 pp); OR 2.2 (95 % CI 1.9–2.6)
Surrogate-to-outcome linkage
Quantifies the AI-decision-support uplift for non-specialists in the exact care-pathway context where CDS devices are deployed. The +10–12 pp correct-triage uplift maps directly to improved referral appropriateness at the primary-care step, on the causal path to appropriate specialist referral and treatment.
CRIT1–7 appraisal
| Criterion | Score | Justification |
|---|---|---|
| CRIT1 Relevance | 3 | Direct — AI decision support for PCPs / NPs in teledermatology triage. |
| CRIT2 Methodology | 2 | Large prospective diagnostic study; randomised case assignment; within-reader AI-vs-no-AI comparison. |
| CRIT3 Reporting | 3 | ORs with 95 % CIs reported. |
| CRIT4 Applicability | 3 | Non-specialist readers — exact intended-use population for AI CDS. |
| CRIT5 Evidence weight | 2 | Large prospective diagnostic study. |
| CRIT6 Risk of bias | 2 | Retrospective image-only cases; dermatologist-panel reference standard imperfect; limited darker-skin representation. |
| CRIT7 Contribution | 3 | Core anchor for the non-specialist AI-uplift claim in Domain 3. |
Aggregate: strong.
Limitations and notes
Retrospective case bank; US teledermatology context; limited FST IV–VI coverage.
Strength as anchor
Strong — the most rigorous published quantification of AI-decision-support uplift specifically for PCPs / NPs. Pairs with Liu 2020 (DLS non-inferiority to dermatologists and superiority to PCPs / NPs) to anchor the primary-care-triage mechanism.