Benchmarking ChatGPT-3.5 and OpenAI o3 Against Clinical Pharmacists: Preliminary Insights into Clinical Accuracy, Sensitivity, and Specificity in Pharmacy MCQs.

Healthcare (Basel) · 2025 · PMC12294861 · PMID 40724776

Europe PMC DOI

hedged sentences

0.0500

closest p · 1.0× alpha

0.0500

boldest claim

The sentences

an important trendp < 0.05actually significant

In a striking manner, this study concludes an important trend that accuracy decreases as question difficulty increases, this trend is valid for both AI tools in a statistically significant difference ( p < 0.05).

also in 373 other papers

a slight trendno p-value reported

This change, with 11 answers changing from correct to incorrect and 3 from incorrect to correct, resulted in a p -value of 0.057, which suggests a slight trend toward a change, but it did not reach statistical significance. 4.

also in 1,229 other papers

did not reach statistical significanceno p-value reported

This change, with 11 answers changing from correct to incorrect and 3 from incorrect to correct, resulted in a p -value of 0.057, which suggests a slight trend toward a change, but it did not reach statistical significance. 4.

also in 34,328 other papers

Quoted from the open-access full text in Europe PMC under the licence the publisher applied. The sentence is reproduced exactly as published; the emphasis is ours.