Large language models versus thrombosis experts: A comparative study on patient education and clinical decision-making in venous thromboembolism.

Background: Large language models (LLMs) have demonstrated remarkable capabilities in various medical fields, yet their performance in thrombosis and haemostasis, particularly in patient education and complex clinical decision-making, is unexplored.

Objectives: We aimed to compare the quality of responses from LLMs versus thrombosis experts and assess clinician's ability to distinguish between them.

Methods: Three experts on thrombosis and haemostasis and three LLMs (Le Chat Pixtral Large, DeepSeek-R1, ChatGPT-4.5) answered three patient education and three clinical decision-making queries. Thirty-seven physicians rated responses for adequacy (1 = "very poor", 10 = "excellent") and estimated their origin (1 = "certainly LLM," 10 = "certainly human"). Mean differences were assessed via t-tests, medians via Wilcoxon tests, and correlations via Spearman's test. All p values were Bonferroni-adjusted.

Results: LLMs provided significantly better patient education responses than experts. Mean adequacy score differences were: Le Chat Pixtral Large +1.6 (95% CI: 1.3-2.0, p<0.01), DeepSeek-R1 +1.7 (95% CI: 1.3-2.1, p<0.001), and ChatGPT-4.5 +1.9 (95% CI: 1.6-2.3, p<0.001). In clinical decision-making, DeepSeek-R1 outperformed experts, (+1.4; 95% CI: 1.1-1.8, p<0.001), whereas Le Chat Pixtral Large (-0.3; 95% CI, -0.8-0.1, p=0.96) and ChatGPT-4.5 (+0.5; 95% CI: 0.0-0.9, p=0.18), performed comparably to experts. Evaluators couldn't distinguish between expert (median: 6.0, interquartile range [IQR]: 3.0-8.0) and LLM-generated responses (median 6.0, IQR: 4.0-8.0).

Conclusion: LLMs outperform experts in VTE-related patient education and match or exceed them in clinical decision-making, providing responses indistinguishable from experts. Though major barriers need to be addressed, LLMs have strong potential to support clinical management and patient education in VTE.