Researchers successfully use GPT-4 to recommend stroke treatments

January 30, 2024

Stroke AI

Ischemic strokes, which occur when blood flow to the brain is blocked, are a major causes of death and disability. 

A new pre-print study evaluated the potential of GPT-4 to assist doctors in making critical decisions in treating stroke patients.

The research team, comprising experts from the Technion-Israel Institute of Technology in Israel and the Mayo Clinic in the US, analyzed data from 100 patients who had shown acute stroke symptoms. 

The team compared GPT- 4’s treatment recommendations with those given by experienced neurologists and the actual treatments administered to the patients.

The aim was to see how well the AI’s suggestions matched up with expert human judgment and real-world medical practice.

One of the key measures used in this study was the Area Under the Curve (AUC). 

Without getting too bogged down in the technicalities, the ROC curve is a way to visualize how well a diagnostic test performs.

It plots the rate of true positives (correctly identified cases) against the rate of false positives (incorrectly identified cases) at various thresholds. 

The AUC, then, is a single number that summarizes the test’s performance across all possible thresholds, with 1.0 representing a perfect test and 0.5 representing a guess.

In the medical world, an AUC of 0.7 to 0.8 is considered acceptable, 0.8 to 0.9 is excellent, and above 0.9 is outstanding.

In this study, GPT-4 achieved an AUC of 0.85 when its recommendations were compared to the opinions of stroke specialists, indicating a high level of agreement and an excellent performance by the AI. 

Compared to the treatments given, the AUC was 0.80, showing that GPT-4’s suggestions were closely aligned with real-world medical practice. 

These results are particularly promising because they suggest that GPT-4 can potentially provide valuable support in emergency rooms, especially when a neurology specialist might not be immediately available.

Moreover, GPT-4 showed a remarkable ability to predict the risk of mortality within 90 days post-stroke. 

The AI model identified patients at high risk with significant accuracy, outperforming some existing machine-learning models specifically trained for this purpose. 

This could be incredibly useful for doctors in prioritizing treatments and managing resources more effectively.

This isn’t the first time LLMs have been used successfully for healthcare applications.

Google recently created Articulate Medical Intelligence Explorer (AMIE), which matched or even outperformed board-certified primary-care physicians in gathering patient information during medical interviews and scored higher in empathy.

Danish researchers even used LLMs to understand how life events affected mortality, with their model beating the next-best by 11%.

Other sophisticated machine learning models have discovered new antibiotics or therapeutic compounds in mere minutes compared to the months or years of traditional experimental techniques.

Join The Future


Clear, concise, comprehensive. Get a grip on AI developments with DailyAI

Sam Jeans

Sam is a science and technology writer who has worked in various AI startups. When he’s not writing, he can be found reading medical journals or digging through boxes of vinyl records.


Stay Ahead with DailyAI

Sign up for our weekly newsletter and receive exclusive access to DailyAI's Latest eBook: 'Mastering AI Tools: Your 2024 Guide to Enhanced Productivity'.

*By subscribing to our newsletter you accept our Privacy Policy and our Terms and Conditions