‘GPT-4 Outperforms Doctor’ – Apparata
Not only does GPT-4 outperform most doctors on the US medical exam. GPT-4 also makes more accurate diagnoses. The solution to the GP shortage? The answer: yes, to a limited extent.
Contents
GPT-4 as a doctor
Isaac Kohane, a rare combination of a Harvard-level computer scientist and a physician, teamed up with two colleagues to put GPT-4 through the ultimate field test: How would the artificial intelligence do in the medical profession? He described his experiences in his forthcoming book, The AI Revolution in Medicine.
Points where GPT-4 outperforms human doctors
Remarkably enough, artificial intelligence appears to outperform many doctors. In the book, the researcher describes that GPT-4 gave the correct answer in more than 90% of the cases on the entrance exam for doctors in the US. Not only did GPT-4 outperform previous models, such as GPT-3 and GPT-3.5, but so did many human physicians.
But that is not everything. The artificial intelligence is also able to explain medical jargon in plain human language, and even translate it into other languages. With this, the artificial intelligence fulfills some of the most important tasks of a doctor, namely making the diagnosis and communicating with a patient in a way that the patient understands.
In an experiment, he gave the AI some medical data from an ultrasound scan, physical examination, and measured hormone levels. The AI was able to diagnose a very rare condition, congenital adrenal hyperplasia. It usually takes doctors years to reach the professional level necessary for this diagnosis.
Stupid mistakes and lack of ethics
Still, GPT-4 has some weaknesses. The AI doesn’t always get it right. There is also a lack of a sense of ethics, where people do have one. In his book, Kohane describes some of the AI’s blunders. For example, the AI sometimes forgets coefficients in formulas and incorrectly adopts answers.
Another dangerous trait of the AI is that it tends to keep defending wrong answers. No matter how human, you really shouldn’t have that when human lives are at stake. One solution the authors propose is to let the AI check itself, in a new session. Another alternative is to show the reasoning that the bot followed step by step.
No replacement doctors, but solid support
So we cannot leave medicine to AI for the time being, but it can be a huge help for often overburdened doctors to also show less common diagnoses and to make communication with patients easier. And the book? Unfortunately this is not yet available, but will be released soon.
