‘Vision-Enabled’ AI Scribes Could Improve GP Notes
24 Mar 2026
A trial by Flinders University suggests AI scribes that can see as well as hear may take more accurate clinical notes than audio-only systems.
The trial used Ray-Ban Meta smart glasses with Google’s Gemini AI model to record 10 pharmacists completing 110 simulated medication history interviews. The vision-enabled AI scribe correctly recorded 98% of all data points, compared to 81% accuracy for audio-only AI. Most of the improvement came from fewer missed details in the video-enabled recordings.
Lead researcher Bradley Menz said video added valuable information often missed by audio alone, such as the strength and form of medication. “People might bring a Webster-pak or show their medications—there’s a lot of visual information that’s lost if the AI can’t see,” he said.
While errors still occurred, mainly from incorrect details, Mr Menz emphasised careful review before using such technology in routine practice.
RACGP experts said the study highlights the potential of smart glasses and vision-enabled AI in general practice. Dr Sean Stevens noted, “Visual cues are often missed in audio-only transcripts. Combining vision with audio improves accuracy and could support better patient care.”
Dr David Adam cautioned that the technology is still early-stage and that more testing in real clinical settings is needed. Smart-glass AI has shown challenges in real-world environments, he said.
The trial shows the promise of vision-enabled AI scribes, with researchers hoping it inspires further work to improve clinical workflows and patient safety.
https://www1.racgp.org.au/
(Source: Flinders University, newsGP)