Artificial Intelligence Assisted Surgical Scene Recognition: A Comparative Study Amongst Healthcare Professionals

Abstract

Objective: This study aimed to compare the ability of a deep-learning platform (the MACSSwin-T model) with healthcare professionals in detecting cerebral aneurysms from operative videos. Secondly, we aimed to compare the neurosurgical team’s ability to detect cerebral aneurysms with and without AI-assistance. Background: Modern microscopic surgery enables the capture of operative video data on an unforeseen scale. Advances in computer vision, a branch of artificial intelligence (AI), have enabled automated analysis of operative video. These advances are likely to benefit clinicians, healthcare systems, and patients alike, yet such benefits are yet to be realised. Methods: In a cross-sectional comparative study, neurosurgeons, anaesthetists, and operating room (OR) nurses, all at varying stages of training and experience, reviewed still frames of aneurysm clipping operations and labelled frames as ‘aneurysm not in frame’ or ‘aneurysm in frame’. Frames then underwent analysis by the AI platform. A second round of data collection was performed whereby the neurosurgical team had AI-assistance. Accuracy of aneurysm detection was calculated for human only, AI only, and AI-assisted human groups. Results: 5,154 individual frame reviews were collated from 338 healthcare professionals. Healthcare professionals correctly labelled 70% of frames without AI assistance, compared to 78% with AI-assistance (OR 1.77, p<0.001). Neurosurgical Attendings showed the greatest improvement, from 77% to 92% correct predictions with AI-assistance (OR 4.24, p=0.003). Conclusion: AI-assisted human performance surpassed both human and AI alone. Notably, across healthcare professionals surveyed, frame accuracy improved across all subspecialties and experience levels, particularly among the most experienced healthcare professionals. These results challenge the prevailing notion that AI primarily benefits junior clinicians, highlighting its crucial role throughout the surgical hierarchy as an essential component of modern surgical practice.