Image Processing and Machine Vision in Surgery and Its Training

Due to recent advancements in artificial intelligence, the utilization of image and video data along with image processing technologies has been significantly used in automating surgical procedures and their training. Today, one or more cameras or imaging devices are commonly used in most operating rooms to capture crucial medical information for further analysis. These visual data can be leveraged to design and develop automated image-guided systems aimed at assisting specialized surgeons during procedures and facilitating their training. Furthermore, these systems can serve as the cognitive knowledge for assisting surgical robots. An image-guided surgical system comprises of various components, among which the identification, segmentation, and tracking of surgical instruments and critical anatomical regions, phase recognition, motion analysis, and gesture recognition, and more importantly as surgical skill assessment, are the most crucial components. Automating these components by the use of image processing and machine vision will certainly contribute to the system's deep and independent understanding of the surgical scene. This article introduces the important datasets relevant to image processing and machine vision in surgical applications. Subsequently, several research endeavors in the realm of image processing, machine vision, on the above-mentioned applications are presented, and some prospect research areas are introduced.