The robot surgeon learned how to operate after watching a video

The system uses simulation training and analyzes video recordings of real operations. The press service of Johns Hopkins University reports on the development.

Researchers from Johns Hopkins University and Stanford University have trained a surgical robot to perform complex medical procedures with high precision. To achieve this, they used simulation training: the robot repeated the actions it had learned by watching hundreds of videos of real surgical operations.

The system uses the same machine learning architecture as the ChatGPT system, but instead of speech and text, it “speaks the language of the robot” – using kinematics and mathematical expressions to control the movements of the surgical manipulator. After watching videos from cameras mounted on the da Vinci robot arms during operations, the model reproduced the same surgical procedures with a level of skill comparable to experienced doctors.

Robot surgeon

According to the developers, the key point was to teach the model to understand individual movements, rather than memorize specific procedures – sequences of actions. “All we need – it's an image input, and then this AI system finds the right solution”, – noted Ji Woon “Brian” Kim, co-author of the study.

The model is so good at learning things we didn't teach it. For example, if she drops a needle, she will automatically pick it up and continue. That's not what I taught her, – says Axel Krieger, co-author of the study from John Hughes University

The study authors say the approach eliminates the need to carefully program robots for each individual step of a medical procedure, paving the way for truly autonomous surgical robots. Now they can quickly learn how to perform almost any surgery, reducing the risk of medication errors and increasing surgical accuracy.

It's really great to have this model. We believe this marks a significant step forward towards a new frontier in medical robotics, says Axel Krieger, co-author of the study from John Hughes University.