World first: robot-assisted lung cancer surgery with augmented reality

Surgeons at Erasmus MC have achieved a world-first by successfully removing a lung tumor with the assistance of augmented reality during a robot-assisted surgery. This technique allows for the projection of virtual images of the tumor, blood vessels, and airways onto the lung during surgery. ‘Augmented reality enables us to navigate more safely and effectively.’

Reading time 2 min
With augmented reality, virtual images of the airways (green) and blood vessels (blue) are projected over the lung during robot-assisted surgery.

In 2020, a revolution occurred in complex lung cancer surgery with the introduction of virtual reality (VR). Surgeons started using VR headsets to meticulously map the anatomy of the lung, pinpointing the exact location of a tumor before surgery.

Less than three years later, the same surgeons at Erasmus MC have taken the next step by performing robot-assisted lung cancer surgery with augmented reality. During the surgery, virtual images of the lung are projected onto the actual lung in real-time. This allows the surgeon to visualize the locations of blood vessels, airways, and the tumor, which are not always visible to the naked eye and are crucial for determining the optimal route to the tumor.

‘With augmented reality, we can navigate more safely and effectively within the patient’s chest’, says Dr. Lex Maat, a cardiothoracic surgeon. The Rotterdam-based surgeons are the first in the world to utilize augmented reality in robot-assisted lung surgery.


The Erasmus MC surgeons employed augmented reality in two robot-assisted surgeries. During these procedures, the surgeon operates from a control panel for a surgical robot, not directly at the operating table. Using joysticks, the surgeon handles surgical instruments that have been inserted into the patient’s body through small incisions. A camera inside the patient provides a view of the surgery on a screen, where virtual images of the lung can be overlaid, creating augmented reality.

‘You can compare augmented reality to GPS’,  explains researcher and cardiothoracic surgeon in training Dr. Amir Sadeghi. He and cardiothoracic surgeon Dr. Sabrina Siregar conducted the first two surgeries utilizing augmented reality. ‘Augmented reality aids the surgeon in performing more precise operations. For example, we can now see that there’s an artery running beneath a specific part of the lung’,  adds Siregar.

Promising results

The first two patients who underwent surgeries with augmented reality are recovering well, and the surgical team is enthusiastic about the results. Sadeghi states: ‘The initial tests were promising; we now know that the technique works effectively. We need to further validate the technology and enhance its user-friendliness so that it can be more readily applied in potential clinical research.’

The team anticipates that augmented reality will lead to faster surgeries with fewer complications such as bleeding, and it may also accelerate the learning curve for novice surgeons when integrated early in their training. ‘The precise benefits for patient care will need to be investigated in future clinical research’, Sadeghi explains. He continues to develop the augmented reality technology with a team of cardiothoracic surgeons, technical physicians, software developers and artificial intelligence specialists.

The augmented reality lung cancer surgery at Erasmus MC was performed thanks to a collaboration with the companies MedicalVR and Orsi Innotech (in partnership with NVIDIA). Application of augmented reality was possible thanks to a recent project in which Sadeghi developed patient-specific virtual and deformable lung models, in collaboration with MedicalVR. This builds on augmented-reality technology developed by the company Orsi Innotech in collaboration with NVIDIA.

Also read