New AI technique could make minimally invasive surgeries safer and more precise
MIT researchers have developed an artificial intelligence system that can match real-time surgical X-rays with a patient’s preoperative 3D scan in seconds. The technique, called xvr, is designed to help clinicians locate and orient surgical tools with sub-millimeter precision during minimally invasive procedures.
The system could support procedures in areas including orthopedics and neurosurgery, where doctors use X-rays to guide instruments through small incisions. By combining the immediacy of 2D X-ray images with the anatomical detail of CT scans or MRIs, xvr aims to make surgical navigation faster and more reliable. as reported by News Mit ## Why 2D X-rays can be difficult to navigate
In procedures such as angioplasty, clinicians insert tools through tiny incisions and use a mobile X-ray scanner to view the operation from different angles. Those images provide real-time feedback, but an X-ray is a flat projection of the body. It does not directly show the precise three-dimensional position or orientation of a catheter, endoscope, or other instrument.
Clinicians therefore may need to register, or align, the live X-ray images with a patient’s preoperative CT or MRI scan. Registration helps show where a tool is located relative to bones, blood vessels, organs, and other anatomical structures.
That process can be slow and demanding. Manual approaches may require clinicians to estimate a tool’s position by entering numbers or selecting anatomical landmarks on a screen. The skill needed to interpret grainy, two-dimensional images also takes years to develop.
“It takes decades of training for a clinician to become skilled enough to see grainy, 2D images and understand how everything is oriented,” said Vivek Gopalakrishnan, a postdoctoral researcher in MIT’s Computer Science and Artificial Intelligence Laboratory and lead author of the research. “We want to make these 2D X-rays more informative, so it becomes safer and easier to do these life-saving procedures.” The ai research story also surfaces in New AI Architecture Mimics the Human..., adding another angle.
Existing AI systems have attempted to automate 2D-to-3D registration, but patient anatomy varies widely. A model that performs well for one person or body region may not work reliably for another. The researchers also point to a shortage of high-quality annotated medical images as a barrier to training a single system that works robustly across patients.
A model tailored to each patient
Rather than building one model expected to work equally well for everyone, the MIT-led team designed xvr to adapt to each individual patient.
The system begins with a patient’s preoperative 3D scan, such as a CT or MRI. It uses a physics-based simulation of the X-ray process to generate thousands of synthetic X-rays from different angles, producing about 1,000 images per second. These simulated images are based on the patient’s own scan rather than generated from unrelated data.
The researchers say that physics-based approach helps ensure the synthetic images remain grounded in the patient’s anatomy. “Because xvr creates patient-specific data in a purely physics-based manner, there is no room for hallucinations,” Gopalakrishnan said.
The synthetic images are then used to train an AI model that can align the patient’s actual X-rays with the 3D scan. Training a new model entirely from scratch for every patient would take about 12 hours, however, which would make the approach impractical for emergency care. The ai research story also surfaces in MIT Researchers Develop Method to Make..., adding another angle.
To solve that problem, the team first trained a more general foundation model. It used whole-body 3D scans from more than 2,000 patients, spanning different ages, imaging modalities, and anatomical regions. Xvr generated synthetic X-rays from those scans, allowing the foundation model to learn the broader task of 2D/3D registration.
When a new patient is introduced, the pretrained model can adapt to that person in about five minutes. According to the researchers, the adapted model achieves the same accuracy as a model trained from scratch, while making patient-specific registration available much faster.
Testing across patients and procedures
The researchers evaluated xvr using what they described as the largest available dataset of real 2D/3D registrations. The dataset included information from five hospitals and covered dozens of bones and organ systems in both adult and pediatric patients.
Across that range, xvr outperformed existing AI-based methods by an order of magnitude in accuracy and robustness, according to the team. It also operated quickly enough for emergency surgical applications and could potentially improve robotic surgery systems.
The researchers connect the technology to procedures that are not widely available in all communities. Gopalakrishnan noted that many Americans live more than an hour from a center capable of performing noninvasive procedures such as emergency stroke interventions. For a practical look at image generation, Audio Transcription is a useful comparison.
“An hour in stroke time is incredibly substantial,” he said. “Making these procedures easier by combining 2D and 3D information enables these types of highly specialized life-saving procedures to be more accessible to much broader parts of the population.”
The work reflects a broader direction in medical AI: rather than relying only on a general-purpose model, systems can use a patient’s own medical data to adapt to individual anatomy. That approach could be particularly useful when small positional errors carry serious consequences.
What comes next for xvr
The researchers say xvr still needs further development before it can be used as a real-time clinical tool. Their next goals include making the system faster, testing its reliability in additional situations, and extending it to more complex cases involving moving body parts.
They are also collaborating with surgical robotics companies and clinical groups to explore how the research could become a practical navigation or deployment tool. The team has spent two years developing and validating the algorithm, but further studies will be needed to determine how consistently it performs in clinical environments.
The research was led by Gopalakrishnan with Polina Golland, an MIT professor and principal investigator in CSAIL, and Neel Dey, an investigator at Harvard Medical School and Massachusetts General Hospital. The paper, “Rapid patient-specific neural networks for X-ray to volume registration,” appears in Nature.