Sensorium addresses the infrastructure behind surgical AI: collecting usable recordings before a model can learn from them. The paper presents an open-source Python library that connects data acquisition, processing, annotation and dataset preparation across configurable robots and sensors.
One workflow for recording and curation
Sensorium uses ROS and ROS 2 to integrate robot and sensor streams. A browser interface supports recording supervision and episode review, while the processing tools handle tasks including timestamp alignment and stereo rectification. Researchers can annotate surgical phases and language instructions.
The library exports episodic datasets in CSV-based representations and the LeRobot format. That matters when the same underlying recording needs to support different learning pipelines: a lab can keep the original evidence while preparing the representations required for a particular experiment.
What the evaluation covers
The authors evaluate the library across 1,516 recordings involving phantoms, ex vivo material and in vivo porcine experiments. They also report preliminary usability feedback. This is evidence about a research data workflow across several experimental configurations; it does not establish the safety or effectiveness of an autonomous clinical system.
The paper motivates the work with an expensive practical constraint. Surgical collection requires preparation, specialist time and, for in vivo experiments, appropriate approvals. Missing streams or poor annotations can therefore waste an opportunity that is difficult to repeat.
Where a research team should start
A useful adoption test would choose one existing robot setup and follow a complete episode from sensor readiness to exported training data. Check time alignment, missing inputs, annotation meaning and whether a second researcher can reconstruct the episode. Keep hardware configuration and processing choices with the data. Sensorium's contribution is a reusable workflow for that preparation; the downstream model and its intended setting still need their own evaluation.
Comments