Teleoperated Robotic Surgery

Semi-autonomous Telesurgery.

I have developed a semi-autonomous teleoperated robotic surgery system (Rahman* et al., 2021), (Gonzalez et al., 2021) that can operate with up to 5 seconds of delay with high efficiency (Figure 2). A key challenge was the scarcity of diverse training data for learning algorithms (Rahman et al., 2021). To address this, I created the DESK (Dexterous Surgical Skills) dataset (Madapana* et al., 2019), (Gonzalez* et al., 2021), which collects robotic surgical skills from various platforms. This dataset facilitates knowledge transfer across different domains, particularly where data is scarce.

My approach enables surgeons to execute procedures remotely by translating their high-level commands into robotic actions. The system breaks down complex surgeries into smaller automated segments, known as “surgemes”, improving operation efficiency even with limited data availability. It has achieved an 87% success rate in peg transfer tasks, outperforming standard teleoperation even with substantial communication delays of 5 seconds, compared to standard teleoperation systems, which can fail with delays as short as 300 millisecond. However, these systems must operate reliably in unpredictable conditions, such as disaster zones, where they may encounter novel scenarios.

References

2021

Biomedical Journal

SARTRES: A semi-autonomous robot teleoperation environment for surgery

Md Masudur Rahman*, Mythra V Balakuntala*, Glebys Gonzalez, Mridul Agarwal, Upinder Kaur, Vishnunandan LN Venkatesh, Natalia Sanchez-Tamayo, Yexiang Xue, Richard M Voyles, Vaneet Aggarwal, and Juan Wachs

Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2021

Bib PDF

@article{rahman2021sartres,
  title = {SARTRES: A semi-autonomous robot teleoperation environment for surgery},
  author = {Rahman*, Md Masudur and Balakuntala*, Mythra V and Gonzalez, Glebys and Agarwal, Mridul and Kaur, Upinder and Venkatesh, Vishnunandan LN and Sanchez-Tamayo, Natalia and Xue, Yexiang and Voyles, Richard M and Aggarwal, Vaneet and Wachs, Juan},
  journal = {Computer Methods in Biomechanics and Biomedical Engineering: Imaging \& Visualization},
  volume = {9},
  number = {4},
  pages = {376--383},
  year = {2021},
  publisher = {Taylor \& Francis},
}

ICRA

Deserts: Delay-tolerant semi-autonomous robot teleoperation for surgery

Glebys Gonzalez, Mridul Agarwal, Mythra V Balakuntala, Md Masudur Rahman, Upinder Kaur, Richard M Voyles, Vaneet Aggarwal, Yexiang Xue, and Juan Wachs

In IEEE International Conference on Robotics and Automation (ICRA) , 2021

Bib PDF

@inproceedings{gonzalez2021deserts,
  title = {Deserts: Delay-tolerant semi-autonomous robot teleoperation for surgery},
  author = {Gonzalez, Glebys and Agarwal, Mridul and Balakuntala, Mythra V and Rahman, Md Masudur and Kaur, Upinder and Voyles, Richard M and Aggarwal, Vaneet and Xue, Yexiang and Wachs, Juan},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  year = {2021},
}

RO-MAN

Sequential prediction with logic constraints for surgical robotic activity recognition

Md Masudur Rahman, Richard M Voyles, Juan Wachs, and Yexiang Xue

In 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) , 2021

Bib PDF

@inproceedings{rahman2021sequential,
  title = {Sequential prediction with logic constraints for surgical robotic activity recognition},
  author = {Rahman, Md Masudur and Voyles, Richard M and Wachs, Juan and Xue, Yexiang},
  booktitle = {30th IEEE International Conference on Robot \& Human Interactive Communication (RO-MAN)},
  pages = {468--475},
  year = {2021},
  organization = {IEEE}
}

MilMed Journal

From the dexterous surgical skill to the battlefield—a robotics exploratory study

Glebys T Gonzalez*, Upinder Kaur*, Md Masudur Rahman*, Vishnunandan Venkatesh, Natalia Sanchez, Gregory Hager, Yexiang Xue, Richard Voyles, and Juan Wachs

Military Medicine, 2021

Bib HTML

@article{gonzalez2021dexterous,
  title = {From the dexterous surgical skill to the battlefield—a robotics exploratory study},
  author = {Gonzalez*, Glebys T and Kaur*, Upinder and Rahman*, Md Masudur and Venkatesh, Vishnunandan and Sanchez, Natalia and Hager, Gregory and Xue, Yexiang and Voyles, Richard and Wachs, Juan},
  journal = {Military Medicine},
  url = {https://academic.oup.com/milmed/article/186/Supplement_1/288/6119495},
  volume = {186},
  number = {Supplement\_1},
  pages = {288--294},
  year = {2021},
  publisher = {Oxford University Press US}
}

2019

IROS

Desk: A robotic activity dataset for dexterous surgical skills transfer to medical robots

Naveen Madapana*, Md Masudur Rahman*, Natalia Sanchez-Tamayo*, Mythra V Balakuntala, Glebys Gonzalez, Jyothsna Padmakumar Bindu, LN Vishnunandan Venkatesh, Xingguang Zhang, Juan Barragan Noguera, Thomas Low, and others

In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2019

Bib PDF

@inproceedings{madapana2019desk,
  title = {Desk: A robotic activity dataset for dexterous surgical skills transfer to medical robots},
  author = {Madapana*, Naveen and Rahman*, Md Masudur and Sanchez-Tamayo*, Natalia and Balakuntala, Mythra V and Gonzalez, Glebys and Bindu, Jyothsna Padmakumar and Venkatesh, LN Vishnunandan and Zhang, Xingguang and Noguera, Juan Barragan and Low, Thomas and others},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year = {2019},
}