Md Masudur Rahman

Research

I work on reinforcement learning and robot learning for medicine. Most AI in medicine learns from data collected in the past. My research builds agents that learn through interaction with living systems: they act, measure the biological response, and adapt, and they defer to a clinician when a situation is beyond what they can handle. I call this co-adaptive intelligence. My work so far is in three areas, summarized below with representative papers. The publications page has the complete list.

Reinforcement learning

Policy-gradient methods such as PPO lose policy entropy as training proceeds, which limits exploration, and their performance can depend on the choice of action distribution and on careful tuning. Robust Policy Optimization (RPO) addresses this by perturbing the parameters of the action distribution during training: a random value is added to the mean of the Gaussian policy, which keeps the policy’s entropy from collapsing. On continuous-control benchmarks from DeepMind Control, OpenAI Gym, PyBullet, and IsaacGym, RPO generally outperforms PPO and baselines based on entropy regularization, alternative action distributions, and data augmentation. RPO is included in the CleanRL and skrl libraries and was used to train simulation expert policies in research on real-time action chunking for robot policies.

A second line of work concerns generalization. Image observations contain many features that are irrelevant to the task, and policies trained on them can fail when those features change. In work published in Findings of NAACL 2024, we convert image observations into natural-language descriptions and train the policy on the text; these policies generalized better than policies trained directly on images. Related papers on generalization and advantage estimation appeared at ECML-PKDD 2022 and ICMLA 2022.

Vision-language models for clinical decision support

Whether a burn needs surgery depends on its depth, which is difficult to judge by visual inspection; the reported accuracy of clinical visual assessment is 60–75%. With clinical collaborators at a U.S. burn center, I develop methods that assess burns from ultrasound, including B-mode and Tissue Doppler Imaging. Labeled data in this setting is scarce, so these methods use pre-trained vision-language models together with first-order logic reasoning.

KODER (MIUA 2025, Best Paper Award) turns clinical knowledge into first-order-logic premises, removes conflicting premises with an SMT solver, and uses the consistent set to guide the model’s reasoning on ultrasound. With no task-specific training, it raised GPT-4o from 33% to 93% accuracy on the surgical decision and reached 87% on three-class burn depth. AURA (AAAI/IAAI 2026, oral) extends this to unified multimodal reasoning with automatically generated hypotheses, which came within two points of expert-written ones (93% versus 95%). AMBUSH-AI, an automated noninvasive burn diagnostic system, is published in Annals of Surgery (2026), and a related system that incorporates electronic medical records appeared in JMIR Medical Informatics (2025).

Semi-autonomous surgical robotics

Teleoperated surgical robots need low-latency, high-bandwidth communication, which is often unavailable in remote or austere settings. In SARTRES, the surgeon works at a standard teleoperation interface while the system recognizes the atomic sub-tasks being performed (surgemes) and the remote robot executes them semi-autonomously. On a peg-transfer task from the DESK dataset, surgeme recognition accuracy was 91.5% and execution success was 86%. DESERTS (ICRA 2021) applies the same approach under communication delay and maintained an 87% task success rate with delays of up to 5 seconds.

Related work includes the DESK dataset for surgical skill transfer (IROS 2019), skill transfer between robots and between procedures (RO-MAN 2019, 2021), and surgical activity recognition with logic constraints (RO-MAN 2021).

Research directions

My future work develops co-adaptive agents in three directions. In sensing and reasoning, I am extending knowledge-constrained reasoning so that a model abstains when evidence is weak and chooses its next observation, such as another ultrasound view, by its expected value of information. In robot learning, I am building on RPO to study reinforcement-learning fine-tuning of pretrained robot policies, and developing manipulation that models how tissue responds to the robot’s own actions. In therapy, I am developing agents that learn response models from costly experiments and select the next experiment within safety limits, starting with focused-ultrasound treatment.