publications

For the up-to-date publication list, please see Google Scholar or Semantic Scholar pages.
For coverage of research in press, please see Press & Media.

Preprints

OG-VLA: 3D-Aware Vision Language Action Model via Orthographic Image Generation.

Ishika Singh, Ankit Goyal, Stan Birchfield, Dieter Fox, Animesh Garg, and Valts Blukis

2025

arXiv Website
AMPLIFY: Actionless Motion Priors for Robot Learning from Videos.

Jeremy A. Collins, Loránd Cheng, Kunal Aneja, Albert Wilcox, Benjamin Joffe, and Animesh Garg

2025

arXiv Website
Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025.

Nitya Thakkar, Mert Yuksekgonul, Jake Silberg, Animesh Garg, Nanyun Peng, Fei Sha, Rose Yu, Carl Vondrick, and James Zou

2025

arXiv Code
Scaling Laws in Scientific Discovery with AI and Robot Scientists.

Pengsong Zhang, Heng Zhang, Huazhe Xu, Renjun Xu, Zhenting Wang, Cong Wang, Animesh Garg, Zhibin Li, Arash Ajoudani, and Xinyu Liu

2025

arXiv

Papers (peer-reviewed)

ACGD: Visual Multitask Policy Learning with Asymmetric Critic Guided Distillation.

Krishnan Srinivasan, Jie Xu, Henry Ang, Eric Heiden, Dieter Fox, Jeannette Bohg, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2025

Visual Multitask Policy Learning, Asymmetric Critic

Reinforcement Learning Visual Learning Policy Learning
PDF Website
AnyPlace: Learning Generalized Object Placement for Robot Manipulation.

Yuchi Zhao, Miroslav Bogdanovic, Chengyuan Luo, Steven Tohme, Kourosh Darvish, Alán Aspuru-Guzik, Florian Shkurti, and Animesh Garg

Conference on Robot Learning (CoRL), 2025

arXiv Website
Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learning.

Albert Wilcox, Mohamed Ghanem, Masoud Moghani, Pierre Barroso, Benjamin Joffe, and Animesh Garg

Conference on Robot Learning (CoRL), 2025

arXiv Website
TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policies.

Liquan Wang, Jiangjie Bian, Eric Heiden, and Animesh Garg

Conference on Robot Learning (CoRL), 2025

Website
CLIMB: Language-Guided Continual Learning for Task Planning with Iterative Model Building.

Walker Byrnes, Miroslav Bogdanovic, Avi Balakirsky, Stephen Balakirsky, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2025

arXiv Website
RoCoDA: Counterfactual Data Augmentation for Data-Efficient Robot Learning from Demonstrations.

Ezra Ameperosa, Jeremy A Collins, Mrinal Jain, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2025

Counterfactual Data Augmentation, Robot Learning, Data Efficiency, causality

Data Augmentation
arXiv Website
SuFIA-BC: Generating High Quality Demonstration Data for Visuomotor Policy Learning in Surgical Subtasks.

Masoud Moghani, Nigel Nelson, Mohamed Ghanem, Andres Diaz-Pinto, Kush Hari, Mahdi Azizian, Ken Goldberg, Sean Huver, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2025

arXiv Video Website
PWM: Policy Learning with Large World Models.

Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg

International Conference on Learning Representations (ICLR), 2025

Reinforcement Learning World Models Policy Learning
arXiv Website
EgoSim: Egocentric Exploration in Virtual Worlds with Multi-modal Conditioning.

Wei Yu, Songheng Yin, Steve Easterbrook, and Animesh Garg

International Conference on Learning Representations (ICLR), 2025

PDF Website
ORGANA: a robotic assistant for automated chemistry experimentation and characterization.

Kourosh Darvish, Marta Skreta, Yuchi Zhao, Naruki Yoshikawa, Sagnik Som, Miroslav Bogdanovic, Yang Cao, Han Hao, Haoping Xu, Alán Aspuru-Guzik, Animesh Garg, and Florian Shkurti

Matter, 2025

arXiv Video Website
Accelerating Discovery in Natural Science Laboratories with AI and Robotics: Perspectives and Challenges from the 2024 IEEE ICRA Workshop, Yokohama, Japan.

Andrew I. Cooper, Patrick Courtney, Kourosh Darvish, Moritz Eckhoff, Hatem Fakhruldeen, Andrea Gabrielli, Animesh Garg, Sami Haddadin, Kanako Harada, Jason Hein, Maria Hübner, Dennis Knobbe, Gabriella Pizzuto, Florian Shkurti, Ruja Shrestha, Kerstin Thurow, 7 more authors

Science Robotics, 2025

arXiv
AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agent.

Tongzhou Mu, Yijie Guo, Jie Xu, Ankit Goyal, Hao Su, Dieter Fox, and Animesh Garg

International Symposium on Robotics Research (ISRR), 2024

arXiv
DexMOTS: Dexterous Manipulation with Differentiable Simulation.

Krishnan Srinivasan, Jeremy Collins, Eric Heiden, Ian Ng, Jeannette Bohg, and Animesh Garg

International Symposium on Robotics Research (ISRR), 2024

arXiv Website
Quest: Self-supervised skill abstractions for learning continuous control.

Atharva Mete, Haotian Xue, Albert Wilcox, Yongxin Chen, and Animesh Garg

Advances in Neural Information Processing Systems (NeurIPS), 2024

arXiv Website
Discovering Robotic Interaction Modes with Discrete Representation Learning.

Liquan Wang, Ankit Goyal, Haoping Xu, and Animesh Garg

Conference on Robot Learning (CoRL), 2024

arXiv Video Website
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation.

Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Reed Garrett, and Ajay Mandlekar

Conference on Robot Learning (CoRL), 2024

arXiv Website
Fast Explicit-Input Assistance for Teleoperation in Clutter.

Nick Walker, Xuning Yang, Animesh Garg, Maya Cakmak, Dieter Fox, and Claudia Pérez-D’Arpino

IEEE International Conference on Intelligent Robots and Systems (IROS), 2024

arXiv
SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistants.

Masoud Moghani, Lars Doorenbos, William Chung-Ho Panitch, Sean Huver, Mahdi Azizian, Ken Goldberg, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2024

arXiv Website
Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation.

Ignat Georgiev, Krishnan Srinivasan, Jie Xu, Eric Heiden, and Animesh Garg

International Conference on Machine Learning (ICML), 2024

arXiv Website
Handypriors: Physically consistent perception of hand-object interactions with differentiable priors.

Shutong Zhang, Yi-Ling Qiao, Guanglei Zhu, Eric Heiden, Dylan Turpin, Jingzhou Liu, Ming Lin, Miles Macklin, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2024

arXiv Website
Open X-Embodiment: Robotic learning datasets and RT-X models.

Open X-Embodiment Collaboration

IEEE International Conference on Robotics and Automation (ICRA), 2024

ICRA Best Paper Award
Website
ORBIT-Surgical: An Open-Simulation Framework for Learning Surgical Augmented Dexterity.

Qinxi Yu, Masoud Moghani, Karthik Dharmarajan, Vincent Schorp, William Chung-Ho Panitch, Jingzhou Liu, Kush Hari, Huang Huang, Mayank Mittal, Ken Goldberg, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2024

arXiv Code Website
Slotdiffusion: Object-centric generative modeling with diffusion models.

Ziyi Wu, Jingyu Hu, Wuyue Lu, Igor Gilitschenski, and Animesh Garg

Advances in Neural Information Processing Systems (NeurIPS, 2023

arXiv Website
Composable part-based manipulation.

Weiyu Liu, Jiayuan Mao, Joy Hsu, Tucker Hermans, Animesh Garg, and Jiajun Wu

Conference on Robot Learning (CoRL), 2023

arXiv
Geometry Matching for Multi-Embodiment Grasping.

Maria Attarian, Muhammad Adil Asif, Jingzhou Liu, Ruthrash Hari, Animesh Garg, Igor Gilitschenski, and Jonathan Tompson

Conference on Robot Learning (CoRL), 2023

arXiv Code Website
Benchmarks for Physical Reasoning AI.

Andrew Melnik, Robin Schiewer, Moritz Lange, Andrei Ioan Muresanu, saeidi, Animesh Garg, and Helge Ritter

Transactions on Machine Learning Research (TMLR), 2023

Survey Certification

PDF Code
Robot-Assisted Vascular Shunt Insertion with the dVRK Surgical Robot.

Karthik Dharmarajan, Will Panitch, Baiyu Shi, Huang Huang, Lawrence Yunliang Chen, Masoud Moghani, Qinxi Yu, Kush Hari, Thomas Low, Danyal Fer, Animesh Garg, and Ken Goldberg

Journal of Medical Robotics Research (JMRR), 2023

link Website
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models.

Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2023

arXiv Website
nerf2nerf: Pairwise Registration of Neural Radiance Fields.

Lily Goli, Daniel Rebain, Sara Sabour, Animesh Garg, and Andrea Tagliasacchi

IEEE International Conference on Robotics and Automation (ICRA), 2023

arXiv Code Website
Self-Supervised Learning of Action Affordances as Interaction Modes.

Liquan Wang, Nikita Dvornik, Rafael Dubeau, Mayank Mittal, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2023

arXiv Website
Fast-Grasp’D: Dexterous Multi-finger Grasp Generation Through Differentiable Simulation.

Dylan Turpin, Tao Zhong, Shutong Zhang, Guanglei Zhu, Jingzhou Liu, Ritvik Singh, Eric Heiden, Miles Macklin, Stavros Tsogkas, Sven Dickinson, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2023

arXiv Video Website
MVTrans: Multi-View Perception of Transparent Objects.

Yi Ru Wang, Yuchi Zhao, Haoping XU, Sagi Eppel, Alan Aspuru-Guzik, Florian Shkurti, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2023

arXiv Video Website
Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward.

Zihan Zhou and Animesh Garg

International Conference on Learning Representations (ICLR), 2023

Website
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models.

Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, and Animesh Garg

International Conference on Learning Representations (ICLR), 2023

arXiv Website
DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting.

Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos

Autonomous Robots (AURO), 2023

arXiv Video Blog Code Website
ORBIT: A Unified Simulation Framework for Interactive Robot Learning Environments.

Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Pooria Poorsarvi Tehrani, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg

IEEE Robotics and Automation Letters (RA-L) and ICRA, 2023

arXiv Code Website
Integration of reinforcement learning in a virtual robotic surgical simulation.

Alexandra T Bourdillon, Animesh Garg, Hanjay Wang, Y Joseph Woo, Marco Pavone, and Jack Boyd

Journal of Surgical Innovation, 2023
Digital pipette: open hardware for liquid transfer in self-driving laboratories.

Naruki Yoshikawa, Kourosh Darvish, Mohammad Ghazi Vakili, Animesh Garg, and Alán Aspuru-Guzik

Digital Discovery, 2023

DOI
Large language models for chemistry robotics.

Naruki Yoshikawa, Marta Skreta, Kourosh Darvish, Sebastian Arellano-Rubach, Zhi Ji, Lasse Bjørn Kristensen, Andrew Zou Li, Yuchi Zhao, Haoping Xu, Artur Kuramshin, Alán Aspuru-Guzik, Florian Shkurti, and Animesh Garg

Autonomous Robots (AURO), 2023

DOI Website
ProgPrompt: program generation for situated robot task planning using large language models.

Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg

Autonomous Robots (AURO), 2023

DOI Website
Bayesian Object Models for Robotic Interaction with Differentiable Probabilistic Programming.

Krishna Murthy Jatavallabhula, Miles Macklin, Dieter Fox, Animesh Garg, and Fabio Ramos

Conference on Robot Learning (CoRL), 2022

PDF Website
RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments.

Haoyu Xiong, Haoyuan Fu, Jieyi Zhang, Chen Bao, Qiang Zhang, Yongxi Huang, Wenqiang Xu, Animesh Garg, and Cewu Lu

Conference on Robot Learning (CoRL), 2022

Oral Presentation
PDF Video Website
Breaking Bad: A Dataset for Geometric Fracture and Reassembly.

Silvia Sellán, Yun-Chun Chen, Ziyi Wu, Animesh Garg, and Alec Jacobson

Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks, 2022

Featured Paper Presentation
arXiv PDF Website
SMPL: Simulated Industrial Manufacturing and Process Control Learning Environments.

Mohan Zhang, Xiaozhou Wang, Benjamin Decardi-Nelson, Bo Song, An Zhang, Jinfeng Liu, Sile Tao, Jiayi Cheng, Xiaohong Liu, Dengdeng Yu, Matthew Poon, and Animesh Garg

Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks, 2022

arXiv Code Website
MoCoDA: Model-based Counterfactual Data Augmentation.

Silviu Pitis, Elliot Creager, Ajay Mandlekar, and Animesh Garg

Advances in Neural Information Processing Systems (NeurIPS), 2022

arXiv
Monotonic Quantile Network for Worst-Case Offline Reinforcement Learning.

Chenjia Bai, Ting Xiao, Zhoufan Zhu, Lingxiao Wang, Fan Zhou, Animesh Garg, Bin He, Peng Liu, and Zhaoran Wang

IEEE Transactions on Neural Networks and Learning Systems, 2022

PDF
InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers.

Tan M. Nguyen, Animesh Garg, Richard G. Baraniuk, and Anima Anandkumar

IEEE Asilomar Conference on Signals, Systems, and Computers, 2022

arXiv
Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger.

Arthur Allshire, Mayank Mittal, Varun Lodaya, Viktor Makoviychuk, Denys Makoviichuk, Felix Widmaier, Manuel Wüthrich, Stefan Bauer, Ankur Handa, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2022

arXiv Website
Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation.

Mayank Mittal, David Hoeller, Farbod Farshidian, Marco Hutter, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2022

arXiv Website
Grasp’D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Hands.

Dylan Turpin, Liquan Wang, Eric Heiden, Yun-Chun Chen, Miles Macklin, Stavros Tsogkas, Sven Dickinson, and Animesh Garg

European Conference on Computer Vision (ECCV), 2022

Oral (Top 2.7%)
arXiv Video Code Website
Continuous-Time Fitted Value Iteration for Robust Policies.

Michael Lutter, Boris Belousov, Shie Mannor, Dieter Fox, Animesh Garg, and Jan Peters

Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2022

arXiv Website
Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamics.

Matthias Weissenbacher, Samarth Sinha, Animesh Garg, and Yoshinobu Kawahara

International Conference on Machine Learning (ICML), 2022

arXiv
Experience Replay with Likelihood-free Importance Weights.

Samarth Sinha, Jiaming Song, Animesh Garg, and Stefano Ermon

Learning for Dynamics and Control (L4DC), 2022

Best Paper Finlist
arXiv
GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model.

Zhaoming Xie, Xingye Da, Buck Babich, Animesh Garg, and Michiel van de Panne

Workshop on Algorithmic Foundations of Robotics (WAFR), 2022

arXiv Website
X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval.

Satya Krishna Gorti, Noel Vouitsis, Junwei Ma, Keyvan Golestan, Maksims Volkovs, Animesh Garg, and Guangwei Yu

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

arXiv Code Website
Modular Action Concept Grounding in Semantic Video Prediction.

Wei Yu, Wenxin Chen, Songheng Yin, Steve Easterbrook, and Animesh Garg

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

arXiv Website
Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors.

Yun-Chun Chen, Haoda Li, Dylan Turpin, Alec Jacobson, and Animesh Garg

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

Website
Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning.

Chenjia Bai, Lingxiao Wang, Zhuoran Yang, Zhi-Hong Deng, Animesh Garg, Peng Liu, and Zhaoran Wang

International Conference on Learning Representations (ICLR), 2022

arXiv
Value Gradient weighted Model-Based Reinforcement Learning.

Claas A Voelcker, Victor Liao, Animesh Garg, and Amir-massoud Farahmand

International Conference on Learning Representations (ICLR), 2022

arXiv PDF
Accelerated Policy Learning with Parallel Differentiable Simulation.

Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, and Miles Macklin

International Conference on Learning Representations (ICLR), 2022

PDF Website
PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks.

Jiankai Sun, De-An Huang, Bo Lu, Yun-Hui Liu, Bolei Zhou, and Animesh Garg

IEEE Robotics and Automation Letters (RA-L) and ICRA, 2022

arXiv Code Website
Centralized Model and Exploration Policy for Multi-Agent RL.

Qizhen Zhang, Chris Lu, Animesh Garg, and Jakob Foerster

International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022

Oral Presentation
arXiv
Convergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings.

Matthew Shunshi Zhang, Murat Erdogdu, and Animesh Garg

Confernce on Artificial Intelligence (AAAI), 2022

arXiv
Real Robot Challenge: A Robotics Competition in the Cloud.

Stefan Bauer, Manuel Wüthrich, Felix Widmaier, Annika Buchholz, Sebastian Stark, Anirudh Goyal, Thomas Steinbrenner, Joel Akpo, Shruti Joshi, Vincent Berenz, Vaibhav Agrawal, Niklas Funk, Julen Urain De Jesus, Jan Peters, Joe Watson, Claire Chen, 25 more authors

Neural Information Processing Systems (NeurIPS) Competitions and Demonstrations Track, 2021

arXiv PDF
Dynamic Bottleneck for Robust Self-Supervised Exploration.

Chenjia Bai, Lingxiao Wang, Lei Han, Animesh Garg, Jianye Hao, Peng Liu, and Zhaoran Wang

Advances in Neural Information Processing Systems (NeurIPS), 2021

arXiv
Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions.

Michael Poli, Stefano Massaroli, Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, and Animesh Garg

Advances in Neural Information Processing Systems (NeurIPS), 2021

arXiv
Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers.

Nikita Dvornik, Isma Hadji, Konstantinos G. Derpanis, Animesh Garg, and Allan D. Jepson

Advances in Neural Information Processing Systems (NeurIPS), 2021

arXiv
Seeing Glass: Joint Point-Cloud and Depth Completion for Transparent Objects.

Haoping Xu, Yi Ru Wang, Sagi Eppel, Alan Aspuru-Guzik, Florian Shkurti, and Animesh Garg

Conference on Robot Learning (CoRL), 2021

Oral Presentation
arXiv Code Website
S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning.

Samarth Sinha, Ajay Mandlekar, and Animesh Garg

Conference on Robot Learning (CoRL), 2021

arXiv
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution.

Valts Blukis, Chris Paxton, Dieter Fox, Animesh Garg, and Yoav Artzi

Conference on Robot Learning (CoRL), 2021

arXiv Code Poster Website
Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos.

Haoyu Xiong, Quanzhou Li, Yun-Chun Chen, Homanga Bharadhwaj, Samarth Sinha, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2021

arXiv Video Website
Learning Latent Actions to Control Assistive Robots.

Dylan P. Losey, Hong Jun Jeon, Mengxi Li, Krishnan Srinivasan, Ajay Mandlekar, Animesh Garg, Jeannette Bohg, and Dorsa Sadigh

Autonomous Robots (AURO), 2021

arXiv
Robust Value Iteration for Continuous Control Tasks.

Michael Lutter, Shie Mannor, Jan Peters, Dieter Fox, and Animesh Garg

Robotics: Systems and Science (RSS), 2021

arXiv Website
GIFT: Generalizable Interaction-aware Functional Tool Affordances without Labels.

Dylan Turpin, Liquan Wang, Stavros Tsogkas, Sven Dickinson, and Animesh Garg

Robotics: Systems and Science (RSS), 2021

arXiv Video Blog
DiSeCT: A Differentiable Simulation Engine for Autonomous Robotic Cutting.

Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos

Robotics: Systems and Science (RSS), 2021

Best Student Paper Award
arXiv Video Blog Code Website
Value Iteration in Continuous Actions, States and Time.

Michael Lutter, Shie Mannor, Jan Peters, Dieter Fox, and Animesh Garg

International Conference on Machine Learning (ICML), 2021

arXiv Website
Principled Exploration via Optimistic Bootstrapping and Backward Induction.

Chenjia Bai, Lingxiao Wang, Lei Han, Jianye Hao, Animesh Garg, Peng Liu, and Zhaoran Wang

International Conference on Machine Learning (ICML), 2021

arXiv Code
Coach-Player Multi-agent Reinforcement Learning for Dynamic Team Composition.

Bo Liu, Qiang Liu, Lei Han, Peter Stone, Animesh Garg, Yuke Zhu, and Anima Anandkumar

International Conference on Machine Learning (ICML), 2021

Long Talk
arXiv
Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning.

Anuj Mahajan, Mikayel Samvelyan, Lei Mao, Viktor Makoviychuk, Animesh Garg, Jean Kossaifi, Shimon Whiteson, Yuke Zhu, and Anima Anandkumar

International Conference on Machine Learning (ICML), 2021

arXiv
LASER: Learning a Latent Action Space for Efficient Reinforcement Learning.

Arthur Allshire, Roberto Martin-Martin, Charles Lin, Shawn Mendes, Silvio Savarese, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2021

arXiv Video Website
Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion.

Zhaoming Xie, Xingye Da, Michiel van de Panne, Buck Babich, and Animesh Garg

IEEE International Conference on Robotics and Automation (ICRA), 2021

arXiv Website
LEAF: Latent Exploration Along the Frontier.

Homanga Bharadhwaj, Animesh Garg, and Florian Shkurti

IEEE International Conference on Robotics and Automation (ICRA), 2021

arXiv Website
Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects.

Xinlei Pan, Animesh Garg, Animashree Anandkumar, and Yuke Zhu

IEEE International Conference on Robotics and Automation (ICRA), 2021

arXiv Website
Latent Skill Planning for Exploration and Transfer.

Kevin Xie, Homanga Bharadhwaj, Danijar Hafner, Animesh Garg, and Florian Shkurti

International Conference on Learning Representations (ICLR), 2021

arXiv Website
C-Learning: Horizon-Aware Cumulative Accessibility Estimation.

Panteha Naderian, Gabriel Loaiza-Ganem, Harry J. Braviner, Anthony L. Caterini, Jesse C. Cresswell, Tong Li, and Animesh Garg

International Conference on Learning Representations (ICLR), 2021

arXiv Website
Conservative Safety Critics for Exploration.

Homanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkurti, and Animesh Garg

International Conference on Learning Representations (ICLR), 2021

arXiv Website
DIBS: Diversity inducing Information Bottleneck in Model Ensembles.

Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal, Hugo Larochelle, Animesh Garg, and Florian Shkurti

Confernce on Artificial Intelligence (AAAI), 2021

arXiv
Unsupervised Disentanglement of Pose, Appearance and Background from Images and Videos.

Aysegul Dundar, Kevin J. Shih, Animesh Garg, Robert Pottorf, Andrew Tao, and Bryan Catanzaro

Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2021

arXiv Code
Causal Discovery in Physical Systems from Videos.

Yunzhu Li, Antonio Torralba, Animashree Anandkumar, Dieter Fox, and Animesh Garg

Advances in Neural Information Processing Systems 33 (NeurIPS), 2020

arXiv Website
Counterfactual Data Augmentation using Locally Factored Dynamics.

Silviu Pitis, Elliot Creager, and Animesh Garg

Advances in Neural Information Processing Systems 33 (NeurIPS), 2020

Outstanding Paper Award at Object-Oriented Learning Workshop, ICML 2020
arXiv Code
Curriculum By Smoothing.

Samarth Sinha, Animesh Garg, and Hugo Larochelle

Advances in Neural Information Processing Systems 33 (NeurIPS), 2020

Spotlight Talk
arXiv
Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion.

Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Animashree Anandkumar, Yuke Zhu, Buck Babich, and Animesh Garg

Conference on Robot Learning (CoRL), 2020

arXiv Video Blog
Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter.

Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Marcus Dominguez-Kuhne, Animesh Garg, Roberto Martín-Martín, and Silvio Saverese

IEEE International Conference on Intelligent Robots and Systems (IROS), 2020

arXiv Website
Ocean: Online Task Inference for Compositional Tasks with Context Adaptation.

Hongyu Ren, Yuke Zhu, Jure Leskovec, Anima Anandkumar, and Animesh Garg

Conference on Uncertainty in Artificial Intelligence (UAI), 2020

arXiv PDF Code
A Programmable Approach to Neural Network Compression.

Vinu Joseph, Ganesh Gopalakrishnan, Saurav Muralidharan, Michael Garland, and Animesh Garg

IEEE Micro, 2020

arXiv PDF Code Website
Semi-Supervised StyleGAN for Disentanglement Learning.

Weili Nie, Tero Karras, Animesh Garg, Shoubhik Debhath, Anjul Patney, Ankit B. Patel, and Anima Anandkumar

International Conference on Machine Learning (ICML), 2020

arXiv Code Slides Website
Angular Visual Hardness.

Beidi Chen, Weiyang Liu, Animesh Garg, Zhiding Yu, Anshumali Shrivastava, Jan Kautz, and Anima Anandkumar

International Conference on Machine Learning (ICML), 2020

arXiv PDF Slides
Controlling Assistive Robots with Learned Latent Actions.

Dylan P. Losey, Krishnan Srinivasan, Ajay Mandlekar, Animesh Garg, and Dorsa Sadigh

IEEE International Conference on Robotics and Automation (ICRA), 2020

arXiv Video Blog
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data.

Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Dieter Fox

IEEE International Conference on Robotics and Automation (ICRA), 2020

arXiv Video Website
Motion Reasoning for Goal-Based Imitation Learning.

De-An Huang, Yu-Wei Chao, Chris Paxton, Xinke Deng, Li Fei-Fei, Juan Carlos Niebles, Animesh Garg, and Dieter Fox

IEEE International Conference on Robotics and Automation (ICRA), 2020

arXiv Video
Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning.

Michelle A. Lee, Carlos Florensa, Jonathan Tremblay, Nathan Ratliff, Animesh Garg, Fabio Ramos, and Dieter Fox

IEEE International Conference on Robotics and Automation (ICRA), 2020

Best Paper Award at 2019 Neurips Workshop on Robot Learning
arXiv Video
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks.

Michelle A. Lee, Yuke Zhu, Peter Zachares, Matthew Tan, Krishnan Srinivasan, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Jeannette Bohg

IEEE Transactions on Robotics (TRO), 2020

arXiv Website
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks.

Roberto Martı́n-Martı́n, Michelle A Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg

IEEE International Conference on Intelligent Robots and Systems (IROS), 2019

arXiv Website
Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity.

Ajay Mandlekar, Jonathan Booher, Max Spero, Albert Tung, Anchit Gupta, Yuke Zhu, Animesh Garg, Silvio Savarese, and Li Fei-Fei

IEEE International Conference on Intelligent Robots and Systems (IROS), 2019

IROS Best Cognitive Robotics Paper Finalist
arXiv Blog Code Website
Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning.

De-An Huang, Danfei Xu, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei, and Juan Carlos Carlos

IEEE International Conference on Intelligent Robots and Systems (IROS), 2019

arXiv Video
Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation.

Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, and Li Fei-Fei

Conference on Robot Learning (CoRL), 2019

Oral Presentation
arXiv Website
AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers.

Andrey Kurenkov, Ajay Mandlekar, Roberto Martín-Martín, Silvio Savarese, and Animesh Garg

Conference on Robot Learning (CoRL), 2019

arXiv Blog Code Website
Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision.

Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, and Silvio Savarese

International Journal of Robotics Research (IJRR), 2019

PDF Video Website
Neural Task Graphs: Generalizing to unseen tasks from a single video demonstration.

De-An Huang, Suraj Nair, Danfei Xu, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, and Juan Carlos Niebles

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019

Oral Presentation
arXiv Video
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks.

Michelle A Lee^*, Yuke Zhu^*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Jeannette contribution)

IEEE International Conference on Robotics and Automation (ICRA), 2019

ICRA Best Paper Award and Finalist: Best Cognitive Robotics Paper
arXiv Video Website
Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter.

Mike Danielczuk, Andrey Kurenkov, Ashwin Balakrishna, Matthew Matl, Roberto Martín-Martín, Animesh Garg, Silvio Savarese, and Ken Goldberg

IEEE International Conference on Robotics and Automation (ICRA), 2019

Abs arXiv Video Website

When operating in unstructured environments such as warehouses, homes, and retail centers, robots are frequently required to interactively search for and retrieve specific objects from cluttered bins, shelves, or tables. Mechanical Search describes the class of tasks where the goal is to locate and extract a known target object. In this paper, we formalize Mechanical Search and study a version where distractor objects are heaped over the target object in a bin. The robot uses an RGBD perception system and control policies to iteratively select, parameterize, and perform one of 3 actions – push, suction, grasp – until the target object is extracted, or either a time limit is exceeded, or no high confidence push or grasp is available. We present a study of 5 algorithmic policies for mechanical search, with 15,000 simulated trials and 300 physical trials for heaps ranging from 10 to 20 objects. Results suggest that success can be achieved in this long-horizon task with algorithmic policies in over 95% of instances and that the number of actions required scales approximately linearly with the size of the heap.
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation.

Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, and Li Fei-Fei

Conference on Robot Learning (CoRL), 2018

arXiv PDF Blog Code Website
Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision.

Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, and Silvio Savarese

Robotics: Systems and Science (RSS), 2018

arXiv Video Website
SWIRL: A Sequential Windowed Inverse Reinforcement Learning Algorithm for Robot Tasks With Delayed Rewards.

Sanjay Krishnan, Animesh Garg, Richard Liaw, Brijen Thananjeyan, Lauren Miller, Florian T Pokorny, and Ken Goldberg

International Journal of Robotics Research (IJRR), 2018

link PDF
Finding It: Weakly-Supervised Reference-Aware Visual Grounding in Instructional Video.

De-An Huang, Shyamal Buch, Lucio Dery, Animesh Garg, Li Fei-Fei, and Juan Carlos Niebles

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Oral Presentation
PDF Poster Website
Neural Task Programming: Learning to generalize across hierarchical tasks.

Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, and Silvio Savarese

IEEE International Conference on Robotics and Automation (ICRA), 2018

arXiv Video Website
DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image.

Andrey Kurenkov^*, Jingwei Ji^*, Animesh Garg, Viraj Mehta, JunYoung Gwak, Christopher Choy, and Silvio contribution)

IEEE Winter Conference on Applications of Computer Vision (WACV), 2018

arXiv Website
AdaPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems.

James Harrison^*, Animesh Garg^*, Boris Ivanovic, Yuke Zhu, Silvio Savarese, Li Fei-Fei, and Marco contribution)

International Symposium on Robotics Research (ISRR), 2017

arXiv
Transition State Clustering: Unsupervised surgical trajectory segmentation for robot learning.

Sanjay Krishnan^*, Animesh Garg^*, Sachin Patil, Colin Lea, Gregory Hager, Pieter Abbeel, and Ken contribution)

International Journal of Robotics Research (IJRR), 2017

PDF Code
Weakly supervised 3D Reconstruction with Adversarial Constraint.

JunYoung Gwak^*, Christopher B Choy^*, Animesh Garg, Manmohan Chandraker, and Silvio contribution)

IEEE Conference on 3D Vision (3DV), 2017

arXiv Code
Adversarially Robust Policy Learning: Active Construction of Physically-Plausible Perturbations.

Ajay Mandlekar^*, Yuke Zhu^*, Animesh Garg^*, Li Fei-Fei, and Silvio contribution)

IEEE International Conference on Intelligent Robots and Systems (IROS), 2017

PDF Video Website
Multilateral Surgical Pattern Cutting in 2D Orthotropic Gauze with Deep Reinforcement Learning Policies for Tensioning.

Brijen Thananjeyan, Animesh Garg, Sanjay Krishnan, Carolyn Chen, Lauren Miller, and Ken Goldberg

IEEE International Conference on Robotics and Automation (ICRA), 2017

PDF Video
SWIRL: A Sequential Windowed Inverse Reinforcement Learning Algorithm for Robot Tasks With Delayed Rewards.

Sanjay Krishnan, Animesh Garg, Richard Liaw, Brijen Thananjeyan, Lauren Miller, Florian T Pokorny, and Ken Goldberg

Workshop on Algorithmic Foundations of Robotics (WAFR), 2016

PDF
Interchangeable Surgical Instrument System with Application to Supervised Automation of Multilateral Tumor Resection. .

Stephen McKinley, Animesh Garg, Siddarth Sen, David V. Gealy, Jonathan McKinley, Yiming Jen, Menglung Guo, Doug Boyd, and Ken Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2016

Best Video Award at 2015 Hamlyn Symposium
Video Website
TSC-DL: Unsupervised Trajectory Segmentation of Multi-Modal Surgical Demonstrations with Deep Learning .

Adithyavairavan Murali^*, Animesh Garg^*, Sanjay Krishnan^*, Florian Pokorny, Pieter Abbeel, Trevor Darrell, and Ken contribution)

IEEE International Conference on Robotics & Automation (ICRA), 2016

Video Code Website
Automating Multi-Throw Multilateral Surgical Suturing with a Mechanical Needle Guide and Sequential Convex Optimization.

Siddarth Sen^*, Animesh Garg^*, David Gealy, Stephen McKinley, Yiming Jen, and Ken contribution)

IEEE International Conference on Robotics & Automation (ICRA), 2016

Video Website
Tumor localization using automated palpation with Gaussian Process Adaptive Sampling.

Animesh Garg, Siddarth Sen, Rishi Kapadia, Yiming Jen, Stephen McKinley, Lauren Miller, and Ken Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2016

PDF Website
A Single-Use Haptic Palpation Probe for Locating Subcutaneous Blood Vessels in Robot-Assisted Minimally Invasive Surgery.

Stephen McKinley, Animesh Garg, Siddarth Sen, Rishi Kapadia, Adithyavairavan Murali, Kirk Nichols, Susan Lim, Sachin Patil, Pieter Abbeel, Allison M. Okamura, and Ken Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2015

Best Poster/Demo Award at ICRA 2015 Workshop on Shared Frameworks for Medical Robotics
Abs PDF

We present the design and evaluation of a novel low-cost palpation probe for Robot assisted Minimally Invasive Surgery (RMIS) for localizing subcutaneous blood vessels. It measures probe tip deflection using a Hall Effect sensor as the spherical tip is moved tangentially across a surface under automated control. The probe is intended to be single-use and disposable, built from 3D printed parts and commercially available electronics. The prototype has a cross-section of less than 15mm×10mm and fits on the end of an 8mm diameter needle driver in the Intuitive Surgical da Vinci ® Research Kit (dVRK). We report experiments for quasi-static sliding palpation with silicone based tissue phantoms with embedded cylinders as subcutaneous blood vessel phantoms. We analyzed signal-to-noise ratios with multiple diameters of silicone cylinders (1.58-4.75 mm) at varying subcutaneous depths (1-5 mm) with a range of indentation depths (0-8 mm) and sliding speeds (0.5-21 mm/s). Results suggest that the probe can detect subcutaneous structures in phantoms of diameter 2.25 mm at a depth of up to 5mm below the tissue surface.
Learning by Observation for Surgical Subtasks: Multilateral Cutting of 3D Viscoelastic and 2D Orthotropic Tissue Phantoms.

Adithyavairavan Murali, Siddarth Sen, Ben Kehoe, Animesh Garg, Seth McFarland, Sachin Patil, W Douglas Boyd, Susan Lim, Pieter Abbeel, and Ken Goldberg

IEEE International Conference on Robotics & Automation (ICRA), 2015

Finalist: Best Paper, Student Paper, and Medical Robotics Paper Award
Abs PDF Video

Automating repetitive surgical subtasks such as suturing, cutting and debridement can reduce surgeon fatigue and procedure times and facilitate supervised tele-surgery. Programming is difficult because human tissue is deformable and highly specular. Using the da Vinci Research Kit (DVRK) robotic surgical assistant, we explore a “Learning By Observation” (LBO) approach where we identify, segment, and parameterize motion sequences and sensor conditions to build a finite state machine (FSM) for each subtask. The robot then executes the FSM repeatedly to tune parameters and if necessary update the FSM structure. We evaluate the approach on two surgical subtasks: debridement of 3D Viscoelastic Tissue Phantoms (3d-DVTP), in which small target fragments are removed from a 3D viscoelastic tissue phantom; and Pattern Cutting of 2D Orthotropic Tissue Phantoms (2d-PCOTP), a step in the standard Fundamentals of Laparoscopic Surgery training suite, in which a specified circular area must be cut from a sheet of orthotropic tissue phantom. We describe the approach and physical experiments with repeatability of 96% for 50 trials of the 3d-DVTP subtask and 70% for 20 trials of the 2d-PCOTP subtask.
Transition State Clustering: Unsupervised Surgical Trajectory Segmentation For Robot Learning.

Sanjay Krishnan^*, Animesh Garg^*, Sachin Patil, Colin Lea, Gregory Hager, Pieter Abbeel, and Ken contribution)

International Symposium on Robotics Research (ISRR), 2015

Code
Evaluation of PC‐ISO for customized, 3D printed, gynecologic 192Ir HDR brachytherapy applicators.

Katherine Mellis, Timmy Siauw, Atchar Sudhyadhom, Rajni Sethi, I-Chow Hsu, Jean Pouliot, Animesh Garg, and Ken Goldberg

Journal of Applied Clinical Medical Physics (JACMP), 2015

Website
Exact Reachability Analysis for Planning Skew-Line Needle Arrangements for Automated Brachytherapy.

Animesh Garg, Timmy Siauw, Guang Yang, Sachin Patil, J Adam M Cunha, I-Chow Hsu, Jean Pouliot, Alper Atamtürk, and Ken Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2014

Abs PDF

When planning skew-line needle arrangements for automated brachytherapy, one objective is to identify a set of candidate needles that enter from a specified entry region, avoid specified organs-at-risk and sufficiently cover the target (tumor) volume. Existing methods use uniform or random sampling to generate a set of candidate needles, which may not adequately cover the target volume. In this paper we present an exact reachability analysis that can be used to guide the selection of candidate needles and to identify which subset of the target volume may not be reachable. Assuming linear needles, convex polyhedral representations of entry zone, organs-at-risk and target volume, we give an exact polynomial time algorithm for checking existence and calculation of the non-reachable set in the target volume. We perform experiments using patient data from 18 brachytherapy cases and found that 11 cases had non-empty occluded volume inside the target ranging from 0.01% to 4.3% of target volume. We also report a sensitivity study showing the change in the occluded volume with dilation of the avoidance volume and entry zone.
Customized Needle Guides for Inserting Non-Parallel Needle Arrangements in Prostate HDR Brachytherapy: A Phantom Study.

Timmy Siauw, J. Adam M. Cunha, Animesh Garg, Ken Goldberg, I Hsu, and Jean Pouliot

Brachytherapy, 2014

PDF
Robot-Guided Open-Loop Insertion of Skew-Line Needle Arrangements for High Dose Rate Brachytherapy.

Animesh Garg, Timmy Siauw, Dmitry Berenson, J Adam M Cunha, I-C Hsu, Jean Pouliot, Dan Stoianovici, and Ken Goldberg

IEEE Transactions on Automation Science and Engineering (TASE), 2013

Abs PDF

We present a study in human-centered automation that has potential to reduce patient side effects from high dose rate brachytherapy (HDR-BT). To efficiently deliver radiation to the prostate while minimizing trauma to sensitive structures such as the penile bulb, we modified the Acubot-RND 7-axis robot to guide insertion of diamond-tip needles into desired skew-line geometric arrangements. We extend and integrate two algorithms: Needle Planning with Integer Programming (NPIP) and Inverse Planning with Integer Programming (IPIP) to compute skew-line needle and dose plans. We performed three physical experiments with anatomically correct phantom models to study performance: two with the robot and one control experiment with an expert human physician (coauthor Hsu) without the robot. All were able to achieve needle arrangements that meet the RTOG-0321 clinical dose objectives with zero trauma to the penile bulb. We analyze systematic and random errors in needle placement; total RMS error for the robot system operating without feedback ranged from 2.6 to 4.3 mm, which is comparable to the RMS error of 2.7 mm obtained in an earlier study for PPI-BT treatment using a robot with 3D ultrasound feedback.
An Algorithm for Computing Customized 3D Printed Implants with Curvature Constrained Channels for Enhancing Intracavitary Brachytherapy Radiation Delivery.

Animesh Garg, Sachin Patil, Timmy Siauw, J Adam M Cunha, I Hsu, Pieter Abbeel, Jean Pouliot, and Ken Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2013

Abs PDF

Brachytherapy is a widely-used treatment modality for cancer in many sites in the body. In brachytherapy, small radioactive sources are positioned proximal to cancerous tumors. An ongoing challenge is to accurately place sources on a set of dwell positions to sufficiently irradiate the tumors while limiting radiation damage to healthy organs and tissues. In current practice, standardized applicators with internal channels are inserted into body cavities to guide the sources. These standardized implants are one-size-fits-all and are prone to shifting inside the body, resulting in suboptimal dosages. We propose a new approach that builds on recent results in 3D printing and steerable needle motion planning to create customized implants containing customized curvature-constrained internal channels that fit securely, minimize air gaps, and precisely guide radioactive sources through printed channels. When compared with standardized implants, customized implants also have the potential to provide better coverage: more potential source dwell positions proximal to tumors. We present an algorithm for computing curvature-constrained channels based on rapidly-expanding randomized trees (RRT). We consider a prototypical case of OB/GYN cervical and vaginal cancer with three treatment options: standardized ring implant (current practice), customized implant with linear channels, and customized implant with curved channels. Results with a two-parameter coverage metric suggest that customized implants with curved channels can offer significant improvement over current practice.
Initial experiments toward automated robotic implantation of skew-line needle arrangements for HDR brachytherapy.

A. Garg, T. Siauw, D. Berenson, A. Cunha, I-Chow Hsu, J. Pouliot, D. Stoianovici, and K. Goldberg

IEEE International Conference on Automation Science & Engg. (CASE), 2012

IEEE CASE Best Application Paper Award
Abs PDF Video

Automation seeks to improve the reliability and quality of processes. This study aims to automate high dose rate brachytherapy (HDR-BT), a radiation therapy that places radioactive sources at the site of the tumor using needles. Although HDR-BT has a high rate of clinical success in curing prostate cancer, it also has several side effects related to needle and dose trauma. A new planning algorithm from previous work optimizes needle arrangements using skew-lines (non-parallel, non-intersecting lines). This paper presents initial experiments towards an automated system for implanting skew-line needle arrangements computed from a planning system. We describe the interface, calibration and integration of the robotic hardware with the planning system, and present experiments using our robotic system to implant needles into anatomically-correct tissue phantoms. Results suggest that this system can achieve HDR-BT treatment objectives with reduced trauma to organs and low demands on operator skill, thus making the procedure more reliable and repeatable. In the future, we believe that robotic HDR-BT will improve overall treatment quality with reduced dependence on physician skill.
Robotic Brachytherapy Demonstration: Implant of HDR Brachytherapy Needle Configuration Computer-Optimized to Avoid Critical Structures Near the Bulb of the Penis.

JA Cunha, T Siauw, A Garg, N Zhang, K Goldberg, D Stoianovici, M Roach III, I-C Hsu, and J Pouliot

Medical Physics, 2012

PDF
Robot-guided Delivery of Brachytherapy Needles along Non-Parallel Paths to Avoid Penile Bulb Puncture.

JAM Cunha, A Garg, T Siauw, N Zhang, Y Zuo, K Goldberg, D Stoianovici, M Roach III, and J Pouliot

Radiotherapy and Oncology, 2012

PDF
Low-Cost Teleoperation of Remotely Located Actuators Based on Dual Tone Multi-Frequency Data Transfer.

Sahil Thakkar, Animesh Garg, Adesh Midha, and Prerna Gaur

2012

Abs PDF

DTMF (Dual Tone Multi Frequency) is a system of signal tones used for telecommunications. DTMF uses two tones to represent each key on the touch pad. DTMF data transfer technique has advantages such as high reliability, constant speed, and high signal to noise ratio, low cost and optimal utilization of existing resources. These features make DTMF an attractive Data Transfer Technique. It finds application in home and car security systems, robot control, SMS and voice call controlled industrial applications. In this paper, we discuss the use of DTMF data transfer in a voice call to control a toy car. Cell phone 1(CP 1), which is at user end, makes a call to cell phone 2(CP 2) at the machine end and establishes a connection. A key is pressed at the user side. Two tones corresponding to one key are encoded and sent through the cell phone network. Both tones are tapped through the earphone jack of cell phone at machine end and are decoded. The output is fed to a Micro-controller. The Micro-controller is connected to the remote control unit of the toy car, which in turn controls the motion of the car. The car moves in various directions according to the key pressed user. The electronic circuit is divided into 2 parts. The transmitter side consisting of Cell phone 1 with an inbuilt encoder and the receiver side consisting of Cell Phone 2, 8870 DTMF decoder and Atmega 16 micro-controller. The programming has been carried out on AVR Bascom®.
The Autotrix: Design and Implementation of an Autonomous Urban Driving System.

Animesh Garg, Anju Toor, Sahil Thakkar, Shiwangi Goel, Sachin Maheshwari, and Satish Chand

2012

Path Planning, Obstacle Avoidance, Monocular Vision, Autonomous Driving, Global Positioning System in Urban Driving
Abs PDF

The Autotrix is an interactive, intelligent, Autonomous Guided Vehicle (AGV) designed to serve in urban environments. Autonomous ground vehicle navigation requires the integration of many technologies such as path planning, odometry, control, obstacle avoidance and situational awareness. The objective of this project is for this prototype to navigate autonomously in an urban environment and reach its destination while detecting and avoiding obstacles on the path .This will be achieved by extracting information from multiple sources of real-time data including digital camera, GPS &ultra sonic sensors, collecting data from this extracted information, processing this data and send controlling instructions to our platform (Autotrix). The significance of this work is in presenting the methods needed for real time navigation; GPS based continuous mapping and obstacle avoidance for intelligent autonomous driving systems.
Object Identification and Mapping using Monocular Vision in an Autonomous Urban Driving System.

Animesh Garg, Anju Toor, Sahil Thakkar, Shiwangi Goel, Sachin Maheshwari, and Satish Chand

International Conference of Machine Vision, 2010

Workshops

DiffSim2Real: Deploying Quadrupedal Locomotion Policies Purely Trained in Differentiable Simulation.

Joshua Bagajo, Clemens Schwarke, Victor Klemm, Ignat Georgiev, Jean-Pierre Sleiman, Jesus Tordesillas, Animesh Garg, and Marco Hutter

2024

arXiv
Uniformly Safe RL with Objective Suppression for Multi-Constraint Safety-Critical Applications.

Zihan Zhou, Jonathan Booher, Khashayar Rohanimanesh, Wei Liu, Aleksandr Petiushko, and Animesh Garg

2023

arXiv
Uniform Priors for Data-Efficient Transfer.

Samarth Sinha, Karsten Roth, Anirudh Goyal, Marzyeh Ghassemi, Hugo Larochelle, and Animesh Garg

2022

arXiv
Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functions.

Yun-Chun Chen, Adithyavairavan Murali, Balakumar Sundaralingam, Wei Yang, Animesh Garg, and Dieter Fox

2022

arXiv
D2RL: Deep Dense Architectures in Reinforcement Learning.

Samarth Sinha, Homanga Bharadhwaj, Aravind Srinivas, and Animesh Garg

2020

arXiv Code Website
Counterfactual Data Augmentation using Locally Factored Dynamics.

Silviu Pitis, Elliot Creager, and Animesh Garg

2020

Outstanding Paper Award at Object-Oriented Learning Workshop, ICML 2020
Code Poster
Offline Policy Optimization in RL with Variance Regularizaton.

Riashat Islam, Samarth Sinha, Homanga Bharadhwaj, Samin Yeasar Arnob, Zhuoran Yang, Animesh Garg, Zhaoran Wang, Lihong Li, and Doina Precup

2020

arXiv
Combining Model-Free and Model-Based Strategies for Sample-Efficient Reinforcement Learning.

Michelle A. Lee, Carlos Florensa, Jonathan Tremblay, Nathan Ratliff, Animesh Garg, Fabio Ramos, and Dieter Fox

2019

Best paper award at the workshop
PDF
Turbulence forecasting via Neural ODE.

Gavin Portwood, Peetak Mitra, Mateus Dias Ribeiro, Tan Minh Nguyen, Balasubramanya Nadiga, Juan Saenz, Micheal Chertkov, Animesh Garg, Anima Anandkumar, Andreas Dengel, Richard Baraniuk, and David Schmidt

2019

PDF
Towards Grasp Transfer using Shape Deformation..

Andrey Kuyenkov^*, Viraj Mehta^*, Jingwei Ji, Animesh Garg, and Silvio contribution)

2017

PDF Poster
Hierarchical Task Generalization with Neural Programs.

Danfei Xu, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, and Silvio Savarese

2017
Hierarchical Task Generalization with Neural Programs.

Danfei Xu, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, and Silvio Savarese

2017

Poster
Adversarially Robust Policy Learning through Active Construction of Physically-Plausible Perturbations.

Ajay Mandlekar^*, Yuke Zhu^*, Animesh Garg^*, Li Fei-Fei, and Silvio contribution)

2017

PDF
HIRL: Hierarchical inverse reinforcement learning for long-horizon tasks with delayed rewards.

Sanjay Krishnan, Animesh Garg, Richard Liaw, Lauren Miller, Florian T Pokorny, and Ken Goldberg

2016

arXiv
On Visual Feature Representations for Transition State Learning in Robotic Task Demonstrations.

Animesh Garg^*, Sanjay Krishnan^*, Adithyavairavan Murali, Trevor Darrell, Pieter Abbeel, and Ken contribution)

2015

PDF Website
Autonomous Tumor Localization and Extraction: Palpation, Incision, Debridement and Adhesive Closure with the da Vinci Research Kit.

Stephen McKinley, Siddarth Sen, Animesh Garg, Yiming Jen, David Gealy, Pieter Abbeel, and Ken Goldberg

2015

Best Video Award
Video
Automated Delivery Instrument for Stem Cell Treatment Using the daVinci Robotic Surgical System.

Stephen McKinley, Animesh Garg, Susan Lim, Sachin Patil, and Ken Goldberg

2015

Poster

Tech Reports

Replan: Robotic replanning with perception and language models.

Marta Skreta, Zihan Zhou, Jia Lin Yuan, Kourosh Darvish, Alán Aspuru-Guzik, and Animesh Garg

2024

arXiv Website
Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Prompting.

Marta Skreta, Naruki Yoshikawa, Sebastian Arellano-Rubach, Zhi Ji, Lasse Bjørn Kristensen, Kourosh Darvish, Alán Aspuru-Guzik, Florian Shkurti, and Animesh Garg

2023

arXiv Video Website
∆-Networks for Efficient Model Patching.

Chaitanya Devaguptapu, Samarth Sinha, K J Joseph, Vineeth N Balasubramanian, and Animesh Garg

2023

arXiv
See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction.

Maria Attarian, Advaya Gupta, Ziyi Zhou, Wei Yu, Igor Gilitschenski, and Animesh Garg

2022

arXiv Website
An Adaptive Robotics Framework for Chemistry Lab Automation.

Naruki Yoshikawa, Andrew Zou Li, Kourosh Darvish, Yuchi Zhao, Haoping Xu, Alan Aspuru-Guzik, Animesh Garg, and Florian Shkurti

2022

arXiv Video Website
Auditing AI models for Verified Deployment under Semantic Specifications.

Homanga Bharadhwaj, De-An Huang, Chaowei Xiao, Animashree Anandkumar, and Animesh Garg

2021

arXiv Blog Website
Making Sense of the Robotized Pandemic Response: A Comparison of Global and Canadian Robot Deployments and Success Factors.

Tim Barfoot, Jessica Burgner-Kahrs, Eric Diller, Animesh Garg, Andrew Goldenberg, Jonathan Kelly, Xinyu Liu, Hani E. Naguib, Goldie Nejat, Angela P. Schoellig, Florian Shkurti, Hallie Siegel, Yu Sun, and Steven Waslander

2020

arXiv Blog
De-anonymization of authors through arXiv submissions during double-blind review.

Homanga Bharadhwaj, Dylan Turpin, Animesh Garg, and Ashton Anderson

2020

arXiv Blog
Video Interpolation and Prediction with Unsupervised Landmarks.

Kevin J. Shih, Aysegul Dundar, Animesh Garg, Robert Pottorf, Andrew Tao, and Bryan Catanzaro

2019

arXiv
Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning.

Richard Liaw, Sanjay Krishnan, Animesh Garg, Daniel Crankshaw, Joseph E Gonzalez, and Ken Goldberg

2017

arXiv

Theses

Optimization and Design for Automation of Brachytherapy Delivery and Learning Robot-Assisted Surgical Sub-Tasks.

Animesh Garg

University of California, Berkeley, 2016

Website
Autonomous Palpation for Tumor Localization: Design of a Palpation Probe and Gaussian Process Adaptive Sampling.

Animesh Garg

EECS Department, University of California, Berkeley, 2016

Website

Patents

Data-efficient demonstration expansion for training a generalist robotic agent.

Tongzhou Mu, Yijie Guo, Jie Xu, Ankit Goyal, Animesh GARG, and Dieter Fox

2025

US Patent App. 18/947,237 (pending)

link
Methods and systems to remotely operate robotic devices.

Ajay U Mandlekar, Yuke Zhu, Animesh Garg, Silvio Savarese, and Fei-Fei Li

2025

US Patent 12,226,913

Granted
link
Scene understanding using language models for robotics systems and applications.

Animesh Garg, Dieter Fox, Tucker Ryer Hermans, and Weiyu Liu

2024

US Patent App. 18/319,546 (pending)

link
Guided uncertainty-aware policy optimization: combining model-free and model-based strategies for sample-efficient learning.

Jonathan Tremblay, Dieter Fox, Michelle Lee, Carlos Florensa, Nathan Donald Ratliff, GARG Animesh, and Fabio Tozeto Ramos

2024

US Patent 12,109,701

Granted
link
Step discovery and localization in instructional videos using a self-supervised transformer.

Mikita Dvornik, Isma Hadji, Ran Zhang, Konstantinos Derpanis, Richard Wildes, GARG Animesh, and Allan Douglas Jepson

2024

US Patent App. 18/227,560 (pending)

link
Controlling position of robot by determining goal proposals by using neural networks.

Ajay Uday Mandlekar, Fabio Tozeto Ramos, Byron Boots, GARG Animesh, and Dieter Fox

2024

US Patent 11,958,529

Granted
link
Prompt generator for use with one or more machine learning processes.

Ishika Singh, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, GARG Animesh, and Valts Blukis

2024

US Patent App. 18/122,594 (pending)

link
Imitation learning system.

Yu-Wei Chao, De-An Huang, Christopher Jason Paxton, GARG Animesh, and Dieter Fox

2024

US Patent 11,893,468 (pending)

link
Video prediction using one or more neural networks.

Kevin Shih, Aysegul Dundar, GARG Animesh, Robert Pottorff, Andrew Tao, and Bryan Catanzaro

2024

US Patent 11,902,705

Granted
link
Predicting object models.

Fabio Tozeto Ramos, GARG Animesh, Krishna Murthy Jatavallabhula, and Miles Macklin

2023

US Patent App. 18/114,146 (pending)

link
Techniques for robot control using neural implicit value functions.

Adithyavairavan Murali, Balakumar Sundaralingam, CHEN Yun-Chun, Dieter Fox, and GARG Animesh

2023

US Patent App. 17/856,699 (pending)

link
Differentiable simulator for robotic cutting.

Eric Heiden, Fabio Tozeto Ramos, Yashraj Narang, Miles Macklin, Dieter Fox, GARG Animesh, and Mike Skolones

2022

US Patent App. 17/732,313 (pending)

link
Precision injector/extractor for robot-assisted minimally-invasive surgery.

Stephen Mckinley, GARG Animesh, Sachin Patil, Susan ML Lim, and Ken Goldberg

2022

US Patent 11,529,205

Granted
link
Language-guided distributional tree search.

Valts Blukis, Christopher Jason Paxton, GARG Animesh, and Dieter Fox

2022

US Patent App. 17/316,564 (pending)

link
Disentanglement of image attributes using a neural network.

Aysegul Dundar, Kevin Jonathan Shih, GARG Animesh, Robert Thomas Pottorff, Andrew Tao, and Bryan Christopher Catanzaro

2022

US Patent App. 17/678,666 (pending)

link
Online task inference for compositional tasks with context adaptation.

GARG Animesh, Hongyu Ren, Yuke Zhu, and Anima Anandkumar

2022

US Patent App. 16/945,753 (pending)

link
Attribute-aware image generation using neural networks.

Weili Nie, Tero Tapani Karras, GARG Animesh, Shoubhik Debnath, Anjul Patney, and Anima Anandkumar

2022

US Patent App. 16/925,085 (pending)

link
Bayesian optimization of sparsity ratios in model compression.

Saurav Muralidharan, Vinu Joseph, GARG Animesh, and Michael Garland

2021

US Patent App. 16/785,044 (pending)

link
Patient-specific temporary implants for accurately guiding local means of tumor control along patient-specific internal channels to treat cancer.

Jean Pouliot, Ken Goldberg, I-Chow Hsu, J Adam M Cunha, GARG Animesh, Sachin Patil, Pieter Abbeel, and Timmy Siauw

2019

US Patent 10,286,197

Granted
link