Interactive, Real-Time, Natural Language-Instructable Robots

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Source arXiv

Machine Learning

Artificial Intelligence

Robotics

Interactive, Real-Time, Natural Language-Instructable Robots

Interactive Language: Talking to Robots in Real Time

We present a framework for building interactive, real-time, naturallanguage-instructable robots in the real world, and we open source related assets (dataset, environment, benchmark, and policies).

Trained with behavioralcloning on a dataset of hundreds of thousands of language-annotated trajectories, a produced policy can proficiently execute an order of magnitude more commands than previous works : specifically we estimate a 93.5% success rate on a set of 87,000 unique natural language strings specifying raw end-to-end visuo-linguo-motor skills in the real world.

We find that the same policy is capable of being guided by a human via real-time language to address a wide range of precise long-horizon rearrangement goals, e.g.

Authors

Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, Pete Florence

◐ Recommended Follows

◐ Latest Activity