Reinforcement Learning from Human Judgments

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

CC BY

Source arXiv

Machine Learning

Human-Computer Interaction

Multiagent Systems

Reinforcement Learning from Human Judgments

Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback

We started by training an agent via behavioural cloning (BC) from a dataset of human-human interactions, as in

Reinforcement learning from human feedback (rlhf) is an important goal in artificial intelligence to create agents that can both interact naturally with humans and learn from their feedback.

Authors

Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, George Powell, Adam Santoro, Guy Scully, Sanjana Srivastava, Tamara von Glehn, Greg Wayne, Nathaniel Wong, Chen Yan, Rui Zhu

Related Topics

Machine learning

Reinforcement learning

Imitation learning

Imitation learning agents

Inter-temporal bradley-terry" (ibt) modelling

Ibt reward models

Inter-temporal bradley-terry reward model

Imitation-based agent

Imitation-trained agent

Multi-modal interactive agent

Solver agent

Introduction

Reinforcement learning (rl) relies on precise reward functions that are cheap to query.

However, such reward functions are not known for many of the open-ended behaviours that people routinely engage in.

For example, consider an everyday interaction, such as asking someone to set a cup down near you.

To instill a broader range of expert-level capabilities with rl, we therefore need a method to produce precise, queryable reward functions that respect the complexity, variability, and ambiguity of human behaviour.

We introduce the inter-temporal bradley-terry reward model to capture human feedback on individual episodes of two-player interactions in a 3d simulated world.

We show that the reward model improves agent behaviour compared to baselines of imitation-learning agents.

These improvements apply both to interactions involving mobile manipulation (environmental locomotion and dexterous physical interaction) as well as tasks involving bi-directional verbal interaction like question-answering.

Result

We present a method to train an imitation-based agent via behavioural cloning from our dataset of human-human interactions in the playhouse environment @cite0.

We then ask humans to interact with this imitation-trained agent in real-time and evaluate their successes in each interaction.

Agents trained with reinforcement learning behavioral cloning obtained a higher success rate compared to agents trained via imitation alone (bc), achieving 92% of performance relative to that achieved by humans in similar positions.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

CC BY

Source arXiv

Machine Learning

Human-Computer Interaction

Multiagent Systems

Reinforcement Learning from Human Judgments

Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback

We started by training an agent via behavioural cloning (BC) from a dataset of human-human interactions, as in

Reinforcement learning from human feedback (rlhf) is an important goal in artificial intelligence to create agents that can both interact naturally with humans and learn from their feedback.

Authors

Related Topics

Machine learning

Reinforcement learning

Imitation learning

Imitation learning agents

Inter-temporal bradley-terry" (ibt) modelling

Ibt reward models

Inter-temporal bradley-terry reward model

Imitation-based agent

Imitation-trained agent

Multi-modal interactive agent

Solver agent

Introduction

Reinforcement learning (rl) relies on precise reward functions that are cheap to query.

However, such reward functions are not known for many of the open-ended behaviours that people routinely engage in.

For example, consider an everyday interaction, such as asking someone to set a cup down near you.

We introduce the inter-temporal bradley-terry reward model to capture human feedback on individual episodes of two-player interactions in a 3d simulated world.

We show that the reward model improves agent behaviour compared to baselines of imitation-learning agents.

Result

We present a method to train an imitation-based agent via behavioural cloning from our dataset of human-human interactions in the playhouse environment @cite0.

We then ask humans to interact with this imitation-trained agent in real-time and evaluate their successes in each interaction.

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Recommended Follows

◐ Latest Activity