Contrastive Decoding

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

CC BY

Source arXiv

Machine Learning

Artificial Intelligence

Computation and Language

Contrastive Decoding

Contrastive Decoding: Open-ended Text Generation as Optimization

Contrastive decoding exploits the contrasts between expert and amateur LM of different sizes by choosing tokens that maximize their log-likelihood difference. CD produces high-quality text that amplifies the good expert behavior and diminishes the undesired amateur behavior.

We propose contrastive decoding (cd), a search objective that returns the difference between likelihood under a large language model (called the expert, e.g.

Opt-13b) and a small language model (called the amateur, e.g.

Opt-125m).

Contrastive decoding is inspired by the fact that the failures of larger language models are even more prevalent in smaller models, and that this difference signals exactly which texts should be preferred.

Contrastive decoding requires zero training and produces higher quality text than decoding from the larger model alone.

It also generalizes across model types (opt and gpt2) and significantly outperforms four strong decoding algorithms in automatic and human evaluations.

Authors

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis

Related Topics

Introduction

Open-ended text generation aims to craft fluent and coherent textual continuations of given prompts, laying foundations for various downstream applications such as writing assistance and story generation.

The canonical approaches often sample from large pre-trained language models, but the generated text is prone to incoherence and topic drift as unlucky sampling choices compound over long sequences.

We propose a new search objective, contrastive decoding (cd), that can generate fluent and diverse text without compromising coherence.

Contrastive decoding exploits the contrasts between expert and amateur language models of different sizes by choosing tokens that maximize their log-likelihood difference, subject to plausibility constraints which restrict the search space to tokens with sufficiently high probability under the expert language model.

Our contrastive decoding method significantly improves the coherence of generated text, and improves or maintains the same fluency levels, according to both human evaluation and automatic metrics.

Furthermore, we provide two more interpretations of contrastive decoding in, from both pragmatics and distinguishability perspectives.

Result

We introduce contrastive decoding (cd), a search based approach to text generation that explicitly rewards coherence, at the expense of topic drift.

Contrastive decoding searches to optimize our objective, avoiding the topic drift that can happen by chance in sampling-based generation techniques.

We conduct human evaluation to compare our contrastive decoding approaches against nucleus sampling (the canonical method that scores high under mauve) and typical decoding (the winning method for diversity metrics) across three different domains (news, wikinews, stories) and three model sizes (1.5b, 6.7b, 13b).

On average across settings, evaluators preferred contrastive decoding 2.6x more than nucleus sampling and 6.4x more than typical decoding when evaluating coherence.

Contrastive decoding achieves comparable or slightly worse diversity compared to nucleus and typical sampling, but it achieves substantially better diversity than other search based methods.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

CC BY

Source arXiv

Machine Learning

Artificial Intelligence

Computation and Language

Contrastive Decoding

Contrastive Decoding: Open-ended Text Generation as Optimization

We propose contrastive decoding (cd), a search objective that returns the difference between likelihood under a large language model (called the expert, e.g.

Opt-13b) and a small language model (called the amateur, e.g.

Opt-125m).

Contrastive decoding requires zero training and produces higher quality text than decoding from the larger model alone.

It also generalizes across model types (opt and gpt2) and significantly outperforms four strong decoding algorithms in automatic and human evaluations.

Authors

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis

Related Topics

Introduction

The canonical approaches often sample from large pre-trained language models, but the generated text is prone to incoherence and topic drift as unlucky sampling choices compound over long sequences.

We propose a new search objective, contrastive decoding (cd), that can generate fluent and diverse text without compromising coherence.

Our contrastive decoding method significantly improves the coherence of generated text, and improves or maintains the same fluency levels, according to both human evaluation and automatic metrics.

Furthermore, we provide two more interpretations of contrastive decoding in, from both pragmatics and distinguishability perspectives.

Result

We introduce contrastive decoding (cd), a search based approach to text generation that explicitly rewards coherence, at the expense of topic drift.

Contrastive decoding searches to optimize our objective, avoiding the topic drift that can happen by chance in sampling-based generation techniques.

On average across settings, evaluators preferred contrastive decoding 2.6x more than nucleus sampling and 6.4x more than typical decoding when evaluating coherence.

Contrastive decoding achieves comparable or slightly worse diversity compared to nucleus and typical sampling, but it achieves substantially better diversity than other search based methods.

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Recommended Follows

◐ Latest Activity