Contrastive Authoring and Reviewing for Machine-Generated Narrative Text

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

CC BY

Source arXiv

Computation and Language

Contrastive Authoring and Reviewing for Machine-Generated Narrative Text

Cut the CARP: Fishing for zero-shot story evaluation

We present contrastive authoring and reviewing pairing (carp): a scalable, efficient method for performing qualitatively superior, zero-shot evaluation of stories.

We show a strong correlation between human evaluation of stories and those of carp.

Model outputs more significantly correlate with corresponding human input than those language-model based methods which utilize finetuning orprompt engineering approaches.

We also present and analyze the story-critiquedataset, a new corpora composed of 1.3 million aligned story-critique pairsderived from over 80,000 stories.

Authors

Shahbuland Matiana, JR Smith, Ryan Teehan, Louis Castricato, Stella Biderman, Leo Gao, Spencer Frazier

◐ Recommended Follows

◐ Latest Activity