LMP: Language Model Programming

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byVik

CC BY

Source arXiv

Artificial Intelligence

Computation and Language

LMP: Language Model Programming

Prompting Is Programming: A Query Language For Large Language Models

Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation.

However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task-and model-specific programs have to be implemented, which may still require ad-hoc interaction.

Based on this, we present the novel idea of language model programming (lmp), which generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting.

We implement lmql (short for language model query language), which leverages the constraints and controlflow from an lmp prompt to generate an efficient inference procedure that minimizes the number of expensive calls to the underlying language model.

Our evaluation shows that we retain or increase the accuracy on several downstream tasks, while also significantly reducing the required amount of computation or cost in the case of pay-to-use apis (13-85% cost savings).

Authors

Luca Beurer-Kellner, Marc Fischer, Martin Vechev

Related Topics

Introduction

Large language models (lm) have proven successful at various language-based tasks such as machine translation, text summarization, question answering, reasoning, code generation from text and many more.

Due to these results lm have become popular beyond the machine learning community and are slowly being integrated into many applications.

However, making the best use of these models and keeping up as new models are released requires a deep understanding of their internals, as well as the use of vendor-specific libraries and implementations.

In this work, we propose the idea of language model programming, extending on natural language prompting by additionally allowing lightweight scripting and constraining of outputs.

This facilitates a front-end/back-end separation for language model prompting, i.e.

Allows a user to specify complex interactions, control flow, and constraints without requiring knowledge of an lm’s internals such as tokenization, implementation, and architecture.

Further, the constructed programs remain agnostic concerning the underlying lm, greatly improving portability.

Result

In this work, we introduce the concept of language model programming, a novel way to interact with (large) language models.

We present, a high-level query language, offering a concise and intuitive syntax.

Implements purpose-designed evaluation semantics, which enable efficient query execution.

We substantiate this claim in a series of case studies, where we demonstrate that complex, state-of-the-art prompting techniques can be implemented as intuitive, concise and efficient programs that reduce (compute) costs by up to @xmath0.0.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byVik

CC BY

Source arXiv

Artificial Intelligence

Computation and Language

LMP: Language Model Programming

Prompting Is Programming: A Query Language For Large Language Models

Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation.

Authors

Luca Beurer-Kellner, Marc Fischer, Martin Vechev

Related Topics

Introduction

Due to these results lm have become popular beyond the machine learning community and are slowly being integrated into many applications.

In this work, we propose the idea of language model programming, extending on natural language prompting by additionally allowing lightweight scripting and constraining of outputs.

This facilitates a front-end/back-end separation for language model prompting, i.e.

Allows a user to specify complex interactions, control flow, and constraints without requiring knowledge of an lm’s internals such as tokenization, implementation, and architecture.

Further, the constructed programs remain agnostic concerning the underlying lm, greatly improving portability.

Result

In this work, we introduce the concept of language model programming, a novel way to interact with (large) language models.

We present, a high-level query language, offering a concise and intuitive syntax.

Implements purpose-designed evaluation semantics, which enable efficient query execution.

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Latest Activity

◐ Recommended Follows