language models - 42Papers

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Machine Learning

Artificial Intelligence

Computation and Language

Cryptography and Security

Red Teaming for Harmful Language Models

Red Teaming Language Models with Language Models

Language models (lms) are often unable to be deployed because of their potential to harm users in hard-to-predict ways.

In this work, we find cases where a target lm behaves in a harmful way, by generating test cases using another lm.

CC BY

Computation and Language

Combining Neural Networks and Episodic Memory in a Language Model

Adaptive Semiparametric Language Models

We present a language model that combines a large parametric neural network(i.e., a transformer) with a non-parametric episodic memory component in an integrated architecture.

Our model uses extended short-term context by caching local hidden states and global long-term memory by retrieving a set of nearest neighbor tokens at each timestep.

CC BY

Artificial Intelligence

Computation and Language

Neural Language Models Are Plagiarizing Training Datasets

Do Language Models Plagiarize?

Language models do not fully understand the context and sensitivity of text and can sometimes memorize phrases or sentences present in their training sets.

In this paper, we investigate whether they not only memorize but also plagiarize training samples when generating artificial texts.

Added to collectionTop 100 Papers By Signal Trends

CC BY

Artificial Intelligence

Computation and Language

Recitation-augmented gEneration for Knowledge-Intensive NLP

Recitation-Augmented Language Models

We propose a new paradigm to help large language models generate more accurate factual knowledge without retrieving from an external corpus, called recitation-augmented generation (recite).

We show that by utilizing recitation as the intermediate step, a recite-and-answer scheme can achieve new state-of-the-art performance in various closed-book question answering (cbqa) tasks.

CC BY

Artificial Intelligence

Computation and Language

The Spanish RoBERTa-base and RoBERTa-large models

Spanish Language Models

We present the pre-trained and performance evaluations of the large and base robertson-robertson - alpha (robertson-robertson - alpha) models, as well as the corresponding performance evaluations.

Bookmarked bySiddhesh

CC BY

Artificial Intelligence

Computation and Language

Ontology-based Knowledge Extraction with Masked Language Models

Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases

We conduct a rigorous study to explore the underlying predicting mechanisms of pre-trained masked language models (mlms) over different extraction paradigms.

By investigating the behaviors of mlms, we find that previous decent performancemainly owes to the biased prompts which overfit dataset artifacts.

Added to collectionsLLM Prompting and Top 100 Papers By Signal Impressions

Computation and Language

Augmented Language Models: a Survey

This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially complex task into simpler

More...

Liked byBonhomiegandalf

CC BY

Machine Learning

Artificial Intelligence

Computation and Language

Using GPT-3 as a Proxy Reward Function for Reinforcement Learning

Reward Design with Language Models

Reinforcement learning (rl) is challenging since specifying human notions of desired behavior may be difficult via reward functions or require many expert demonstrations.

This paper explores how to simplify reward design by prompting a large language model (llm) such as gpt-3 as a proxy reward function, where the user provides a textual prompt containing a few examples(few-shot) or a description (zero-shot) of the desired behavior.

Computation and Language

On Language Models for Creoles

Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with m

More...

CC BY

Artificial Intelligence

Computation and Language

Tool Augmented Language Models

TALM: Tool Augmented Language Models

We present a text-only approach to augment language models with non-differentiable tools, and an iterative"self-play"technique to bootstrap performance starting from few tooldemonstrations.

At a given model scale, tool augmented language models significantly outperform non-augmented language models.

Artificial Intelligence

Computation and Language

Language Models as Fact Checkers?

Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data. In this paper, we leverage this implicit knowledge to create an effec

More...

Computation and Language

Efficient Nearest Neighbor Language Models

Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints. W

More...

Machine Learning

Programming Languages

Stats Machine Learning

Structural Language Models of Code

We address the problem of any-code completion - generating a missing piece of source code in a given program without any restriction on the vocabulary or structure. We introduce a new approach to any-

More...

CC BY

Computation and Language

Re-using Activations from Previous Models for Efficient Training and Inference

Embedding Recycling for Language Models

Embedding recycling (er):re-using activations from previous model runs when performing training or inference.

We show that our method provides a 100% speedup during training and a 55-86% speedup for inference for text classification and entity recognition tasks in the scientific domain.

Machine Learning

Artificial Intelligence

Computation and Language

Language Models as Knowledge Embeddings

Knowledge embeddings (KE) represent a knowledge graph (KG) by embedding entities and relations into continuous vector spaces. Existing methods are mainly structure-based or description-based. Structur

More...

Machine Learning

Artificial Intelligence

Computation and Language

Editing Factual Knowledge in Language Models

The factual knowledge acquired during pretraining and stored in the parameters of Language Models (LM) can be useful in downstream tasks (e.g., question answering or textual inference). However, some

More...

Added to collectionTop 100 Trending AI & Computer Science Papers

Computation and Language

Language Models are Few-Shot Learners

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in a

More...

Machine Learning

Computation and Language

Generating Datasets with Pretrained Language Models

To obtain high-quality sentence embeddings from pretrained language models, they must either be augmented with additional pretraining objectives or finetuned on large amounts of labeled text pairs. Wh

More...

Added to collectionTop 100 Papers By Signal Impressions

Multiagent Systems

Computation and Language

Language Models as Agent Models

Language models (LMs) are trained on collections of documents, written by individual human agents to achieve specific goals in an outside world. During training, LMs have access only to text of these

More...

Home

Discover

Bookmarks

◐ Latest Activity

◐ Recommended Follows