Automated coding with transformer models

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Source arXiv

Machine Learning

Computation and Language

Automated coding with transformer models

BERT for Long Documents: A Case Study of Automated ICD Coding

Model architecture proposed for handling long text inputs.

In this paper we present a simple and scalable method to process long text with the existing transformer models such as bert.we show that this method significantly improves the previous results reported for transformer models in automated icd coding, and is able to outperform one of the prominent cnn-based methods.

Authors

Arash Afkanpour, Shabir Adeel, Hansenclever Bassani, Arkady Epshteyn, Hongbo Fan, Isaac Jones, Mahan Malihi, Adrian Nauth, Raj Sinha, Sanjana Woonna, Shiva Zamani, Elli Kanal, Mikhail Fomitchev, Donny Cheung

Related Topics

Classifier

Convolutional neural networks

BERT

Transformer models

CNN

Recurrent neural networks

Bert models

Cnn-based models

Transformer architecture

Transformer-based language models

Cnn-based methods

Bert language model

Transformer-based solutions

Introduction

The international classification of diseases (icd) codes provide a standard way of keeping track of diagnoses and procedures during a patient visit.

These codes are used worldwide for epidemiological studies, billing and reimbursement, and research in health care.

Assigning codes to a clinical note, such as a discharge summary, is done by professional medical coders.

On the surface, the problem can be considered as a multi-label document classification problem.

However, there are aspects of the problem that make it particularly challenging.

The primary challenge is that there are tens of thousands of classes.

Many of the codes are related to rare conditions and are mentioned infrequently in text, which makes it difficult to train a reliable classifier for them.

Due to these challenges there is an incentive to automate the coding process.

Examining the literature of automated medical coding methods reveals that transformer-based solutions fail to outperform convolutional neural networks (cnn) based solutions.

We present new empirical evidence that shows even the standard transformer models can outperform some of the previous prominent methods and are a viable solution for automated medical coding.

Read the Paper

Read Paper