Language Model

Language Model

Dataset

Recommendation

Analysis

What Does BERT Look at? An Analysis of BERT’s Attention (TODO)

Kevin Clark, Urvashi Khandelwal, Omer Levy, Christopher D. Manning
Computer Science Department, Stanford University
Facebook AI Research ACL’19

Method

A Neural Probabilistic Language Model

Yoshua Bengio, Rejean Ducharme, Pascal Vincent, Christian Jauvin Departement d’Informatique et Recherche Operationnelle JMLR’03

Efficient Estimation of Word Representations in Vector Space

Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
Google
Computer Science’13

Distributed Representations of Words and Phrases and their Compositionality

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean
Google
NIPS’13

GloVe: Global Vectors for Word Representation (TODO)

Jeffrey Pennington, Richard Socher, Christopher D. Mannning
Computer Science Department, Stanford University
EMNLP’14

Deep contextualized word representations

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
Allen Institue for artificial Intelligence
ACL’18

Improving Language Understanding by Generative Pre-Training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever
OpenAI

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
Google AI Language
NAACL’19

Paper

Analysis

Method