Grammar Error Correction using Masked Language Models

Propose a system for grammatical error correction using a masked language model. Introduce a framework consisting of two tasks: tagging, which chooses and arbitrarily reorders a subset of input tokens to keep, and insertion, which infills the missing tokens with another pretrained masked language model.