Denoising Sequence-to-Sequence Modeling for Removing Spelling Mistakes

Abstract

Rule-based spelling correction system focused on finding the most matched word with the misspelled word. But this approach does not work well inside a sentence with multiple errors that has a combination of possible correct words to replace but only one current sentence. Replacing each word individually will result in errors. So, the spelling corrector system must understand the context of the sentence including the tense and gender of the subject and so on. The most popular example of typing mistake correction is the one Google provides in their search engine. It was introduced quite a while ago but no such good performing system is developed by anyone else. In this work, we have proposed a spelling correction system using deep learning. The basic intuition of our approach is taken from denoising autoencoder. Here we have trained the model with noisy input generated by changing, removing or adding extra character at random position inside the sequence. The job of the model is to model this noisy input to output the original errorless sequence. We have experimented with large English dataset and reported the performance in terms of character level accuracy. The proposed model has shown impressive results in correcting the spelling mistakes.