CBOS: Continuos bag of sentences for learning sentence embeddings

Abstract

There has been recent work learning distributed sentence representations, which utilise neighbouring sentences as context for learning the embedding of a sentence. The setting is reminiscent of training word embeddings, yet no work has reported a baseline using the same training objective as learning word vectors. We fill this gap by empirically investigating the use of a Continuous Bag-of-Word (CBOW) objective, predicting the current sentence using its context sentences. We name this method a Continuous Bag-of-Sentences (CBOS) method. Results on standard benchmark show that CBOS is a highly competitive baseline for training sentence embeddings, outperforming most existing methods for text similarity measurement.