Research on Segmentation in Deleting Repeat Questions

Abstract

It is an effective method that deletes repeat questions to provide quality of question bank. The main way is semantic similarity computing to solve repeat questions deleting but which depends on correct segmentation of text. A highly efficient and accurate method to segment words is proposed which combines word segmentation based on dictionary with reverse maximum matching and based on statistics with suffix arrays. And the idea references design of memory. The accuracy of word segmentation may achieve 97% and have 10% of increase in constructing question bank for principles of computer organization.