An Arabic Word Similarity Measure for Semantic Conversational Agents

Abstract

Word similarity measures are used to measure the semantic relatedness between two words. Whereas traditional English measures exist, relatively little research has been undertaken in developing such measures for Modern Standard Arabic largely due to the linguistic challenges of the language. Domain coverage is also an issue when looking to select the best measure for incorporation into a semantic conversational agent. The information source used within the measure should be general yet capable of dealing with domain specific language to ensure robust and appropriate responses. This paper proposes a word similarity measure that utilises the length, and depth of the words from within a domain specific lexical tree that is used as the information source. The measure is compared with an existing Arabic word similarity measure through evaluation on a generic published dataset and the results show the new measure gives high correlation with human ratings.