Towards Intelligent Reading through Multimodal and Contextualized Word LookUp

Abstract

This paper presents Koob, an eBook Reader app that coalesces three key ideas to enhance students’ language learning, specifically for ambiguous words. The first idea is to improve the effectiveness of word lookup functionality through contextualization – by incorporating word sense disambiguation (WSD) techniques to show the contextually relevant definition at the top. The second idea is to augment WSD results with crowd-sourcing solutions. The last idea seeks to reinforce students’ learning by augmenting textual information with a visual aid, pictures related to the word, as part of the word lookup functionality. An empirical evaluation demonstrates that existing WSD techniques can successfully employed to dynamically reorder definitions such that the most relevant definition is at the top of the list for more than 80% of the instances.