Using TF-IDF on Kisan Call Centre Dataset for Obtaining Query Answers

Abstract

Getting semantic similarity in short texts plays an important role for many tasks in the field of information retrieval. This helps in getting search results, fetching answers to queries, building summary of documents etc. We present an approach for manually and automatically getting answers to the different problems and queries of the farmers for their day to day agricultural work. Using this approach, we can provide a query to the model, to find relevant questions asked for that query and their possible answers. We have first preprocessed the data and converted to a similarity matrix which we save in a database using mongoDb. By taking the saved data from database we trained the model to get the information of the query based on similarity between the sentences of the queries, and then the application will find the best possible answer according to the similarity. We will be using Term-frequency-inverse document frequency (TF-IDF) to find the similar queries. With TF-IDF, every word is given weight, the TF-IDF is measured by frequency the relevance is not taken into consideration for this model. This information can be used in trainings to boost call agent effectiveness and improve the customer experience. As an added bonus, more effective communication reduces handle time and operating costs.