Using Clustering and Text Mining to Create a Reference Price Database

Abstract

Since 2004, Brazil’s Office of the Comptroller General (CGU) has been publishing several data related to government expenditures in the Transparency Portal. In 2010, CGU started publishing daily every financial statement produced by the Federal Government. Nevertheless, inconsistencies which hinder accountability have been found in this data base. This paper presents how CGU uses clustering and text mining techniques to retrieve essential information for a good accountability, which includes what was bought, the price paid per item, a price reference per product, etc. This analysis has allowed CGU to draw some preliminary conclusions which are presented as a means to illustrate the research results. Finally, this information will eventually be incorporated in the Transparency Portal, allowing every citizen to understand how much the Government is really paying, in general, for products. Thus, improving social control and providing a solid accountability not only to CGU, as an internal control agency, but also to Brazil’s citizens who, in the end, are the ones paying the bill.