Cleaning Collections Data Using OpenRefine

Abstract

Collection maintenance, including weeding, is a key component of my position as an academic science librarian. In an ideal world we receive perfect data that are clean and ready to use. But unfortunately, that is not always the case. In large deselection projects you might receive holdings and circulation records in separate files which, once combined, may contain many undesired duplicated line items. I will demonstrate how you can effectively and quickly use the facet row feature in OpenRefine to deduplicate data. The benefit of this method is that you select which of the duplicated items will be kept and which will be deleted. Once OpenRefine is downloaded and opened, you work in a web user interface to upload your data, clean and transform the data, and then download from the browser to a CSV file in Excel. With practice, I have found that this only takes a few minutes for thousands of line items, and ensures I am able to select the data I want.