Design Of Specialized Storage for Heterogeneous Project Data

This article describes an approach for designing a storage system that will allow persistence of large number of heterogeneous entities, used to describe software development process in various process supporting tools (bug trackers, repositories, wiki engines, etc). Goal of this work is to extract core semantics essences from entities like issues, code commits, builds in continuous integration systems and others and persist them together in one indexable, integrated and searchable systems disregarding from which tool each of them came from. Taking into high level of variety in data shapes and forms, requirement to perform complicated cross-entities queries and potentially high volumes of data -designing a storage for such systems turns into a challenging task.