Database technologies in the world of big data

Abstract

Now we have a number of database technologies called usually NoSQL, like key-value, column-oriented, and document stores as well as search engines and graph databases. Whereas SQL software vendors offer advanced products with the capability to handle highly complex queries and transactions, NoSQL databases share rather characteristics concerning scaling and performance, as e.g. auto-sharding, distributed query support, and integrated caching. Their drawbacks can be a lack of schema or data consistency, difficulty in testing and maintaining, and absence of a higher query language. Complex data modelling and the SQL language as the only access tool to data are missing here. On the other hand, last studies show that both SQL and NoSQL databases have value for both for transactional and analytical Big Data. Top databases providers offer rearchitected database technologies combining row data stores with columnar in-memory compression enabling processing large data sets and analytical querying, often over massive, continuous data streams. The technological progress led to development of massively parallel processing analytic databases. The paper presents some details of current database technologies, their pros and cons in different application environments, and emerging trends in this area.