Mixing Biology and Computer Science Concepts to Design Resilient Data Lakes

Abstract

Data lakes appeared a few years ago, introduced in particular to meet the challenges of storing and exploiting IoT data. They were first considered as a new technical and commercial tool, sold by the main database software editors. More recently, they have become the subject of research, in particular to define what a data lake should be, what it should provide in terms of services, and how it should be built. In this work, we have tried to return to the origins of data lakes, starting from the name “lake”. We present here how we worked, between biologists and computer scientists, to understand the links between natural and data lakes. In this article, we first explore the links between the disciplines of biology and computer science before declining these links for the particular theme of lakes. This could appear as a work of transferring knowledge from biology to computer science, and a “simple” application of the concepts. However, we had to interact and understand each other’s concepts and issues to align a possible comparison between the disciplines, for example to determine at what scale to establish the biological comparison, from DNA to the more macro system of the animal and plant ecosystem present in a natural lake. For this reason, we are inspired by a hybrid method based on ecological and logistical network topology to propose the resilient structure for the data lake. Thus, we use the Ecological Network Analysis (ENA) as a bio-inspired method and Graph theory as a logistical-inspired framework to study the interdisciplinary resilience strategies for the data lake network.