Breadcrumbs for your Deep Learning Model: Following Provenance Traces with DLProv

To train a Deep Learning (DL) model, a workflow must be executed with four well-defined activities: (i) Acquiring data, (ii) Preprocessing, (iii) Splitting and balancing the dataset, and (iv) Building and training the model. After generating several DL models, they undergo a process called model selection. After being selected, the DL model is put into a production environment to make predictions on new data. One of the challenges in supporting these analyses is related to providing relationships between candidate models, their datasets for train, test, and validation, input data, and other derivations paths. These relationships are also essential for trust, reproducibility, and evolution of the selected model. While existing solutions allow monitoring and analyzing the artifacts generated throughout the DL workflow, they often fail to establish relationships for supporting data derivation within the DL workflow. DLProv is a provenance-centric service to support DL workflow analyses and reproducibility. DLProv captures provenance data and exports provenance graphs for DL model reproducibility. DLProv is W3C PROV compliant, ensuring standardized prospective and retrospective provenance, and enables provenance capture in arbitrary execution frameworks.