AI tools for generating Digital Heritage Twins enhancing storytelling in educational games

Digitization of Cultural Heritage (CH) artifacts contributes to the preservation of their historical and societal significance. Their digital counterparts can be widely disseminated through Building Information Modeling applications, historical databases, serious video games, digital educational programs, etc. In this context, we present a (semi-) automated user-friendly framework for 3D asset acquisition, enhancement, and semantic enrichment. These include computer vision modules developed for super-resolution and style transfer on 2D video frames, which are employed to generate 3D assets via multi-view reconstruction. The proposed framework further supports an Active Learning (AL) process that produces images from novel viewing angles inside virtual environments. These views are exploited to improve previously generated 3D assets. 3D content creators can utilize the proposed framework to digitize new or modify and enhance existing assets regardless of size or shape with minor editing from a single video. Several image quality assessment metrics indicate the validity of our methodology.