JEMA-SINDYc: End-to-end Control using Joint Embedding Multimodal Alignment in Directed Energy Deposition

Abstract

Bringing AI models from digital to real-world applications presents significant challenges due to the complexity and variability of physical environments, often leading to unexpected model behaviors. We propose a framework that learns to translate images into control actions by modeling multimodal real-time data and system dynamics. This end-to-end controller offers enhanced explainability and robustness, making it well suited for complex manufacturing processes. This end-to-end framework differs from traditional approaches that rely on manually engineered features by learning complex relationships directly from raw data. Labels are only required during training to define the observable feature to be optimized. This adaptability significantly reduces development time and enhances scalability across varying conditions. This approach was tested in the Directed Energy Deposition (L-DED) process, a laser-based metal additive manufacturing technique that produces near-net-shape parts with exceptional energy efficiency and flexibility in both geometry and material selection. L-DED is inherently complex, involving multiphysics interactions, multiscale phenomena, and dynamic behaviors, which make modeling and optimization difficult. Effective control is crucial to ensure part quality in this dynamic environment. To address these challenges, we introduce Joint Embedding Multimodal Alignment with Sparse Identification of Nonlinear Dynamics for control (JEMA-SINDYc). It combines an image-based JEMA monitoring model, which predicts the melt pool size using only the on-axis sensor with labels provided by the off-axis camera, and dynamic modeling using SINDYc, which acts as a World Model by capturing system dynamics within the embedding space. Together, these components enable the development of an advanced controller trained via Behavioral Cloning. This approach improves part quality by minimizing porosity and reducing deformation. Thin-walled cylindrical parts were produced to validate and compare this approach with other control strategies, including both open-loop and JEMA-PID. This framework improves the reliability of AI-driven manufacturing and enhances control of complex industrial processes, potentially enabling wider adoption of the process.