Optimizing Fuel Injection Timing for Multiple Injection Using Reinforcement Learning and Functional Mock-up Unit for a Small-bore Diesel Engine

Reinforcement learning (RL) is a computational approach to understanding and automating goal-directed learning and decision-making. The difference from other computational approaches is the emphasis on learning by an agent from direct interaction with its environment to achieve long-term goals [1]. In this work, the RL algorithm was implemented using Python. This then enables the RL algorithm to make decisions to optimize the output from the system and provide real-time adaptation to changes and their retention for future usage. A diesel engine is a complex system where a RL algorithm can address the NOx–soot emissions trade-off by controlling fuel injection quantity and timing. This study used RL to optimize the fuel injection timing to get a better NO–soot trade-off for a common rail diesel engine. The diesel engine utilizes a pilot–main and a pilot–main–post-fuel injection strategy. Change of fuel injection quantity was not attempted in this study as the main objective was to demonstrate the use of RL algorithms while maintaining a constant indicated mean effective pressure. A change in fuel quantity has a larger influence on the indicated mean effective pressure than a change in fuel injection timing. The focus of this work was to present a novel methodology of using the 3D combustion data from analysis software in the form of a functional mock-up unit (FMU) and showcasing the implementation of a RL algorithm in Python language to interact with the FMU to reduce the NO and soot emissions by suggesting changes to the main injection timing in a pilot–main and pilot–main–post-injection strategy. RL algorithms identified the operating injection strategy, i.e., main injection timing for a pilot–main and pilot–main–post-injection strategy, reducing NO emissions from 38% to 56% and soot emissions from 10% to 90% for a range of fuel injection strategies.