Ultra Low-Power Deep Learning Applications at the Edge with Jetson Orin AGX Hardware

The latest NVIDIA Jetson Orin AGX hardware provides new capabilities for “at the edge processing,”, where sensor information is collected. The computing architecture does this by providing massive computation due to its high performance, small form factor, and low power consumption. The recently released (2022) Orin and the novel research completed on this effort were combined to accelerate development and demonstration of a new concept of operation for machine learning at the edge. This research included the development of a concept that uses the deep learning object detector, YOLOv4-tiny with the Jetson Orin AGX that obtains data through a video feed from a drone to emulate autonomous capabilities for onboard embedded computing. Further, this research included the development of model-based solutions on the both the public (VisDrone) and newly collected optical datasets. Extending on this further, the technical approach included applying these concepts through experiments and demonstrations. Specifically, a data collection and processing plan were developed and implemented. Importantly, our technical approach allowed us to rapidly move from non-real time processing and successfully demonstrate real-time, in flight capabilities. In summary, this research included the use of new compute hardware, novel processing algorithms, and a unique concept of operation. This technical approach resulted in the real-time detection of targets (vehicles) from various flight altitudes (nominally 400 ft) using newly collected electro-optical (EO) data obtained in real time through the drone's High-Definition Multimedia Interface (HDMI).