Real-Time, Content Aware Camera -- Algorithm -- Hardware Co-Adaptation for Minimal Power Video Encoding

Abstract

In this paper, a content aware, low power video encoder design is presented in which the algorithms and hardware are co-optimized to adapt concurrently to video content in real-time. Natural image statistical models are used to form spatiotemporal predictions about the content of future frames. A key innovation in this work is that that the predictions are used as parameters in a feedback control loop to intelligently down sample (change the resolution of the frame image across different parts of the image) the video encoder input immediately at the camera, thus reducing the amount of work required by the encoder per frame. A multiresolution frame representation is used to produce regular data structures which allow for efficient hardware design. The hardware is co-optimized with the algorithm to reduce power based on the reduced input size resulting from the algorithm. The design also allows for selectable, graceful degradation of video quality while reducing power consumption.