Journal of Real-Time Image Processing最新文献_第2页

YOLOv8s-CFB: a lightweight method for real-time detection of apple fruits in complex environments YOLOv8s-CFB：在复杂环境中实时检测苹果果实的轻量级方法

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-31 DOI: 10.1007/s11554-024-01543-4

Bing Zhao, Aoran Guo, Ruitao Ma, Yanfei Zhang, Jinliang Gong

{"title":"YOLOv8s-CFB: a lightweight method for real-time detection of apple fruits in complex environments","authors":"Bing Zhao, Aoran Guo, Ruitao Ma, Yanfei Zhang, Jinliang Gong","doi":"10.1007/s11554-024-01543-4","DOIUrl":"https://doi.org/10.1007/s11554-024-01543-4","url":null,"abstract":"With the development of apple-picking robots, deep learning models have become essential in apple detection. However, current detection models are often disrupted by complex backgrounds, leading to low recognition accuracy and slow speeds in natural environments. To address these issues, this study proposes an improved model, YOLOv8s-CFB, based on YOLOv8s. This model introduces partial convolution (PConv) in the backbone network, enhances the C2f module, and forms a new architecture, CSPPC, to reduce computational complexity and improve speed. Additionally, FocalModulation technology replaces the original SPPF module to enhance the model’s ability to recognize key areas. Finally, the bidirectional feature pyramid (BiFPN) is introduced to adaptively learn the importance of weights at each scale, effectively retaining multi-scale information through a bidirectional context information transmission mechanism, and improving the model’s detection ability for occluded targets. Test results show that the improved YOLOv8 network achieves better detection performance, with an average accuracy of 93.86%, a parameter volume of 8.83 M, and a detection time of 0.7 ms. The improved algorithm achieves high detection accuracy with a small weight file, making it suitable for deployment on mobile devices. Therefore, the improved model can efficiently and accurately detect apples in complex orchard environments in real time.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

YOLO9tr: a lightweight model for pavement damage detection utilizing a generalized efficient layer aggregation network and attention mechanism YOLO9tr：利用广义高效层聚合网络和关注机制进行路面损坏检测的轻量级模型

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-31 DOI: 10.1007/s11554-024-01545-2

Sompote Youwai, Achitaphon Chaiyaphat, Pawarotorn Chaipetch

{"title":"YOLO9tr: a lightweight model for pavement damage detection utilizing a generalized efficient layer aggregation network and attention mechanism","authors":"Sompote Youwai, Achitaphon Chaiyaphat, Pawarotorn Chaipetch","doi":"10.1007/s11554-024-01545-2","DOIUrl":"https://doi.org/10.1007/s11554-024-01545-2","url":null,"abstract":"Maintaining road pavement integrity is crucial for ensuring safe and efficient transportation. Conventional methods for assessing pavement condition are often laborious and susceptible to human error. This paper proposes YOLO9tr, a novel lightweight object detection model for pavement damage detection, leveraging the advancements of deep learning. YOLO9tr is based on the YOLOv9 architecture, incorporating a partial attention block that enhances feature extraction and attention mechanisms, leading to improved detection performance in complex scenarios. The model is trained on a comprehensive dataset comprising road damage images from multiple countries. This dataset includes an expanded set of damage categories beyond the standard four types (longitudinal cracks, transverse cracks, alligator cracks, and potholes), providing a more nuanced classification of road damage. This broadened classification range allows for a more accurate and realistic assessment of pavement conditions. Comparative analysis demonstrates YOLO9tr’s superior precision and inference speed compared to state-of-the-art models like YOLOv8, YOLOv9 and YOLOv10, achieving a balance between computational efficiency and detection accuracy. The model achieves a high frame rate of up to 136 FPS, making it suitable for real-time applications such as video surveillance and automated inspection systems. The research presents an ablation study to analyze the impact of architectural modifications and hyperparameter variations on model performance, further validating the effectiveness of the partial attention block. The results highlight YOLO9tr’s potential for practical deployment in real-time pavement condition monitoring, contributing to the development of robust and efficient solutions for maintaining safe and functional road infrastructure.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ESC-YOLO: optimizing apple fruit recognition with efficient spatial and channel features in YOLOX ESC-YOLO：利用 YOLOX 中的高效空间和通道特征优化苹果果实识别

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-29 DOI: 10.1007/s11554-024-01540-7

Jun Sun, Yifei Peng, Chen Chen, Bing Zhang, Zhaoqi Wu, Yilin Jia, Lei Shi

{"title":"ESC-YOLO: optimizing apple fruit recognition with efficient spatial and channel features in YOLOX","authors":"Jun Sun, Yifei Peng, Chen Chen, Bing Zhang, Zhaoqi Wu, Yilin Jia, Lei Shi","doi":"10.1007/s11554-024-01540-7","DOIUrl":"https://doi.org/10.1007/s11554-024-01540-7","url":null,"abstract":"Accurate localization of apple fruits and recognition of occlusion types in complex orchard environments play an important role in precision agriculture. This work proposes an efficient fruit recognition model called Efficient Spatial and Channel Feature YOLOX (ESC-YOLO). ESC-YOLO is built upon YOLOX and fully leverages and emphasizes spatial channel information, ensuring coherence between global information and local features. The optimization strategies for the backbone network involve adopting EfficientViT as the foundational backbone, integrating Spatial and Channel Reconstruction Convolution (SCConv) into the input stem to reorganize spatial channel features and reduce redundancy, and constructing the Efficient-MBConv module, which is optimally combined with the EfficientViTBlock for feature extraction. The optimization strategies for the neck network involve utilizing the Centralized Feature Pyramid Net (CFPNet) as the neck network and employing a Simple, Parameter-Free Attention Module (SimAM) to enhance model performance. In this work, we adopted the lightweight model of the ESC-YOLO for performance evaluation, namely ESC-YOLO-S. It achieves a 4.26% improvement in Top-1 mean Average Precision (mAP) compared to YOLOX-S and significantly reduces the false and missed detections caused by various types of occlusions. Therefore, the improved model meets the requirements for high-precision identification in complex orchard environments.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Slim-YOLO-PR_KD: an efficient pose-varied object detection method for underground coal mine Slim-YOLO-PR_KD：一种高效的煤矿井下姿态变化物体检测方法

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-28 DOI: 10.1007/s11554-024-01539-0

Huaxing Mu, Jueting Liu, Yanyun Guan, Wei Chen, Tingting Xu, Zehua Wang

{"title":"Slim-YOLO-PR_KD: an efficient pose-varied object detection method for underground coal mine","authors":"Huaxing Mu, Jueting Liu, Yanyun Guan, Wei Chen, Tingting Xu, Zehua Wang","doi":"10.1007/s11554-024-01539-0","DOIUrl":"https://doi.org/10.1007/s11554-024-01539-0","url":null,"abstract":"Real-time object detection in underground coal mine is a crucial task in the development of AI-assisted supervision systems. Due to the complex environment of the underground coal mine, limited computing resources, and the variability of object poses, the general object detection algorithms cannot provide good performance. Hence, an improved underground pose-varied object detection method named Slim-YOLO-PR_KD has been proposed. By designing an efficient pose-varied attention module (EPA) for the backbone network, providing a receive field block (RFB) module for the neck network, and optimizing the loss function, the underground pose-varied detection model YOLO-PR is obtained, which achieved good accuracy but reduced speed. For YOLO-PR, the study improved the original module by designing RFB_SK, a lightweight C2f_GSG module, a shared parameter detection head and selectively replaced modules to slim down the whole network, resulting in a lightweight detection model Slim-YOLO-PR. By using an attention guided knowledge distillation of underground object detection method and using YOLO-PR as the teacher model, the efficient pose-varied detection model Slim-YOLO-PR_KD for coal mine underground is proposed. The experimental results show that compared with the baseline model, the proposed Slim-YOLO-PR_KD has a faster detection speed, achieving higher detection accuracy while reducing model parameters and computational complexity by 42% and 46% respectively, making it capable of performing real-time underground detection tasks. Compared with other general detection models, Slim-YOLO-PR_KD exhibits excellent performance in real-time pose-varied object detection tasks in complex environments of underground coal mines.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Energy-efficient real-time visual image adversarial generation and processing algorithm for new energy vehicles 面向新能源汽车的节能型实时视觉图像对抗生成与处理算法

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-28 DOI: 10.1007/s11554-024-01544-3

Yinghuan Li, Jicheng Liu

{"title":"Energy-efficient real-time visual image adversarial generation and processing algorithm for new energy vehicles","authors":"Yinghuan Li, Jicheng Liu","doi":"10.1007/s11554-024-01544-3","DOIUrl":"https://doi.org/10.1007/s11554-024-01544-3","url":null,"abstract":"With the rapid development of deep learning in the last decade, generating and processing real-time images have become one of critical methods in intelligent driving systems for new energy vehicles. However, the real-time images captured by sensors are susceptible to variations in various environments, including different weather and lighting conditions. To enhance the real-time image generation performance for new energy vehicles in complex environments, and improve real-time visual image processing capabilities, this study proposes an energy-efficient real-time visual image adversarial generation and processing algorithm, called as ENV-GAN. It hypothesizes a shared latent domain among mixed image domains after analyzing driving situations under various weather and lighting conditions. Mappings are established between different image domains. Besides, a multi-encoder weight-sharing technique is utilized to enhances the generative adversarial network model. Additionally, the algorithm integrates an attention module to enhance the modelâ€™s image generation. Experimental results and analysis demonstrate that the new algorithm outperforms existing algorithms in tasks such as defogging, rain removal, and lighting enhancement, offering high energy efficiency and low energy consumption.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hardware-friendly fast rate-distortion optimization algorithm for AV1 encoder 针对 AV1 编码器的硬件友好型快速速率失真优化算法

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-27 DOI: 10.1007/s11554-024-01535-4

Ran Tang, Xiaofeng Huang, Yan Cui, Xinnan Guo, Yang Zhou, Haibing Yin, Chenggang Yan

{"title":"Hardware-friendly fast rate-distortion optimization algorithm for AV1 encoder","authors":"Ran Tang, Xiaofeng Huang, Yan Cui, Xinnan Guo, Yang Zhou, Haibing Yin, Chenggang Yan","doi":"10.1007/s11554-024-01535-4","DOIUrl":"https://doi.org/10.1007/s11554-024-01535-4","url":null,"abstract":"The rate distortion optimization (RDO) process aims at achieving optimal coding performance by determining the optimal coding mode according to a certain strategy in the AV1 video coding. However, the high computational complexity and strong data dependency in RDO impede real-time applications. To address these issues, a fast RDO algorithm suitable for hardware implementation is proposed. Firstly, we propose a high-frequency coefficients zero-setting approach to optimize the hardware memory occupation. Then, in the rate-distortion calculation stage, an efficient rate estimation method is proposed based on a statistical feature for the number of quantization coefficients, and the distortion estimation method is proposed by considering intrinsic features in the all-zero block. Finally, a reconstruction approximate model is proposed to solve the low parallelism issue caused by the coupling of pixel reconstruction and prediction data. Experimental results show that the proposed algorithm achieves 68.49% and 50.77% time-saving by 2.73% and 2.95% Bjøntegaard delta rate (BD-Rate) increase on average under all intra (AI) and random access (RA) configurations, respectively.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142227593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Efficient spatio-temporal network for action recognition 用于动作识别的高效时空网络

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-23 DOI: 10.1007/s11554-024-01541-6

Yanxiong Su, Qian Zhao

引用次数: 0

Energy efficiency assessment in advanced driver assistance systems with real-time image processing on custom Xilinx DPUs 利用定制 Xilinx DPU 进行实时图像处理，评估高级驾驶辅助系统的能效

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-22 DOI: 10.1007/s11554-024-01538-1

Güner Tatar, Salih Bayar

{"title":"Energy efficiency assessment in advanced driver assistance systems with real-time image processing on custom Xilinx DPUs","authors":"Güner Tatar, Salih Bayar","doi":"10.1007/s11554-024-01538-1","DOIUrl":"https://doi.org/10.1007/s11554-024-01538-1","url":null,"abstract":"The rapid advancement in embedded AI, driven by integrating deep neural networks (DNNs) into embedded systems for real-time image and video processing, has been notably pushed by AI-specific platforms like the AMD Xilinx Vitis AI on the MPSoC-FPGA platform. This platform utilizes a configurable Deep Processing Unit (DPU) for scalable resource utilization and operating frequencies. Our study employed a detailed methodology to assess the impact of various DPU configurations and frequencies on resource utilization and energy consumption. The findings reveal that increasing the DPU frequency enhances resource utilization efficiency and improves performance. Conversely, lower frequencies significantly reduce resource utilization, with only a borderline decrease in performance. These trade-offs are influenced not only by frequency but also by variations in DPU parameters. These findings are critical for developing energy-efficient AI-driven systems in Advanced Driver Assistance Systems (ADAS) based on real-time video processing. By leveraging the capabilities of Xilinx Vitis AI deployed on the Kria KV260 MPSoC platform, we explore the intricacies of optimizing energy efficiency through multi-task learning in real-time ADAS applications.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FPGA-based hardware/firmware co-design for real-time radiometric correction onboard microsatellite 基于 FPGA 的硬件/软件协同设计，用于在微型卫星上进行实时辐射校正

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-20 DOI: 10.1007/s11554-024-01536-3

Youcef Ghelamallah, Azzeddine Rachedi

{"title":"FPGA-based hardware/firmware co-design for real-time radiometric correction onboard microsatellite","authors":"Youcef Ghelamallah, Azzeddine Rachedi","doi":"10.1007/s11554-024-01536-3","DOIUrl":"https://doi.org/10.1007/s11554-024-01536-3","url":null,"abstract":"Remote sensing images are inevitably produced with radiometric artifacts due to the photo-response non-uniformity of charge-coupled device (CCD) sensors. In situations where time constraints demand the prompt acquisition of imaging products, integrating an onboard radiometric correction system becomes essential. This paper advocates for a hardware–firmware co-design approach to achieve radiometric correction within the payload front-end electronics (FEE), leveraging the capabilities of field programmable gate array circuits (FPGA). The selection of an appropriate CCD sensor and optical device is guided by a thorough payload mission analysis, ensuring compliance with the specifications derived from Alsat-1B, the Algerian microsatellite launched in September 2016. Simulation results demonstrate that the designed FPGA firmware effectively controls the CCD sensor and configures its settings to achieve real-time radiometric correction of the acquired pixels in accordance with the mission requirements. To ensure efficient utilization during imaging operations, a hardware solution for onboard storage and in-orbit update of the radiometric coefficients has been considered for the radiometric correction system.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Real-time low-light video enhancement on smartphones 智能手机上的实时弱光视频增强功能

IF 3 4区计算机科学

Journal of Real-Time Image Processing Pub Date : 2024-08-19 DOI: 10.1007/s11554-024-01532-7

Yiming Zhou, Callen MacPhee, Wesley Gunawan, Ali Farahani, Bahram Jalali

{"title":"Real-time low-light video enhancement on smartphones","authors":"Yiming Zhou, Callen MacPhee, Wesley Gunawan, Ali Farahani, Bahram Jalali","doi":"10.1007/s11554-024-01532-7","DOIUrl":"https://doi.org/10.1007/s11554-024-01532-7","url":null,"abstract":"Real-time low-light video enhancement on smartphones remains an open challenge due to hardware constraints such as limited sensor size and processing power. While night mode cameras have been introduced in smartphones to acquire high-quality images in light-constrained environments, their usability is restricted to static scenes as the camera must remain stationary for an extended period to leverage long exposure times or burst imaging techniques. Concurrently, significant process has been made in low-light enhancement on images coming out from the camera’s image signal processor (ISP), particularly through neural networks. These methods do not improve the image capture process itself; instead, they function as post-processing techniques to enhance the perceptual brightness and quality of captured imagery for display to human viewers. However, most neural networks are computationally intensive, making their mobile deployment either impractical or requiring considerable engineering efforts. This paper introduces VLight, a novel single-parameter low-light enhancement algorithm that enables real-time video enhancement on smartphones, along with real-time adaptation to changing lighting conditions and user-friendly fine-tuning. Operating as a custom brightness-booster on digital images, VLight provides real-time and device-agnostic enhancement directly on users’ devices. Notably, it delivers real-time low-light enhancement at up to 67 frames per second (FPS) for 4K videos locally on the smartphone.","PeriodicalId":51224,"journal":{"name":"Journal of Real-Time Image Processing","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0