IEEE Journal on Emerging and Selected Topics in Circuits and Systems最新文献_第8页

Human–Machine Collaborative Image Compression Method Based on Implicit Neural Representations 基于隐式神经表征的人机协作图像压缩方法

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-09 DOI: 10.1109/JETCAS.2024.3386639

Huanyang Li;Xinfeng Zhang

{"title":"Human–Machine Collaborative Image Compression Method Based on Implicit Neural Representations","authors":"Huanyang Li;Xinfeng Zhang","doi":"10.1109/JETCAS.2024.3386639","DOIUrl":"10.1109/JETCAS.2024.3386639","url":null,"abstract":"With the explosive increase in the volume of images intended for analysis by AI, image coding for machine have been proposed to transmit information in a machine-interpretable format, thereby enhancing image compression efficiency. However, such efficient coding schemes often lead to issues like loss of image details and features, and unclear semantic information due to high data compression ratio, making them less suitable for human vision domains. Thus, it is a critical problem to balance image visual quality and machine vision accuracy at a given compression ratio. To address these issues, we introduce a human-machine collaborative image coding framework based on Implicit Neural Representations (INR), which effectively reduces the transmitted information for machine vision tasks at the decoding side while maintaining high-efficiency image compression for human vision against INR compression framework. To enhance the model’s perception of images for machine vision, we design a semantic embedding enhancement module to assist in understanding image semantics. Specifically, we employ the Swin Transformer model to initialize image features, ensuring that the embedding of the compression model are effectively applicable to downstream visual tasks. Extensive experimental results demonstrate that our method significantly outperforms other image compression methods in classification tasks while ensuring image compression efficiency.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"198-208"},"PeriodicalIF":3.7,"publicationDate":"2024-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140593443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FPGA Codec System of Learned Image Compression With Algorithm-Architecture Co-Optimization 算法-架构协同优化的学习图像压缩 FPGA 编解码器系统

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-08 DOI: 10.1109/JETCAS.2024.3386328

Heming Sun;Qingyang Yi;Masahiro Fujita

{"title":"FPGA Codec System of Learned Image Compression With Algorithm-Architecture Co-Optimization","authors":"Heming Sun;Qingyang Yi;Masahiro Fujita","doi":"10.1109/JETCAS.2024.3386328","DOIUrl":"10.1109/JETCAS.2024.3386328","url":null,"abstract":"Learned Image Compression (LIC) has shown a coding ability competitive to traditional standards. To address the complexity issue of LIC, various hardware accelerators are required. As one category of accelerators, FPGA has been used because of its good reconfigurability and high power efficiency. However, the prior work developed the algorithm of LIC neural network at first, and then proposed an associated FPGA hardware. This separate manner of algorithm and architecture development can easily cause a layout problem such as routing congestion when the hardware utilization is high. To mitigate this problem, this paper gives an algorithm-architecture co- optimization of LIC. We first restrict the input and output channel parallelism with some constraints to ease the routing issue with more DSP usage. After that, we adjust the numbers of channels to increase the DSP efficiency. As a result, compared with one recent work with a fine-grained pipelined architecture, we can reach up to 1.5x faster throughput with almost the same coding performance on the Kodak dataset. Compared with another recent work accelerated by AMD/Xilinx DPU, we can reach faster throughput with better coding performance.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"334-347"},"PeriodicalIF":3.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative Refinement for Low Bitrate Image Coding Using Vector Quantized Residual 使用矢量量化残差进行低比特率图像编码的生成式改进

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-05 DOI: 10.1109/JETCAS.2024.3385653

Yuzhuo Kong;Ming Lu;Zhan Ma

{"title":"Generative Refinement for Low Bitrate Image Coding Using Vector Quantized Residual","authors":"Yuzhuo Kong;Ming Lu;Zhan Ma","doi":"10.1109/JETCAS.2024.3385653","DOIUrl":"10.1109/JETCAS.2024.3385653","url":null,"abstract":"Despite the significant progress in recent deep learning-based image compression, the reconstructed visual quality still suffers at low bitrates due to the lack of high-frequency information. Existing methods deploy the generative adversarial networks (GANs) as an additional loss to supervise the rate-distortion (R-D) optimization, capable of producing more high-frequency components for visually pleasing reconstruction but also introducing unexpected fake textures. This work, instead, proposes to generate high-frequency residuals to refine an image reconstruction compressed using existing image compression solutions. Such a residual signal is calculated between the decoded image and its uncompressed input and quantized to proper codeword vectors in a learnable codebook for decoder-side generative refinement. Extensive experiments demonstrate that our method can restore high-frequency information given images compressed by any codecs and outperform the state-of-the-art generative image compression algorithms or perceptual-oriented post-processing approaches. Moreover, the proposed method using vector quantized residual exhibits remarkable robustness and generalizes to both rules-based and learning-based compression models, which can be used as a plug-and-play module for perceptual optimization without re-training.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"185-197"},"PeriodicalIF":3.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PKU-AIGI-500K: A Neural Compression Benchmark and Model for AI-Generated Images PKU-AIGI-500K：人工智能生成图像的神经压缩基准和模型

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-05 DOI: 10.1109/JETCAS.2024.3385629

Xunxu Duan;Siwei Ma;Hongbin Liu;Chuanmin Jia

{"title":"PKU-AIGI-500K: A Neural Compression Benchmark and Model for AI-Generated Images","authors":"Xunxu Duan;Siwei Ma;Hongbin Liu;Chuanmin Jia","doi":"10.1109/JETCAS.2024.3385629","DOIUrl":"10.1109/JETCAS.2024.3385629","url":null,"abstract":"In recent years, artificial intelligence-generated content (AIGC) enabled by foundation models has received increasing attention and is undergoing remarkable development. Text prompts can be elegantly translated/converted into high-quality, photo-realistic images. This remarkable feature, however, has introduced extremely high bandwidth requirements for compressing and transmitting the vast number of AI-generated images (AIGI) for such AIGC services. Despite this challenge, research on compression methods for AIGI is conspicuously lacking but undeniably necessary. This research addresses this critical gap by introducing the pioneering AIGI dataset, PKU-AIGI-500K, encompassing over 105k+ diverse prompts and 528k+ images derived from five major foundation models. Through this dataset, we delve into exploring and analyzing the essential characteristics of AIGC images and empirically prove that existing data-driven lossy compression methods achieve sub-optimal or less efficient rate-distortion performance without fine-tuning, primarily due to a domain shift between AIGIs and natural images. We comprehensively benchmark the rate-distortion performance and runtime complexity analysis of conventional and learned image coding solutions that are openly available, uncovering new insights for emerging studies in AIGI compression. Moreover, to harness the full potential of redundant information in AIGI and its corresponding text, we propose an AIGI compression model (Cross-Attention Transformer Codec, CATC) trained on this dataset as a strong baseline. Subsequent experimental results demonstrate that our proposed model achieves up to 30.09% bitrate reduction compared to the state-of-the-art (SOTA) H.266/VVC codec and outperforms the SOTA learned codec, paving the way for future research in AIGI compression.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"172-184"},"PeriodicalIF":3.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592692","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization 使用生成模型的视觉信号编码和处理调查：技术、标准和优化

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-21 DOI: 10.1109/JETCAS.2024.3403524

Zhibo Chen;Heming Sun;Li Zhang;Fan Zhang

{"title":"Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization","authors":"Zhibo Chen;Heming Sun;Li Zhang;Fan Zhang","doi":"10.1109/JETCAS.2024.3403524","DOIUrl":"10.1109/JETCAS.2024.3403524","url":null,"abstract":"This paper provides a survey of the latest developments in visual signal coding and processing with generative models. Specifically, our focus is on presenting the advancement of generative models and their influence on research in the domain of visual signal coding and processing. This survey study begins with a brief introduction of well-established generative models, including the Variational Autoencoder (VAE) models, Generative Adversarial Network (GAN) models, Autoregressive (AR) models, Normalizing Flows and Diffusion models. The subsequent section of the paper explores the advancements in visual signal coding based on generative models, as well as the ongoing international standardization activities. In the realm of visual signal processing, our focus lies on the application and development of various generative models in the research of visual signal restoration. We also present the latest developments in generative visual signal synthesis and editing, along with visual signal quality assessment using generative models and quality assessment for generative models. The practical implementation of these studies is closely linked to the investigation of fast optimization. This paper additionally presents the latest advancements in fast optimization on visual signal coding and processing with generative models. We hope to advance this field by providing researchers and practitioners a comprehensive literature review on the topic of visual signal coding and processing with generative models.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"149-171"},"PeriodicalIF":3.7,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141105359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency 预测 360° 视频显著性：具有时空一致性的 ConvLSTM 编码器-解码器网络

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-18 DOI: 10.1109/JETCAS.2024.3377096

Zhaolin Wan;Han Qin;Ruiqin Xiong;Zhiyang Li;Xiaopeng Fan;Debin Zhao

{"title":"Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency","authors":"Zhaolin Wan;Han Qin;Ruiqin Xiong;Zhiyang Li;Xiaopeng Fan;Debin Zhao","doi":"10.1109/JETCAS.2024.3377096","DOIUrl":"10.1109/JETCAS.2024.3377096","url":null,"abstract":"360° videos have been widely used with the development of virtual reality technology and triggered a demand to determine the most visually attractive objects in them, aka 360° video saliency prediction (VSP). While generative models, i.e., variational autoencoders or autoregressive models have proved their effectiveness in handling spatio-temporal data, utilizing them in 360° VSP is still challenging due to the problem of severe distortion and feature alignment inconsistency. In this study, we propose a novel spatio-temporal consistency generative network for 360° VSP. A dual-stream encoder-decoder architecture is adopted to process the forward and backward frame sequences of 360° videos simultaneously. Moreover, a deep autoregressive module termed as axial-attention based spherical ConvLSTM is designed in the encoder to memorize features with global-range spatial and temporal dependencies. Finally, motivated by the bias phenomenon in human viewing behavior, a temporal-convolutional Gaussian prior module is introduced to further improve the accuracy of the saliency prediction. Extensive experiments are conducted to evaluate our model and the state-of-the-art competitors, demonstrating that our model has achieved the best performance on the databases of PVS-HM and VR-Eyetracking.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"311-322"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140171922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IEEE Circuits and Systems Society 电气和电子工程师学会电路与系统协会

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3364895

引用次数: 0

Incoming Editorial 来稿编辑

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2023.3335798

Wen-Hsiao Peng

{"title":"Incoming Editorial","authors":"Wen-Hsiao Peng","doi":"10.1109/JETCAS.2023.3335798","DOIUrl":"https://doi.org/10.1109/JETCAS.2023.3335798","url":null,"abstract":"The IEEE Journal On Emerging and Selected Topics in Circuits and Systems (JETCAS) is a periodical sponsored by the IEEE Circuits and Systems Society (CASS). Since its advent about a decade ago, JETCAS has published quarterly special issues on emerging and selected topics that cover the entire field of interest of the CASS. Particular emphasis has been put on emerging areas that are expected to grow over time in scientific and professional importance. For example, the special issues published in the last two years touched upon industry x.0 applications, unconventional computing techniques, memristive circuits and systems, quantum computation, processing-in-memory machine learning, and highly renewable penetrated power systems. Some of these special issues have become valuable references in many forefront technology developments within and beyond CASS. Thanks to the strong leadership by Prof. Ho Ching (Herbert) Iu, the outgoing Editor-in-Chief, and the remarkable work of his editorial board, JETCAS is now one of the leading journals in the CASS, with an impact factor of 4.6-5.8 from 2022 to 2023. Its LinkedIn profile page (\u0000<uri>https://bit.ly/3FLIBFs</uri>\u0000) has attracted more than 1000+ followers since it went online in 2020.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 1","pages":"1-2"},"PeriodicalIF":4.6,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10472163","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140123312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Publication Information 电气和电子工程师学会电路与系统新专题与选题期刊》出版信息

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3364891

引用次数: 0

TechRxiv: Share Your Preprint Research with the World! TechRxiv：与世界分享您的预印本研究成果！

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3371131

引用次数: 0