IEEE Journal on Emerging and Selected Topics in Circuits and Systems最新文献_第10页

Generative Refinement for Low Bitrate Image Coding Using Vector Quantized Residual 使用矢量量化残差进行低比特率图像编码的生成式改进

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-05 DOI: 10.1109/JETCAS.2024.3385653

Yuzhuo Kong;Ming Lu;Zhan Ma

{"title":"Generative Refinement for Low Bitrate Image Coding Using Vector Quantized Residual","authors":"Yuzhuo Kong;Ming Lu;Zhan Ma","doi":"10.1109/JETCAS.2024.3385653","DOIUrl":"10.1109/JETCAS.2024.3385653","url":null,"abstract":"Despite the significant progress in recent deep learning-based image compression, the reconstructed visual quality still suffers at low bitrates due to the lack of high-frequency information. Existing methods deploy the generative adversarial networks (GANs) as an additional loss to supervise the rate-distortion (R-D) optimization, capable of producing more high-frequency components for visually pleasing reconstruction but also introducing unexpected fake textures. This work, instead, proposes to generate high-frequency residuals to refine an image reconstruction compressed using existing image compression solutions. Such a residual signal is calculated between the decoded image and its uncompressed input and quantized to proper codeword vectors in a learnable codebook for decoder-side generative refinement. Extensive experiments demonstrate that our method can restore high-frequency information given images compressed by any codecs and outperform the state-of-the-art generative image compression algorithms or perceptual-oriented post-processing approaches. Moreover, the proposed method using vector quantized residual exhibits remarkable robustness and generalizes to both rules-based and learning-based compression models, which can be used as a plug-and-play module for perceptual optimization without re-training.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"185-197"},"PeriodicalIF":3.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PKU-AIGI-500K: A Neural Compression Benchmark and Model for AI-Generated Images PKU-AIGI-500K：人工智能生成图像的神经压缩基准和模型

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-04-05 DOI: 10.1109/JETCAS.2024.3385629

Xunxu Duan;Siwei Ma;Hongbin Liu;Chuanmin Jia

{"title":"PKU-AIGI-500K: A Neural Compression Benchmark and Model for AI-Generated Images","authors":"Xunxu Duan;Siwei Ma;Hongbin Liu;Chuanmin Jia","doi":"10.1109/JETCAS.2024.3385629","DOIUrl":"10.1109/JETCAS.2024.3385629","url":null,"abstract":"In recent years, artificial intelligence-generated content (AIGC) enabled by foundation models has received increasing attention and is undergoing remarkable development. Text prompts can be elegantly translated/converted into high-quality, photo-realistic images. This remarkable feature, however, has introduced extremely high bandwidth requirements for compressing and transmitting the vast number of AI-generated images (AIGI) for such AIGC services. Despite this challenge, research on compression methods for AIGI is conspicuously lacking but undeniably necessary. This research addresses this critical gap by introducing the pioneering AIGI dataset, PKU-AIGI-500K, encompassing over 105k+ diverse prompts and 528k+ images derived from five major foundation models. Through this dataset, we delve into exploring and analyzing the essential characteristics of AIGC images and empirically prove that existing data-driven lossy compression methods achieve sub-optimal or less efficient rate-distortion performance without fine-tuning, primarily due to a domain shift between AIGIs and natural images. We comprehensively benchmark the rate-distortion performance and runtime complexity analysis of conventional and learned image coding solutions that are openly available, uncovering new insights for emerging studies in AIGI compression. Moreover, to harness the full potential of redundant information in AIGI and its corresponding text, we propose an AIGI compression model (Cross-Attention Transformer Codec, CATC) trained on this dataset as a strong baseline. Subsequent experimental results demonstrate that our proposed model achieves up to 30.09% bitrate reduction compared to the state-of-the-art (SOTA) H.266/VVC codec and outperforms the SOTA learned codec, paving the way for future research in AIGI compression.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"172-184"},"PeriodicalIF":3.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592692","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization 使用生成模型的视觉信号编码和处理调查：技术、标准和优化

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-21 DOI: 10.1109/JETCAS.2024.3403524

Zhibo Chen;Heming Sun;Li Zhang;Fan Zhang

{"title":"Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization","authors":"Zhibo Chen;Heming Sun;Li Zhang;Fan Zhang","doi":"10.1109/JETCAS.2024.3403524","DOIUrl":"10.1109/JETCAS.2024.3403524","url":null,"abstract":"This paper provides a survey of the latest developments in visual signal coding and processing with generative models. Specifically, our focus is on presenting the advancement of generative models and their influence on research in the domain of visual signal coding and processing. This survey study begins with a brief introduction of well-established generative models, including the Variational Autoencoder (VAE) models, Generative Adversarial Network (GAN) models, Autoregressive (AR) models, Normalizing Flows and Diffusion models. The subsequent section of the paper explores the advancements in visual signal coding based on generative models, as well as the ongoing international standardization activities. In the realm of visual signal processing, our focus lies on the application and development of various generative models in the research of visual signal restoration. We also present the latest developments in generative visual signal synthesis and editing, along with visual signal quality assessment using generative models and quality assessment for generative models. The practical implementation of these studies is closely linked to the investigation of fast optimization. This paper additionally presents the latest advancements in fast optimization on visual signal coding and processing with generative models. We hope to advance this field by providing researchers and practitioners a comprehensive literature review on the topic of visual signal coding and processing with generative models.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"149-171"},"PeriodicalIF":3.7,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141105359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency 预测 360° 视频显著性：具有时空一致性的 ConvLSTM 编码器-解码器网络

IF 3.7 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-18 DOI: 10.1109/JETCAS.2024.3377096

Zhaolin Wan;Han Qin;Ruiqin Xiong;Zhiyang Li;Xiaopeng Fan;Debin Zhao

{"title":"Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency","authors":"Zhaolin Wan;Han Qin;Ruiqin Xiong;Zhiyang Li;Xiaopeng Fan;Debin Zhao","doi":"10.1109/JETCAS.2024.3377096","DOIUrl":"10.1109/JETCAS.2024.3377096","url":null,"abstract":"360° videos have been widely used with the development of virtual reality technology and triggered a demand to determine the most visually attractive objects in them, aka 360° video saliency prediction (VSP). While generative models, i.e., variational autoencoders or autoregressive models have proved their effectiveness in handling spatio-temporal data, utilizing them in 360° VSP is still challenging due to the problem of severe distortion and feature alignment inconsistency. In this study, we propose a novel spatio-temporal consistency generative network for 360° VSP. A dual-stream encoder-decoder architecture is adopted to process the forward and backward frame sequences of 360° videos simultaneously. Moreover, a deep autoregressive module termed as axial-attention based spherical ConvLSTM is designed in the encoder to memorize features with global-range spatial and temporal dependencies. Finally, motivated by the bias phenomenon in human viewing behavior, a temporal-convolutional Gaussian prior module is introduced to further improve the accuracy of the saliency prediction. Extensive experiments are conducted to evaluate our model and the state-of-the-art competitors, demonstrating that our model has achieved the best performance on the databases of PVS-HM and VR-Eyetracking.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 2","pages":"311-322"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140171922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IEEE Circuits and Systems Society 电气和电子工程师学会电路与系统协会

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3364895

引用次数: 0

Incoming Editorial 来稿编辑

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2023.3335798

Wen-Hsiao Peng

{"title":"Incoming Editorial","authors":"Wen-Hsiao Peng","doi":"10.1109/JETCAS.2023.3335798","DOIUrl":"https://doi.org/10.1109/JETCAS.2023.3335798","url":null,"abstract":"The IEEE Journal On Emerging and Selected Topics in Circuits and Systems (JETCAS) is a periodical sponsored by the IEEE Circuits and Systems Society (CASS). Since its advent about a decade ago, JETCAS has published quarterly special issues on emerging and selected topics that cover the entire field of interest of the CASS. Particular emphasis has been put on emerging areas that are expected to grow over time in scientific and professional importance. For example, the special issues published in the last two years touched upon industry x.0 applications, unconventional computing techniques, memristive circuits and systems, quantum computation, processing-in-memory machine learning, and highly renewable penetrated power systems. Some of these special issues have become valuable references in many forefront technology developments within and beyond CASS. Thanks to the strong leadership by Prof. Ho Ching (Herbert) Iu, the outgoing Editor-in-Chief, and the remarkable work of his editorial board, JETCAS is now one of the leading journals in the CASS, with an impact factor of 4.6-5.8 from 2022 to 2023. Its LinkedIn profile page (\u0000<uri>https://bit.ly/3FLIBFs</uri>\u0000) has attracted more than 1000+ followers since it went online in 2020.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 1","pages":"1-2"},"PeriodicalIF":4.6,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10472163","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140123312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Publication Information 电气和电子工程师学会电路与系统新专题与选题期刊》出版信息

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3364891

引用次数: 0

TechRxiv: Share Your Preprint Research with the World! TechRxiv：与世界分享您的预印本研究成果！

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3371131

引用次数: 0

Guest Editorial: Integrated Devices, Circuits, and Systems for the 6G Era 特邀社论：面向 6G 时代的集成器件、电路和系统

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3367094

Xi Zhu;Roberto Gómez-García;Chun-Hsing Li;Bryan Schwitter

引用次数: 0

IEEE Journal on Emerging and Selected Topics in Circuits and Systems information for authors 供作者参考的《IEEE 电路与系统新兴选题期刊

IF 4.6 2区工程技术

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Pub Date : 2024-03-13 DOI: 10.1109/JETCAS.2024.3364893

引用次数: 0