MoA-VR: A Mixture-of-Agents System Toward All-in-One Video Restoration

IF 8.5 1区 工程技术 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Lu Liu;Chunlei Cai;Shaocheng Shen;Jianfeng Liang;Weimin Ouyang;Tianxiao Ye;Jian Mao;Huiyu Duan;Jiangchao Yao;Xiaoyun Zhang;Qiang Hu;Guangtao Zhai
{"title":"MoA-VR: A Mixture-of-Agents System Toward All-in-One Video Restoration","authors":"Lu Liu;Chunlei Cai;Shaocheng Shen;Jianfeng Liang;Weimin Ouyang;Tianxiao Ye;Jian Mao;Huiyu Duan;Jiangchao Yao;Xiaoyun Zhang;Qiang Hu;Guangtao Zhai","doi":"10.1109/JSTSP.2025.3627128","DOIUrl":null,"url":null,"abstract":"Real-world videos often suffer from complex degradations, such as noise, compression artifacts, and low-light distortions, due to diverse acquisition and transmission conditions. Existing restoration methods typically require professional manual selection of specialized models or rely on monolithic architectures that fail to generalize across varying degradations. Inspired by expert experience, we propose MoA-VR, the first <underline>M</u>ixture-<underline>o</u>f-<underline>A</u>gents <underline>V</u>ideo <underline>R</u>estoration system that mimics the reasoning and processing procedures of human professionals through three coordinated agents: Degradation Identification, Routing and Restoration, and Restoration Quality Assessment. Specifically, we construct a large-scale and high-resolution video degradation recognition benchmark and build a vision-language model (VLM) driven degradation identifier. We further introduce a self-adaptive router powered by large language models (LLMs), which autonomously learns effective restoration strategies by observing tool usage patterns. To assess intermediate and final processed video quality, we construct the <underline>Res</u>tored <underline>V</u>ideo <underline>Q</u>uality (Res-VQ) dataset and design a dedicated VLM-based video quality assessment (VQA) model tailored for restoration tasks. Extensive experiments demonstrate that MoA-VR effectively handles diverse and compound degradations, consistently outperforming existing baselines in terms of both objective metrics and perceptual quality. These results highlight the potential of integrating multimodal intelligence and modular reasoning in general-purpose video restoration systems.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"19 8","pages":"1822-1837"},"PeriodicalIF":8.5000,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Journal of Selected Topics in Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/11224772/","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/11/4 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

Real-world videos often suffer from complex degradations, such as noise, compression artifacts, and low-light distortions, due to diverse acquisition and transmission conditions. Existing restoration methods typically require professional manual selection of specialized models or rely on monolithic architectures that fail to generalize across varying degradations. Inspired by expert experience, we propose MoA-VR, the first Mixture-of-Agents Video Restoration system that mimics the reasoning and processing procedures of human professionals through three coordinated agents: Degradation Identification, Routing and Restoration, and Restoration Quality Assessment. Specifically, we construct a large-scale and high-resolution video degradation recognition benchmark and build a vision-language model (VLM) driven degradation identifier. We further introduce a self-adaptive router powered by large language models (LLMs), which autonomously learns effective restoration strategies by observing tool usage patterns. To assess intermediate and final processed video quality, we construct the Restored Video Quality (Res-VQ) dataset and design a dedicated VLM-based video quality assessment (VQA) model tailored for restoration tasks. Extensive experiments demonstrate that MoA-VR effectively handles diverse and compound degradations, consistently outperforming existing baselines in terms of both objective metrics and perceptual quality. These results highlight the potential of integrating multimodal intelligence and modular reasoning in general-purpose video restoration systems.
MoA-VR:实现一体化视频还原的混合代理系统
由于不同的采集和传输条件,现实世界的视频经常遭受复杂的退化,如噪声、压缩伪影和低光失真。现有的恢复方法通常需要专业的手动选择专门的模型,或者依赖于无法在不同的降级中泛化的整体架构。在专家经验的启发下,我们提出了MoA-VR,这是第一个混合智能体视频修复系统,它通过三个协调的智能体:退化识别、路由和修复、修复质量评估来模拟人类专业人员的推理和处理过程。具体而言,我们构建了一个大规模、高分辨率的视频退化识别基准,并构建了一个视觉语言模型驱动的退化标识符。我们进一步介绍了一个由大型语言模型(llm)驱动的自适应路由器,它通过观察工具使用模式自主学习有效的恢复策略。为了评估中间和最终处理的视频质量,我们构建了恢复视频质量(Res-VQ)数据集,并设计了一个专门针对恢复任务的基于vmm的视频质量评估(VQA)模型。大量的实验表明,MoA-VR有效地处理了各种和复合的退化,在客观指标和感知质量方面始终优于现有的基线。这些结果突出了在通用视频恢复系统中集成多模态智能和模块化推理的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IEEE Journal of Selected Topics in Signal Processing
IEEE Journal of Selected Topics in Signal Processing 工程技术-工程:电子与电气
CiteScore
19.00
自引率
1.30%
发文量
135
审稿时长
3 months
期刊介绍: The IEEE Journal of Selected Topics in Signal Processing (JSTSP) focuses on the Field of Interest of the IEEE Signal Processing Society, which encompasses the theory and application of various signal processing techniques. These techniques include filtering, coding, transmitting, estimating, detecting, analyzing, recognizing, synthesizing, recording, and reproducing signals using digital or analog devices. The term "signal" covers a wide range of data types, including audio, video, speech, image, communication, geophysical, sonar, radar, medical, musical, and others. The journal format allows for in-depth exploration of signal processing topics, enabling the Society to cover both established and emerging areas. This includes interdisciplinary fields such as biomedical engineering and language processing, as well as areas not traditionally associated with engineering.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书