{"title":"Low‐resolution activity recognition using super‐resolution and model ensemble networks","authors":"Tinglong Liu, Haiyan Wang","doi":"10.4218/etrij.2023-0523","DOIUrl":null,"url":null,"abstract":"In real‐world video super‐resolution, the complexity and diversity of degradations pose substantial challenges during both training and inference. Videos captured in real‐world settings often depict activities at varying resolutions. Typically, these activities are filmed from a distance that reduces the resolution of imagery, which thus lacks discriminative features. To address this problem, we introduce an activity recognition solution. First, a unique integration of data transformation and attention‐based average discriminator are employed for super‐resolution feature augmentation. This approach mitigates the lack of discriminative cues in low‐resolution videos. Subsequently, high‐resolution features extracted from the recovered data are directly fed into a model ensemble for activity recognition. We evaluate the resulting method on the TinyVIRAT‐v2 and HMDB51 datasets, achieving improved visual quality by leveraging the super‐resolution and model ensemble strategy. The proposed method enhances the quality of textures and boosts activity recognition in low‐resolution videos.","PeriodicalId":11901,"journal":{"name":"ETRI Journal","volume":null,"pages":null},"PeriodicalIF":1.3000,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ETRI Journal","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.4218/etrij.2023-0523","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0
Abstract
In real‐world video super‐resolution, the complexity and diversity of degradations pose substantial challenges during both training and inference. Videos captured in real‐world settings often depict activities at varying resolutions. Typically, these activities are filmed from a distance that reduces the resolution of imagery, which thus lacks discriminative features. To address this problem, we introduce an activity recognition solution. First, a unique integration of data transformation and attention‐based average discriminator are employed for super‐resolution feature augmentation. This approach mitigates the lack of discriminative cues in low‐resolution videos. Subsequently, high‐resolution features extracted from the recovered data are directly fed into a model ensemble for activity recognition. We evaluate the resulting method on the TinyVIRAT‐v2 and HMDB51 datasets, achieving improved visual quality by leveraging the super‐resolution and model ensemble strategy. The proposed method enhances the quality of textures and boosts activity recognition in low‐resolution videos.
期刊介绍:
ETRI Journal is an international, peer-reviewed multidisciplinary journal published bimonthly in English. The main focus of the journal is to provide an open forum to exchange innovative ideas and technology in the fields of information, telecommunications, and electronics.
Key topics of interest include high-performance computing, big data analytics, cloud computing, multimedia technology, communication networks and services, wireless communications and mobile computing, material and component technology, as well as security.
With an international editorial committee and experts from around the world as reviewers, ETRI Journal publishes high-quality research papers on the latest and best developments from the global community.