Proceedings of the 14th Conference on ACM Multimedia Systems最新文献_第4页

Semi-coupled Congestion Control for Multi-site Parallel Downloading 多站点并行下载的半耦合拥塞控制

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3597030

Chenfei Tian, Shaorui Ren, Yixuan Zhang, Mingwei Xu

引用次数: 0

FBDT: Forward and Backward Data Transmission Across RATs for High Quality Mobile 360-Degree Video VR Streaming FBDT:高质量移动360度VR视频流的正向和反向数据传输

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590987

S. Srinivasan, Samuel Shippey, Ehsan Aryafar, Jacob Chakareski

{"title":"FBDT: Forward and Backward Data Transmission Across RATs for High Quality Mobile 360-Degree Video VR Streaming","authors":"S. Srinivasan, Samuel Shippey, Ehsan Aryafar, Jacob Chakareski","doi":"10.1145/3587819.3590987","DOIUrl":"https://doi.org/10.1145/3587819.3590987","url":null,"abstract":"The metaverse encompasses many virtual universes and relies on streaming high-quality 360° videos to VR/AR headsets. This type of video transmission requires very high data rates to meet the desired Quality of Experience (QoE) for all clients. Simultaneous data transmission across multiple Radio Access Technologies (RATs) such as WiFi and WiGig is a key solution to meet this required capacity demand. However, existing transport layer multi-RAT traffic aggregation schemes suffer from Head-of-Line (HoL) blocking and sub-optimal traffic splitting across the RATs, particularly when there is a high fluctuation in their channel conditions. As a result, state-of-the-art multi-path TCP (MPTCP) solutions can achieve aggregate transmission data rates that are lower than that of using only a single WiFi RAT in many practical settings, e.g., when the client is mobile. We make two key contributions to enable high quality mobile 360° video VR streaming using multiple RATs. First, we propose the design of FBDT, a novel multi-path transport layer solution that can achieve the sum of individual transmission rates across the RATs despite their system dynamics. We implemented FBDT in the Linux kernel and showed substantial improvement in transmission throughput relative to state-of-the-art schemes, e.g, 2.5x gain in a dual-RAT scenario (WiFi and WiGig) when the VR client is mobile. Second, we formulate an optimization problem to maximize a mobile VR client's viewport quality by taking into account statistical models of how clients explore the 360° look-around panorama and the transmission data rate of each RAT. We explore an iterative method to solve this problem and evaluate its performance through measurement-driven simulations leveraging our testbed. We show up to 12 dB increase in viewport quality when our optimization framework is employed.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126839413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SketchBuddy: Context-Aware Sketch Enrichment and Enhancement SketchBuddy:上下文感知素描丰富和增强

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590980

Aishwarya Agarwal, A. Srivastava, I. Nair, S. Mishra, Vineeth Dorna, S. Nangi, Balaji Vasan Srinivasan

{"title":"SketchBuddy: Context-Aware Sketch Enrichment and Enhancement","authors":"Aishwarya Agarwal, A. Srivastava, I. Nair, S. Mishra, Vineeth Dorna, S. Nangi, Balaji Vasan Srinivasan","doi":"10.1145/3587819.3590980","DOIUrl":"https://doi.org/10.1145/3587819.3590980","url":null,"abstract":"Sketching is a visual thinking tool available to humans for several decades. With the advent of modern sketching technologies, artists use sketches to express and iterate their ideas. To accelerate sketch-based ideation and illustration workflows, we propose a novel framework, SketchBuddy, which retrieves diverse fine-grained object suggestions to enrich a sketch and coherently inserts it into the scene. Sketchbuddy detects objects in the input sketch to estimate the scene context which is then utilized for the recommendation and insertion. We propose a novel multi-modal transformer based framework for obtaining context-aware fine-grained object recommendations. We train a CNN-based bounding box classifier to extract information from the input scene and the recommended objects to infer plausible locations for insertion. While prior works focus on sketches at object-level only, SketchBuddy is the first work in the direction of scene-level sketching assistance. Our extensive evaluations comparing SketchBuddy against competing baselines across several metrics and agreements with human preferences demonstrate its value on several aspects.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132848921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

QUTY: Towards Better Understanding and Optimization of Short Video Quality 质量:朝着更好地理解和优化短视频质量

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590984

Haodan Zhang, Yixuan Ban, Zongming Guo, Zhimin Xu, Qian Ma, Yue Wang, Xinggong Zhang

{"title":"QUTY: Towards Better Understanding and Optimization of Short Video Quality","authors":"Haodan Zhang, Yixuan Ban, Zongming Guo, Zhimin Xu, Qian Ma, Yue Wang, Xinggong Zhang","doi":"10.1145/3587819.3590984","DOIUrl":"https://doi.org/10.1145/3587819.3590984","url":null,"abstract":"Short video applications such as TikTok and Instagram have attracted tremendous attention recently. However, it is very limited for industry and academia to understand the user's Quality of Experience (QoE) on short video, let alone how to improve the QoE in short video streaming. In this paper, we dug into the factors that affect the user's QoE and then propose a system which models and optimizes user's QoE. We unveil the QoE formulation of short video by diving into the understanding of users' viewing behavior, and analyzing large dataset (more than 10 million records) from Douyin (a short video application). We find that: (a) the increase of rebuffering duration, rebuffering times, and starting delay will decrease the user retention ratio, whereas the video bitrate has little effect, (b) the users exhibit different viewing behavior patterns such as scrolling video fastly or slowly, which can be utilize to improve QoE. Over these findings, we propose QUTY, a QoE-driven short video streaming system, which utilizes a data-driven approach to quantify QoE of short video and optimizes it with a Hierarchical Reinforcement Learning (HRL) method. Our evaluations show that QUTY can reduce the rebuffering ratio by up to 49.9%, reduce the rebuffering times by up to 55.8%, reduce the startup delay by up to 81.9%, and improve the QoE by up to 8.5% compared with the existing short video streaming approaches.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"34 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113989618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TASQ: Temporal Adaptive Streaming over QUIC 任务:QUIC上的时序自适应流

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590991

Akram Ansari, Yang Liu, Mea Wang, Emir Halepovic

{"title":"TASQ: Temporal Adaptive Streaming over QUIC","authors":"Akram Ansari, Yang Liu, Mea Wang, Emir Halepovic","doi":"10.1145/3587819.3590991","DOIUrl":"https://doi.org/10.1145/3587819.3590991","url":null,"abstract":"Traditional Adaptive BitRate (ABR) streaming faces a challenge of providing smooth experience under highly variable network conditions, especially when low latency is required. Effective adaptation techniques exist for deep-buffer scenarios, such as streaming long-form Video-on-Demand content, but remain elusive for short-form or low-latency cases, when even a short segment may be delivered too late and cause a stall. Recently proposed temporal adaptation aims to mitigate this problem by being robust to losing a part of the video segment, essentially dropping the tail of the segment intentionally to avoid the stall. In this paper, we analyze this approach in the context of a recently adopted codec AV1 and find that it does not always provide the promised benefits. We investigate the root causes and find that a combination of codec efficiency and TCP behavior can defeat the benefits of temporal adaptation. We develop a solution based on QUIC, and present the results showing that the benefits of temporal adaptation that still apply to AV1, including reduced stall time up to 65% compared to the original TCP-based approach. In addition, we present a novel way to use the stream management features of QUIC to benefit Quality-of-Experience (QoE) and reduce wasted data in video streaming.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126236822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Dataset of Food Intake Activities Using Sensors with Heterogeneous Privacy Sensitivity Levels 使用具有异构隐私敏感级别的传感器的食物摄入活动数据集

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3592553

Yi-Hung Wu, Hsin-Che Chiang, S. Shirmohammadi, Cheng-Hsin Hsu

{"title":"A Dataset of Food Intake Activities Using Sensors with Heterogeneous Privacy Sensitivity Levels","authors":"Yi-Hung Wu, Hsin-Che Chiang, S. Shirmohammadi, Cheng-Hsin Hsu","doi":"10.1145/3587819.3592553","DOIUrl":"https://doi.org/10.1145/3587819.3592553","url":null,"abstract":"Human activity recognition, which involves recognizing human activities from sensor data, has drawn a lot of interest from researchers and practitioners as a result of the advent of smart homes, smart cities, and smart systems. Existing studies on activity recognition mostly concentrate on coarse-grained activities like walking and jumping, while fine-grained activities like eating and drinking are understudied because it is more difficult to recognize fine-grained activities than coarse-grained ones. As such, food intake activity recognition in particular is under investigation in the literature despite its importance for human health and well-being, including telehealth and diet management. In order to determine sensors' practical recognition accuracy, preferably with the least amount of privacy intrusion, a dataset of food intake activities utilizing sensors with varying degrees of privacy sensitivity is required. In this study, we collected such a dataset by collecting fine-grained food intake activities using sensors of heterogeneous privacy sensitivity levels, namely a mmWave radar, an RGB camera, and a depth camera. Solutions to recognize food intake activities can be developed using this dataset, which may provide a more comprehensive picture of the accuracy and privacy trade-offs involved with heterogeneous sensors.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129982326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Performance and User Experience Studies of HILLES: Home-based Immersive Lower Limb Exergame System HILLES的性能和用户体验研究:基于家庭的沉浸式下肢运动系统

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590985

Yu-Yen Chung, T. Annaswamy, B. Prabhakaran

{"title":"Performance and User Experience Studies of HILLES: Home-based Immersive Lower Limb Exergame System","authors":"Yu-Yen Chung, T. Annaswamy, B. Prabhakaran","doi":"10.1145/3587819.3590985","DOIUrl":"https://doi.org/10.1145/3587819.3590985","url":null,"abstract":"Head-Mounted Devices (HMDs) have become popular for home-based immersive gaming. However, using lower limb motion in the immersive virtual environment is still restricted. This work introduces an RGB-D camera-based motion capture system alongside a standalone HMD for Home-based Immersive Lower Limbs Exergame Systems (HILLES) in a seated pose. With the advance of neural network models, camera-based 3D body tracking accuracy is increasing. Nevertheless, the high demand for computing resources on model inference may compromise the game engine's performance. Accordingly, HILLES applies a distributed architecture to leverage the resources effectively. The system performances, such as frames per second and latency, are compared with a centralized system. For an immersive exergame, a pet walking around could raise safety issues. Hence, we also showcase that the camera system can provide an additional safety feature by combining an object detection model. Besides, another challenge in games focusing on lower limb interactions is the safe reachability of different virtual objects from a seated pose. Accordingly, in the user study, a stomping game with two reachability enhancements, including leg extension and seated navigation, is implemented based on the HILLES to evaluate and explore the gaming experience. The result shows that the system motivates the leg exercise, and the added enhancements may adjust the game difficulty. However, the enhancements may also distract users from focusing on leg exertion. The derived insight could benefit the lower limb exergame design in the future.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130872162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Deep Feature Compression with Rate-Distortion Optimization for Networked Camera Systems 基于率失真优化的网络摄像机系统深度特征压缩

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590974

A. Ikusan, Rui Dai

{"title":"Deep Feature Compression with Rate-Distortion Optimization for Networked Camera Systems","authors":"A. Ikusan, Rui Dai","doi":"10.1145/3587819.3590974","DOIUrl":"https://doi.org/10.1145/3587819.3590974","url":null,"abstract":"Deep-learning-based video analysis solutions have become indispensable components in today's intelligent sensing applications. In a networked camera system, an efficient way to analyze the captured videos is to extract the features for deep learning at local cameras or edge devices and then transmit the features to powerful processing hubs for further analysis. As there exists substantial redundancy among different feature maps from the same video frame, the feature maps could be compressed before transmission to save bandwidth. This paper introduces a new rate-distortion optimized framework for compressing the intermediate deep features from the key frames of a video. First, to reduce the redundancy among different features, a feature selection strategy is designed based on hierarchical clustering. The selected features are then quantized, repacked as videos, and further compressed using a standardized video encoder. Furthermore, the proposed framework incorporates rate-distortion models that are built for three representative computer vision tasks: image classification, image segmentation, and image retrieval. A corresponding rate-distortion optimization module is designed to enhance the performance of common computer vision tasks under rate constraints. Experimental results show that the proposed deep feature compression framework can boost the compression performance over the standard HEVC video encoder.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122744059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multimodal Cascaded Framework with Metric Learning Robust to Missing Modalities for Person Classification 基于度量学习的多模态级联框架对缺失模态具有鲁棒性

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3590989

Vijay John, Yasutomo Kawanishi

{"title":"Multimodal Cascaded Framework with Metric Learning Robust to Missing Modalities for Person Classification","authors":"Vijay John, Yasutomo Kawanishi","doi":"10.1145/3587819.3590989","DOIUrl":"https://doi.org/10.1145/3587819.3590989","url":null,"abstract":"This paper addresses the missing modality problem in multimodal person classification, where an incomplete multimodal input with one modality missing is classified into predefined person classes. A multimodal cascaded framework with three deep learning models is proposed, where model parameters, outputs, and latent space learnt at a given step are transferred to the model in a subsequent step. The cascaded framework addresses the missing modality problem by, firstly, generating the complete multimodal data from the incomplete multimodal data in the feature space via a latent space. Subsequently, the generated and original multimodal features are effectively merged and embedded into a final latent space to estimate the person label. During the learning phase, the cascaded framework uses two novel latent loss functions, the missing modality joint loss, and latent prior loss to learn the different latent spaces. The missing modality joint loss ensures that the similar class latent data are close to each other, even if a modality is missing. In the cascaded framework, the latent prior loss learns the final latent space using a previously learnt latent space as a prior. The proposed framework is validated on the audio-visible RAVDESS and the visible-thermal Speaking Faces datasets. A detailed comparative analysis and an ablation analysis are performed, which demonstrate that the proposed framework enhances the robustness of person classification even under conditions of missing modalities, reporting an average of 21.75% increase and 25.73% increase over the baseline algorithms on the RAVDESS and Speaking Faces datasets.","PeriodicalId":330983,"journal":{"name":"Proceedings of the 14th Conference on ACM Multimedia Systems","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117062966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Media over QUIC: Initial Testing, Findings and Results QUIC上的媒体:初步测试、发现和结果

Proceedings of the 14th Conference on ACM Multimedia Systems Pub Date : 2023-06-07 DOI: 10.1145/3587819.3593937

Zafer Gurel, Tugce Erkilic Civelek, Atakan Bodur, Senem Bilgin, Deniz Yeniceri, A. Begen

引用次数: 1