IEEE Transactions on Human-Machine Systems最新文献_第9页

hmOS: An Extensible Platform for Task-Oriented Human–Machine Computing hmOS：面向任务的人机计算可扩展平台

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-07-02 DOI: 10.1109/THMS.2024.3414432

Hui Wang;Zhiwen Yu;Yao Zhang;Yanfei Wang;Fan Yang;Liang Wang;Jiaqi Liu;Bin Guo

{"title":"hmOS: An Extensible Platform for Task-Oriented Human–Machine Computing","authors":"Hui Wang;Zhiwen Yu;Yao Zhang;Yanfei Wang;Fan Yang;Liang Wang;Jiaqi Liu;Bin Guo","doi":"10.1109/THMS.2024.3414432","DOIUrl":"10.1109/THMS.2024.3414432","url":null,"abstract":"With rapid advancements in artificial intelligence (AI) technologies, AI-powered machines are increasingly capable of collaborating with humans to enhance decision-making in various human–machine collaboration scenarios, e.g., medical diagnosis, criminal justice, and autonomous driving. As a result, human–machine computing (HMC) has emerged as a promising computing paradigm that integrates the expertise of humans with the reliable data processing capabilities of machines. Using HMC to facilitate the processing of domain-specific tasks has a lot of potential, but is limited in system-level scalability, i.e., there is no one common easy-to-use interface. In this article, we present human-machine operating system \u0000<monospace>(hmOS)</monospace>\u0000, an open extensible platform for researchers to experiment with HMC for investigating system-centric human–machine collaboration problems. \u0000<monospace>hmOS</monospace>\u0000 supports flexible human–machine collaboration on the strength of the quality-aware task decomposition and allocation. To achieve that, the underlying system architecture and runtime environment are first developed to build a foundational abstraction for the kernel of \u0000<monospace>hmOS</monospace>\u0000. Second, \u0000<monospace>hmOS</monospace>\u0000 facilitates flexible human–machine collaboration through a suitability-based task allocation mechanism, quality estimation guided by fuzzy rules, and iterative feedback on result tuning. We implement the newly proposed \u0000<monospace>hmOS</monospace>\u0000 in a prototype featuring interactive interfaces. Finally, we conduct extensive and realistic experiments to validate the effectiveness of our platform across diverse tasks, showcasing the broad feasibility of \u0000<monospace>hmOS</monospace>\u0000.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"536-545"},"PeriodicalIF":3.5,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141516476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Robust Object Selection in Spontaneous Gaze-Controlled Application Using Exponential Moving Average and Hidden Markov Model 使用指数移动平均法和隐马尔可夫模型在自发凝视控制应用中进行稳健的目标选择

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-07-02 DOI: 10.1109/THMS.2024.3413781

Suatmi Murnani;Noor Akhmad Setiawan;Sunu Wibirama

{"title":"Robust Object Selection in Spontaneous Gaze-Controlled Application Using Exponential Moving Average and Hidden Markov Model","authors":"Suatmi Murnani;Noor Akhmad Setiawan;Sunu Wibirama","doi":"10.1109/THMS.2024.3413781","DOIUrl":"10.1109/THMS.2024.3413781","url":null,"abstract":"The human gaze is a promising input modality for interactive applications due to its advantages: giving benefits to motion-impaired people while providing faster, intuitive, and easy interaction. The most common form of gaze interaction is object selection. During the last decade, gaze gestures and smooth pursuit-based interaction have been emerging techniques for spontaneous object selection in various gaze-controlled applications. Unfortunately, the challenge of spontaneous interaction demands no prior gaze-to-screen calibration, which leads to inaccurate object selection. To overcome the accuracy issue, this article proposes a novel method for spontaneous gaze interaction based on Pearson product-moment correlation as a measure of similarity, an exponential moving average filter for signal denoising, and a hidden Markov model to perform eye movement classification. Based on experimental results, our approach yielded the best object selection accuracy and success time of \u0000<inline-formula><tex-math>$text{89.60}pm text{10.59}%$</tex-math></inline-formula>\u0000 and \u0000<inline-formula><tex-math>$text{4364}pm text{235.86}$</tex-math></inline-formula>\u0000 ms, respectively. Our results imply that spontaneous interaction for gaze-controlled applications is possible with careful consideration of the underlying techniques to handle noisy data generated by the eye tracker. Furthermore, the proposed method is promising for future development of interactive touchless display systems that comply with the health protocols of the World Health Organization during the COVID-19 pandemic.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"485-498"},"PeriodicalIF":3.5,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10582450","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141516475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Systematic Review on Custom Data Gloves 定制数据手套系统综述

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-28 DOI: 10.1109/THMS.2024.3394674

Valerio Belcamino;Alessandro Carfì;Fulvio Mastrogiovanni

{"title":"A Systematic Review on Custom Data Gloves","authors":"Valerio Belcamino;Alessandro Carfì;Fulvio Mastrogiovanni","doi":"10.1109/THMS.2024.3394674","DOIUrl":"10.1109/THMS.2024.3394674","url":null,"abstract":"Hands are a fundamental tool humans use to interact with the environment and objects. Through hand motions, we can obtain information about the shape and materials of the surfaces we touch, modify our surroundings by interacting with objects, manipulate objects and tools, or communicate with other people by leveraging the power of gestures. For these reasons, sensorized gloves, which can collect information about hand motions and interactions, have been of interest since the 1980s in various fields, such as human–machine interaction and the analysis and control of human motions. Over the last 40 years, research in this field explored different technological approaches and contributed to the popularity of wearable custom and commercial products targeting hand sensorization. Despite a positive research trend, these instruments are not widespread yet outside research environments and devices aimed at research are often ad hoc solutions with a low chance of being reused. This article aims to provide a systematic literature review for custom gloves to analyze their main characteristics and critical issues, from the type and number of sensors to the limitations due to device encumbrance. The collection of this information lays the foundation for a standardization process necessary for future breakthroughs in this research field.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"520-535"},"PeriodicalIF":3.5,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Bilateral Teleoperation Strategy Augmented by EMGP-VH for Live-Line Maintenance Robot 由 EMGP-VH 增强的双边远程操作战略，用于现场线路维护机器人

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-26 DOI: 10.1109/THMS.2024.3412910

Shaodong Li;Peiyuan Gao;Yongzheng Chen

{"title":"A Bilateral Teleoperation Strategy Augmented by EMGP-VH for Live-Line Maintenance Robot","authors":"Shaodong Li;Peiyuan Gao;Yongzheng Chen","doi":"10.1109/THMS.2024.3412910","DOIUrl":"10.1109/THMS.2024.3412910","url":null,"abstract":"In robot-assisted live-line maintenance, bilateral teleoperation is still a popular and effective approach in assisting operators to accomplish hazards tasks. Particularly, teleoperation under overhead power lines attach greater expectation on safe operation and telepresence. In this article, we propose a visual-haptic bilateral teleoperation strategy, i.e., \u0000<italic>EMGP-VH\u0000, based on visual guidance, haptic constraint and mixed reality (MR) augmentation. To the best of our knowledge, electromagnetic field is first applied to serve the path planning of teleoperation in live-line maintenance. In visual guidance, EMG-potential fields are integrated into \u0000<italic>RRT*\u0000 to calculate a low-energy path. At the same time, real-time haptic constraint is calculated based on a tube virtual fixture. MR augmentation also works as an indispensable part in both the platform construction and visual guidance. Our proposal has been extensively compared using seven objective performances and three subjective questionnaires both in simulation and real-world experiment with five different scenes and two approaches state-of-the-art, respectively. Functionality of \u0000<italic>EMGP-RRT*\u0000 and effectiveness of haptic constraint are further analyzed. Results show that \u0000<italic>EMGP-RRT*\u0000 has significant improvements both in searching efficiency and safety performances; and the proposed system (\u0000<italic>EMGP-VH\u0000) significantly contributes to improving telepresence and ensuring safe operations during live-line maintenance, resulting in a 30% reduction in operation time and a 60% decrease in trajectory offset.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"362-374"},"PeriodicalIF":3.5,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

BERT-Based Semantic-Aware Heterogeneous Graph Embedding Method for Enhancing App Usage Prediction Accuracy 基于 BERT 的语义感知异构图嵌入法提高应用程序使用预测准确性

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-26 DOI: 10.1109/THMS.2024.3412273

Xi Fang;Hui Yang;Liu Shi;Yilong Wang;Li Li

{"title":"BERT-Based Semantic-Aware Heterogeneous Graph Embedding Method for Enhancing App Usage Prediction Accuracy","authors":"Xi Fang;Hui Yang;Liu Shi;Yilong Wang;Li Li","doi":"10.1109/THMS.2024.3412273","DOIUrl":"10.1109/THMS.2024.3412273","url":null,"abstract":"With the widespread adoption of smartphones and mobile Internet, understanding user behavior and improving user experience are critical. This article introduces semantic-aware (SA)-BERT, a novel model that integrates spatio-temporal and semantic information to represent App usage effectively. Leveraging BERT, SA-BERT captures rich contextual information. By introducing a specific objective function to represent the cooccurrence of App-time-location paths, SA-BERT can effectively model complex App usage structures. Based on this method, we adopt the learned embedding vectors in App usage prediction tasks. We evaluate the performance of SA-BERT using a large-scale real-world dataset. As demonstrated in the numerous experimental results, our model outperformed other strategies evidently. In terms of the prediction accuracy, we achieve a performance gain of 34.9% compared with widely used the SA representation learning via graph convolutional network (SA-GCN), and 134.4% than the context-aware App usage prediction with heterogeneous graph embedding. In addition, we reduced 79.27% training time compared with SA-GCN.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"465-474"},"PeriodicalIF":3.5,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Modeling Brake Perception Response Time in On-Road and Roadside Hazards Using an Integrated Cognitive Architecture 利用综合认知架构模拟公路和路边危险中的制动感知响应时间

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-25 DOI: 10.1109/THMS.2024.3408841

Umair Rehman;Shi Cao;Carolyn G. Macgregor

引用次数: 0

LANDER: Visual Analysis of Activity and Uncertainty in Surveillance Video LANDER：对监控视频中的活动和不确定性进行可视化分析

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-24 DOI: 10.1109/THMS.2024.3409722

Tong Li;Guodao Sun;Baofeng Chang;Yunchao Wang;Qi Jiang;Yuanzhong Ying;Li Jiang;Haixia Wang;Ronghua Liang

{"title":"LANDER: Visual Analysis of Activity and Uncertainty in Surveillance Video","authors":"Tong Li;Guodao Sun;Baofeng Chang;Yunchao Wang;Qi Jiang;Yuanzhong Ying;Li Jiang;Haixia Wang;Ronghua Liang","doi":"10.1109/THMS.2024.3409722","DOIUrl":"10.1109/THMS.2024.3409722","url":null,"abstract":"Vision algorithms face challenges of limited visual presentation and unreliability in pedestrian activity assessment. In this article, we introduce LANDER, an interactive analysis system for visual exploration of pedestrian activity and uncertainty in surveillance videos. This visual analytics system focuses on three common categories of uncertainties in object tracking and action recognition. LANDER offers an overview visualization of activity and uncertainty, along with spatio-temporal exploration views closely associated with the scene. Expert evaluation and user study indicate that LANDER outperforms traditional video exploration in data presentation and analysis workflow. Specifically, compared to the baseline method, it excels in reducing retrieval time (\u0000<inline-formula><tex-math>$p< $</tex-math></inline-formula>\u0000 0.01), enhancing uncertainty identification (\u0000<inline-formula><tex-math>$p< $</tex-math></inline-formula>\u0000 0.05), and improving the user experience (\u0000<inline-formula><tex-math>$p< $</tex-math></inline-formula>\u0000 0.05).","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"427-440"},"PeriodicalIF":3.5,"publicationDate":"2024-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Personalized Trajectory-based Risk Prediction on Curved Roads with Consideration of Driver Turning Behavior and Workload 基于轨迹的弯道个性化风险预测，考虑驾驶员的转弯行为和工作量

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-18 DOI: 10.1109/THMS.2024.3407333

Yahui Liu;Jingyuan Li;Yingbo Sun;Xuewu Ji;Chen Lv

{"title":"Personalized Trajectory-based Risk Prediction on Curved Roads with Consideration of Driver Turning Behavior and Workload","authors":"Yahui Liu;Jingyuan Li;Yingbo Sun;Xuewu Ji;Chen Lv","doi":"10.1109/THMS.2024.3407333","DOIUrl":"https://doi.org/10.1109/THMS.2024.3407333","url":null,"abstract":"Accurate and robust risk prediction on curved roads can significantly reduce lane departure accidents and improve traffic safety. However, limited study has considered dynamic driver-related factors in risk prediction, resulting in poor algorithm adaptiveness to individual differences. This article presents a novel personalized risk prediction method with consideration of driver turning behavior and workload by using the predicted vehicle trajectory.First, driving simulation experiments are conducted to collect synchronized trajectory data, vehicle dynamic data, and eye movement data. The drivers are distracted by answering questions via a Bluetooth headset, leading to an increased cognitive workload. Secondly, the \u0000<italic>k\u0000-means clustering algorithm is utilized to extract two turning behaviors: driving toward the inner and outer side of a curved road. The turning behavior of each trajectory is then recognized using the trajectory data. In addition, the driver workload is recognized using the vehicle dynamic features and eye movement features. Thirdly, an extra personalization index is introduced to a long short-term memory encoder–decoder trajectory prediction network. This index integrates the driver turning behavior and workload information. After introducing the personalization index, the root-mean-square errors of the proposed network are reduced by 15.6%, 23.5%, and 29.1% with prediction horizons of 2, 3, and 4 s, respectively. Fourthly, the risk potential field theory is employed for risk prediction using the predicted trajectory data. This approach implicitly incorporates the driver's personalized information into risk prediction.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"406-415"},"PeriodicalIF":3.5,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141725650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Distributed Formation Control for a Class of Human-in-the-Loop Multiagent Systems 一类人在圈多代理系统的分布式编队控制

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-17 DOI: 10.1109/THMS.2024.3398631

Xiao-Xiao Zhang;Huai-Ning Wu;Jin-Liang Wang

{"title":"Distributed Formation Control for a Class of Human-in-the-Loop Multiagent Systems","authors":"Xiao-Xiao Zhang;Huai-Ning Wu;Jin-Liang Wang","doi":"10.1109/THMS.2024.3398631","DOIUrl":"https://doi.org/10.1109/THMS.2024.3398631","url":null,"abstract":"In this article, the distributed formation control problem for a class of human-in-the-loop (HiTL) multiagent systems (MASs) is studied. A hidden Markov jump MAS is employed to model the HiTL MAS, which integrates the human models, the MAS model, and their interactions. The HiTL MAS investigated in this article is composed of two parts: a leader without human in the control loop and a group of followers in which each follower is simultaneously controlled by a human operator and an automation. For each follower, a hidden Markov model is used for modeling the human behaviors in consideration of the random nature of human internal state (HIS) reasoning and the uncertainty from HIS observation. By means of a stochastic Lyapunov function, a necessary and sufficient condition is first developed in terms of the linear matrix inequalities (LMIs) to ensure the formation of the HiTL MAS in the mean-square sense. Then, an LMI approach to the human-assistance control design is proposed for the automations in the followers to guarantee the mean-square formation of the HiTL MAS. Finally, simulation results are presented to verify the effectiveness of the proposed methods.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"416-426"},"PeriodicalIF":3.5,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141725599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Utilizing Gramian Angular Fields and Convolution Neural Networks in Flex Sensors Glove for Human–Computer Interaction 在柔性传感器手套中利用格拉米安角场和卷积神经网络实现人机交互

IF 3.5 3区计算机科学

IEEE Transactions on Human-Machine Systems Pub Date : 2024-06-06 DOI: 10.1109/THMS.2024.3404101

Chana Chansri;Jakkree Srinonchat

{"title":"Utilizing Gramian Angular Fields and Convolution Neural Networks in Flex Sensors Glove for Human–Computer Interaction","authors":"Chana Chansri;Jakkree Srinonchat","doi":"10.1109/THMS.2024.3404101","DOIUrl":"https://doi.org/10.1109/THMS.2024.3404101","url":null,"abstract":"The current sensor systems using the human–computer interface to develop a hand gesture recognition system remain challenging. This research presents the development of hand gesture recognition with 16-DoF glove sensors combined with a convolution neural network. The flex sensors are attached to 16 pivot joints of the human hand on the glove so that each knuckle flex can be measured while holding the object. The 16-DoF point sensors collecting circuit and adjustable buffer circuit were developed in this research to work with the Arduino Nano microcontroller to record each sensor's signal. This article investigates the time-series data of the flex sensor signal into 2-D colored images, concatenating the signals into one bigger image with a Gramian angular field and then recognition through a deep convolutional neural network (DCNN). The 16-DoF glove sensors were proposed for testing with three experiments using 8 models of DCNN recognition. These were conducted on 20 hand gesture recognition, 12 hand sign recognition, and object manipulation according to shape. The experimental results indicated that the best performance for the hand grasp experiment is 99.49% with Resnet 101, the hand sign experiment is 100% with Alexnet, and the object attribute experiment is 99.77% with InceptionNet V3.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 4","pages":"475-483"},"PeriodicalIF":3.5,"publicationDate":"2024-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141725620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0