Fengwei Gu;Jun Lu;Chengtao Cai;Qidan Zhu;Zhaojie Ju
{"title":"RTSformer: A Robust Toroidal Transformer With Spatiotemporal Features for Visual Tracking","authors":"Fengwei Gu;Jun Lu;Chengtao Cai;Qidan Zhu;Zhaojie Ju","doi":"10.1109/THMS.2024.3370582","DOIUrl":"10.1109/THMS.2024.3370582","url":null,"abstract":"In complex environments, trackers are extremely susceptible to some interference factors, such as fast motions, occlusion, and scale changes, which result in poor tracking performance. The reason is that trackers cannot sufficiently utilize the target feature information in these cases. Therefore, it has become a particularly critical issue in the field of visual tracking to utilize the target feature information efficiently. In this article, a composite transformer involving spatiotemporal features is proposed to achieve robust visual tracking. Our method develops a novel toroidal transformer to fully integrate features while designing a template refresh mechanism to provide temporal features efficiently. Combined with the hybrid attention mechanism, the composite of temporal and spatial feature information is more conducive to mining feature associations between the template and search region than a single feature. To further correlate the global information, the proposed method adopts a closed-loop structure of the toroidal transformer formed by the cross-feature fusion head to integrate features. Moreover, the designed score head is used as a basis for judging whether the template is refreshed. Ultimately, the proposed tracker can achieve the tracking task only through a simple network framework, which especially simplifies the existing tracking architectures. Experiments show that the proposed tracker outperforms extensive state-of-the-art methods on seven benchmarks at a real-time speed of 56.5 fps.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140166154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Balint K. Hodossy;Annika S. Guez;Shibo Jing;Weiguang Huo;Ravi Vaidyanathan;Dario Farina
{"title":"Leveraging High-Density EMG to Investigate Bipolar Electrode Placement for Gait Prediction Models","authors":"Balint K. Hodossy;Annika S. Guez;Shibo Jing;Weiguang Huo;Ravi Vaidyanathan;Dario Farina","doi":"10.1109/THMS.2024.3371099","DOIUrl":"10.1109/THMS.2024.3371099","url":null,"abstract":"To control wearable robotic systems, it is critical to obtain a prediction of the user's motion intent with high accuracy. Surface electromyography (sEMG) recordings have often been used as inputs for these devices, however bipolar sEMG electrodes are highly sensitive to their location. Positional shifts of electrodes after training gait prediction models can therefore result in severe performance degradation. This study uses high-density sEMG (HD-sEMG) electrodes to simulate various bipolar electrode signals from four leg muscles during steady-state walking. The bipolar signals were ranked based on the consistency of the corresponding sEMG envelope's activity and timing across gait cycles. The locations were then compared by evaluating the performance of an offline temporal convolutional network (TCN) that mapped sEMG signals to knee angles. The results showed that electrode locations with consistent sEMG envelopes resulted in greater prediction accuracy compared to hand-aligned placements (\u0000<italic>p</i>\u0000 < 0.01). However, performance gains through this process were limited, and did not resolve the position shift issue. Instead of training a model for a single location, we showed that randomly sampling bipolar combinations across the HD-sEMG grid during training mitigated this effect. Models trained with this method generalized over all positions, and achieved 70% less prediction error than location specific models over the entire area of the grid. Therefore, the use of HD-sEMG grids to build training datasets could enable the development of models robust to spatial variations, and reduce the impact of muscle-specific electrode placement on accuracy.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10474208","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140166273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automated Classification of Cognitive Visual Objects Using Multivariate Swarm Sparse Decomposition From Multichannel EEG-MEG Signals","authors":"Shailesh Vitthalrao Bhalerao;Ram Bilas Pachori","doi":"10.1109/THMS.2024.3395153","DOIUrl":"10.1109/THMS.2024.3395153","url":null,"abstract":"In visual object decoding, magnetoencephalogram (MEG) and electroencephalogram (EEG) activation patterns demonstrate the utmost discriminative cognitive analysis due to their multivariate oscillatory nature. However, high noise in the recorded EEG-MEG signals and subject-specific variability make it extremely difficult to classify subject's cognitive responses to different visual stimuli. The proposed method is a multivariate extension of the swarm sparse decomposition method (MSSDM) for multivariate pattern analysis of EEG-MEG-based visual activation signals. In comparison, it is an advanced technique for decomposing nonstationary multicomponent signals into a finite number of channel-aligned oscillatory components that significantly enhance visual activation-related sub-bands. The MSSDM method adopts multivariate swarm filtering and sparse spectrum to automatically deliver optimal frequency bands in channel-specific sparse spectrums, resulting in improved filter banks. By combining the advantages of the multivariate SSDM and Riemann's correlation-assisted fusion feature (RCFF), the MSSDM-RCFF algorithm is investigated to improve the visual object recognition ability of EEG-MEG signals. We have also proposed time–frequency representation based on MSSDM to analyze discriminative cognitive patterns of different visual object classes from multichannel EEG-MEG signals. A proposed MSSDM is evaluated on multivariate synthetic signals and multivariate EEG-MEG signals using five classifiers. The proposed fusion feature and linear discriminant analysis classifier-based framework outperformed all existing state-of-the-art methods used for visual object detection and achieved the highest accuracy of 86.42% using tenfold cross-validation on EEG-MEG multichannel signals.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.5,"publicationDate":"2024-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anne Tryphosa Kamatham;Meena Alzamani;Allison Dockum;Siddhartha Sikdar;Biswarup Mukherjee
{"title":"SonoMyoNet: A Convolutional Neural Network for Predicting Isometric Force From Highly Sparse Ultrasound Images","authors":"Anne Tryphosa Kamatham;Meena Alzamani;Allison Dockum;Siddhartha Sikdar;Biswarup Mukherjee","doi":"10.1109/THMS.2024.3389690","DOIUrl":"10.1109/THMS.2024.3389690","url":null,"abstract":"Ultrasound imaging or sonomyography has been found to be a robust modality for measuring muscle activity due to its ability to image deep-seated muscles directly while providing superior spatiotemporal specificity compared with surface electromyography-based techniques. Quantifying the morphological changes during muscle activity involves computationally expensive approaches for tracking muscle anatomical structures or extracting features from brightness-mode (B-mode) images and amplitude-mode signals. This article uses an offline regression convolutional neural network called SonoMyoNet to estimate continuous isometric force from sparse ultrasound scanlines. SonoMyoNet learns features from a few equispaced scanlines selected from B-mode images and utilizes the learned features to estimate continuous isometric force accurately. The performance of SonoMyoNet was evaluated by varying the number of scanlines to simulate the placement of multiple single-element ultrasound transducers in a wearable system. Results showed that SonoMyoNet could accurately predict isometric force with just four scanlines and is immune to speckle noise and shifts in the scanline location. Thus, the proposed network reduces the computational load involved in feature tracking algorithms and estimates muscle force from the global features of sparse ultrasound images.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140939947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hand Segmentation With Dense Dilated U-Net and Structurally Incoherent Nonnegative Matrix Factorization-Based Gesture Recognition","authors":"Kankana Roy;Rajiv R. Sahay","doi":"10.1109/THMS.2024.3390415","DOIUrl":"10.1109/THMS.2024.3390415","url":null,"abstract":"Robust segmentation of hands in a cluttered environment for hand gesture recognition has remained a challenge in computer vision. In this work, a two-stage gesture recognition framework is proposed. In the first stage, we segment hands using the proposed deep learning algorithm, and in the second stage, we use these segmented hands to classify gestures using a novel structurally incoherent nonnegative matrix factorization approach. We propose a new deep learning framework for hand segmentation called densely dilated U-Net. We exploit recently proposed dense blocks and dilated convolution layers in our work. To cope with the scarcity of labeled datasets we extend our densely dilated U-Net for semisupervised hand segmentation using hand bounding boxes as cues. We provide quantitative and qualitative evaluation of proposed hand segmentation model on several public hand segmentation datasets including EgoHands, GTEA, EYTH, EDSH, and HOF. Semisupervised segmentation results are also obtained on two hand detection datasets including VIVA and CVRR. As an extension of our work, we show semisupervised segmentation and gesture recognition results using segmented hands on NUS-II cluttered hand gesture dataset. To validate the efficiency of our semisupervised algorithm we evaluate it on OUHands dataset with real ground truth labels. For gesture classification, we propose a novel structurally incoherent nonnegative matrix factorization algorithm. We propose to use CNN features extracted from segmented images for nonnegative matrix factorization. Experimental results on NUS-II and OUHands datasets demonstrate that our two-stage approach for gesture recognition yields superior results.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140939745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Working Conditions of Industrial Robot Operators–An Overview of Technology Dissemination, Job Characteristics, and Health Indicators in Modern Production Workplaces","authors":"Matthias Hartwig;Patricia Rosen;Sascha Wischniewski","doi":"10.1109/THMS.2024.3368525","DOIUrl":"10.1109/THMS.2024.3368525","url":null,"abstract":"Flexible robotic systems change not only the production workflow as a whole but also the individual working conditions of their operators. The aim of this analysis of our study with more than 5900 participants was to get an overview of demographics, job characteristics, and health indicators of robot operators in comparison to nonrobotic machine operators and employees in Germany. We collected data by telephone interviews measuring technology use, stressors, and resources at work as well as health indicators. Results indicate systematic differences in working stressors and resources for robot users compared to other machine users as well as employees in general. In particular, the scope for decision-making at work was smaller for robot users, especially regarding the amount of work or the speed of work. Only isolated links could be found regarding the health indicators. The results therefore imply constant consideration of human factors to ensure productive as well as healthy working conditions with robots in modern industry.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140055231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zemin Liu;Qingsong Ai;Haojie Liu;Wei Meng;Quan Liu
{"title":"Human-Like Trajectory Planning Based on Postural Synergistic Kernelized Movement Primitives for Robot-Assisted Rehabilitation","authors":"Zemin Liu;Qingsong Ai;Haojie Liu;Wei Meng;Quan Liu","doi":"10.1109/THMS.2024.3360111","DOIUrl":"10.1109/THMS.2024.3360111","url":null,"abstract":"The motor synergy pattern is an intrinsic characteristic found in natural human movements, particularly in the upper limb. It is essential to improve the multijoint coordination ability for stroke patients by integrating the synergy pattern into rehabilitation tasks and trajectory design. However, current robot-assisted rehabilitation systems tend to overlook the incorporation of a multijoint synergy model. This article proposes postural synergistic kernelized movement primitives (PSKMP) method for the human-like trajectory planning of robot-assisted upper limb rehabilitation. First, the demonstrated trajectory obtained from the motion capture system is subject to principal component analysis to extract postural synergies. Then, the PSKMP is proposed by kernelizing the postural synergistic subspaces with the kernel treatment to preserve human natural movement characteristics. Finally, the rehabilitation trajectory accord with human motion habits can be generated based on generalized postural synergistic subspaces. This approach has undergone practical validation on an upper limb rehabilitation robot, and the experimental results show that the proposed method enables the generation of human-like trajectories adapted to new task points, in accordance with the natural movement style of human. This method holds great significance in promoting the recovery of coordination ability of stroke patients.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Extracting Human Levels of Trust in Human–Swarm Interaction Using EEG Signals","authors":"Jesus A. Orozco;Panagiotis Artemiadis","doi":"10.1109/THMS.2024.3356421","DOIUrl":"10.1109/THMS.2024.3356421","url":null,"abstract":"Trust is an essential building block of human civilization. However, when it relates to artificial systems, it has been a barrier to intelligent technology adoption in general. This article addresses the gap in determining levels of trust in scenarios that include humans interacting with a swarm of robots. Electroencephalography (EEG) recordings of the human observers of the different swarms allow for extracting specific EEG features related to different trust levels. Feature selection and machine learning methods comprise a classification system that would allow recognition of different levels of human trust in those human–swarm interaction scenarios. The results of this study suggest that EEG correlates of swarm trust exist and are distinguishable in machine learning feature classification with very high accuracy. Moreover, comparing common EEG features across all human subjects used in this study allows for the generalization of the classification method, providing solid evidence of specific areas and features of the human brain where activations are related to levels of human–swarm trust. This work has direct implications for effective human–machine teaming with applications to many fields, such as exploration, search and rescue operations, surveillance, environmental monitoring, and defense. In these applications, quantifying levels of human trust in the deployed swarm is of utmost importance because it can lead to swarm controllers that adapt their output based on the human's perceived trust level.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xin Lin;Shucong Yin;Hao Du;Yuquan Leng;Chenglong Fu
{"title":"Design and Investigation of a Suspended Backpack With Wide-Range Variable Stiffness Suspension for Reducing Energetic Cost","authors":"Xin Lin;Shucong Yin;Hao Du;Yuquan Leng;Chenglong Fu","doi":"10.1109/THMS.2024.3355474","DOIUrl":"10.1109/THMS.2024.3355474","url":null,"abstract":"Suspended backpacks have been acknowledged for their advantages in load carriage, leading to the development of various designs aimed at enhancing their performance. However, current suspended backpacks typically possess fixed stiffness or limited adjustability, thereby limiting their adaptability to different load carriage tasks, such as varying walking speeds and load masses. This article introduced a suspended backpack design capable of modulating its stiffness over a wide range while maintaining a lightweight profile. The variable stiffness suspension (VSS) was integrated into the load frame of the suspended backpack and utilized a motor to adjust the stiffness by generating spring-like force based on the relative displacement between the load and the body. Experimental validation was conducted to assess the stiffness modulation of the suspended backpack. The VSS enabled the stiffness modulation of the suspended backpack ranging from 424 to 2182 N/m, which corresponded to the desired stiffness range for a 10–25 kg load at walking speeds for 3.5–6 km/h. Moreover, the mechanics of the carriers were analyzed to evaluate the impact of the suspended backpack on the individuals. Results showed that the designed VSS suspended backpack could reduce peak push-off force by 20.71% under the high working condition and energetic cost by 30.39% under the midworking condition. However, a tradeoff exists between minimizing the peak accelerative load force and energetic cost. The proposed design holds the potential for enhancing performance across various load carriage tasks, including human-in-the-loop energetic optimization.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alejandro L. Callara;Gianluca Rho;Sara Condino;Vincenzo Ferrari;Enzo Pasquale Scilingo;Alberto Greco
{"title":"Behavioral, Peripheral, and Central Neural Correlates of Augmented Reality Guidance of Manual Tasks","authors":"Alejandro L. Callara;Gianluca Rho;Sara Condino;Vincenzo Ferrari;Enzo Pasquale Scilingo;Alberto Greco","doi":"10.1109/THMS.2024.3354413","DOIUrl":"10.1109/THMS.2024.3354413","url":null,"abstract":"Objective: The use of commercially available optical-see-through (OST) head-mounted displays (HMDs) in their own peripersonal space leads the user to experience two perception conflicts that deteriorate their performance in precision manual tasks: the vergence-accommodation conflict (VAC) and the focus rivalry. In this work, we aim characterizing for the first time the psychophysiological response associated with user's incorrect focus cues during the execution of an augmented reality (AR)-guided manual task with the Microsoft HoloLens OST-HMD. Methods: 21 subjects underwent to a “connecting-the-dots” experiment with and without the use of AR, and in both binocular and monocular conditions. For each condition, we quantified the changes in autonomic nervous system (ANS) activity of subjects by analyzing the electrodermal activity (EDA) and heart rate variability. Moreover, we analyzed the neural central correlates by means of power measures of brain activity and multivariate autoregressive measures of brain connectivity extracted from the electroencephalogram (EEG). Results: No statistically significant differences of ANS correlates were observed among tasks, although all EDA-related features varied between rest and task conditions. Conversely, significant differences among conditions were present in terms of EEG-power variations in the \u0000<inline-formula><tex-math>$mu$</tex-math></inline-formula>\u0000 (8–13) Hz and \u0000<inline-formula><tex-math>$beta$</tex-math></inline-formula>\u0000 (13–30) Hz bands. In addition, significant changes in the causal interactions of a brain network involved in motor movement and eye-hand coordination comprising the precentral gyrus, the precuneus, and the fusiform gyrus were observed. Conclusion: The physiological plausibility of our results suggest promising future applicability to investigate more complex scenarios, such as AR-guided surgery.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10418203","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}