{"title":"IEEE Transactions on Human-Machine Systems Information for Authors","authors":"","doi":"10.1109/THMS.2024.3458755","DOIUrl":"https://doi.org/10.1109/THMS.2024.3458755","url":null,"abstract":"","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"C4-C4"},"PeriodicalIF":3.5,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10684408","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hongguang Pan;Zhuoyi Li;Yunpeng Fu;Xuebin Qin;Jianchen Hu
{"title":"Reconstructing Visual Stimulus Representation From EEG Signals Based on Deep Visual Representation Model","authors":"Hongguang Pan;Zhuoyi Li;Yunpeng Fu;Xuebin Qin;Jianchen Hu","doi":"10.1109/THMS.2024.3407875","DOIUrl":"10.1109/THMS.2024.3407875","url":null,"abstract":"Reconstructing visual stimulus representation is a significant task in neural decoding. Until now, most studies have considered functional magnetic resonance imaging (fMRI) as the signal source. However, fMRI-based image reconstruction methods are challenging to apply widely due to the complexity and high cost of acquisition equipment. Taking into account the advantages of the low cost and easy portability of electroencephalogram (EEG) acquisition equipment, we propose a novel image reconstruction method based on EEG signals in this article. First, to meet the high recognizability of visual stimulus images in a fast-switching manner, we construct a visual stimuli image dataset and obtain the corresponding EEG dataset through EEG signals collection experiment. Second, we introduce the deep visual representation model (DVRM), comprising a primary encoder and a subordinate decoder, to reconstruct visual stimuli representation. The encoder is designed based on residual-in-residual dense blocks to learn the distribution characteristics between EEG signals and visual stimulus images. Meanwhile, the decoder is designed using a deep neural network to reconstruct the visual stimulus representation from the learned deep visual representation. The DVRM can accommodate the deep and multiview visual features of the human natural state, resulting in more precise reconstructed images. Finally, we evaluate the DVRM based on the quality of the generated images using our EEG dataset. The results demonstrate that the DVRM exhibits an excellent performance in learning deep visual representation from EEG signals, generating reconstructed representation of images that are realistic and highly resemble the original images.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"711-722"},"PeriodicalIF":3.5,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142254852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maurice Kolff;Joost Venrooij;Markus Schwienbacher;Daan M. Pool;Max Mulder
{"title":"Reliability and Models of Subjective Motion Incongruence Ratings in Urban Driving Simulations","authors":"Maurice Kolff;Joost Venrooij;Markus Schwienbacher;Daan M. Pool;Max Mulder","doi":"10.1109/THMS.2024.3450831","DOIUrl":"10.1109/THMS.2024.3450831","url":null,"abstract":"In moving-base driving simulators, the sensation of the inertial car motion provided by the motion system is controlled by the motion cueing algorithm (MCA). Due to the difficulty of reproducing the inertial motion in urban simulations, accurate prediction tools for subjective evaluation of the simulator's inertial motion are required. In this article, an open-loop driving experiment in an urban scenario is discussed, in which 60 participants evaluated the motion cueing through an overall rating and a continuous rating method. Three MCAs were tested that represent different levels of motion cueing quality. It is investigated under which conditions the continuous rating method provides reliable data in urban scenarios through the estimation of Cronbach's alpha and McDonald's omega. Results show that the \u0000<italic>better</i>\u0000 the motion cueing is rated, the \u0000<italic>lower</i>\u0000 the reliability of that rating data is, and the less the continuous rating and overall rating correlate. This suggests that subjective ratings for motion quality are dominated by (moments of) incongruent motion, while congruent motion is less important. Furthermore, through a forward regression approach, it is shown that participants' rating behavior can be described by a first-order low-pass filtered response to the lateral specific force mismatch (66.0%), as well as a similar response to the longitudinal specific force mismatch (34.0%). By this better understanding of the acquired ratings in urban driving simulations, including their reliability and predictability, incongruences can be more accurately targeted and reduced.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"634-645"},"PeriodicalIF":3.5,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142254853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Building Contextualized Trust Profiles in Conditionally Automated Driving","authors":"Lilit Avetisyan;Jackie Ayoub;X. Jessie Yang;Feng Zhou","doi":"10.1109/THMS.2024.3452411","DOIUrl":"10.1109/THMS.2024.3452411","url":null,"abstract":"Trust is crucial for ensuring the safety, security, and widespread adoption of automated vehicles (AVs), and if trust is lacking, drivers and the general public may hesitate to embrace this technology. This research seeks to investigate contextualized trust profiles in order to create personalized experiences for drivers in AVs with varying levels of reliability. A driving simulator experiment involving 70 participants revealed three distinct contextualized trust profiles (i.e., \u0000<italic>confident copilots</i>\u0000, \u0000<italic>myopic pragmatists</i>\u0000, and \u0000<italic>reluctant automators</i>\u0000) identified through K-means clustering, and analyzed in relation to drivers' dynamic trust, dispositional trust, initial learned trust, personality traits, and emotions. The experiment encompassed eight scenarios where participants were requested to take over control from the AV in three conditions: a control condition, a false alarm condition, and a miss condition. To validate the models, a multinomial logistic regression model was constructed using the shapley additive explanations explainer to determine the most influential features in predicting contextualized trust profiles, achieving an F1-score of 0.90 and an accuracy of 0.89. In addition, an examination of how individual factors impact contextualized trust profiles provided valuable insights into trust dynamics from a user-centric perspective. The outcomes of this research hold significant implications for the development of personalized in-vehicle trust monitoring and calibration systems to modulate drivers' trust levels, thereby enhancing safety and user experience in automated driving.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"658-667"},"PeriodicalIF":3.5,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring Factors Related to Drivers’ Mental Model of and Trust in Advanced Driver Assistance Systems Using an ABN-Based Mixed Approach","authors":"Chunxi Huang;Jiyao Wang;Song Yan;Dengbo He","doi":"10.1109/THMS.2024.3436876","DOIUrl":"10.1109/THMS.2024.3436876","url":null,"abstract":"Drivers’ appropriate mental models of and trust in advanced driver assistance systems (ADAS) are essential to driving safety in vehicles with ADAS. Although several previous studies evaluated drivers’ ADAS mental models of and trust in adaptive cruise control and lane-keeping assist systems, research gaps still exist. Specifically, recent developments in ADAS have made more advanced functions available but they have been under-investigated. Furthermore, the widely adopted proportional correctness-based scores may not differentiate drivers’ objective ADAS mental model and subjective bias toward the ADAS. Finally, most previous studies adopted only regression models to explore the influential factors and thus may have ignored the underlying association among the factors. Therefore, our study aimed to explore drivers’ mental models of and trust in emerging ADAS by using the sensitivity (i.e., \u0000<italic>d’</i>\u0000) and response bias (i.e., \u0000<italic>c</i>\u0000) measures from the signal detection theory. We modeled the data from 287 drivers using additive Bayesian network (ABN) and further interpreted the graph model using regression analysis. We found that different factors might be associated with drivers’ objective knowledge of ADAS and subjective bias toward the existence of functions/limitations. Furthermore, drivers’ subjective bias was more associated with their trust in ADAS compared to objective knowledge. The findings from our study provide new insights into the influential factors on drivers’ mental models of ADAS and better reveal how mental models can affect trust in ADAS. It also provides a case study on how the mixed approach with ABN and regression analysis can model observational data.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"646-657"},"PeriodicalIF":3.5,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fusion of Temporal Transformer and Spatial Graph Convolutional Network for 3-D Skeleton-Parts-Based Human Motion Prediction","authors":"Mayank Lovanshi;Vivek Tiwari;Rajesh Ingle;Swati Jain","doi":"10.1109/THMS.2024.3452133","DOIUrl":"10.1109/THMS.2024.3452133","url":null,"abstract":"The field of human motion prediction has gained prominence, finding applications in various domains such as intelligent surveillance and human–robot interaction. However, predicting full-body human motion poses challenges in capturing joint interactions, handling diverse movement patterns, managing occlusions, and ensuring real-time performance. To address these challenges, the proposed model adopts a skeleton-parted strategy to dissect the skeleton structure, enhancing coordination and fusion between body parts. This novel method combines transformer-enabled graph convolutional networks for predicting human motion in 3-D skeleton data. It integrates a temporal transformer (T-Transformer) for comprehensive temporal feature extraction and a spatial graph convolutional network (S-GCN) for capturing spatial characteristics of human motion. The model's performance is evaluated on two comprehensive human motion datasets, Human3.6M and CMU motion capture (CMU Mocap), containing numerous videos encompassing short and long human motion sequences. Results indicate that the proposed model outperforms state-of-the-art methods on both datasets, significantly improving the average mean per joint positional error (avg-MPJPE) by 3.50% and 11.45% for short-term and long-term motion prediction, respectively. Similarly, on the CMU Mocap dataset, it achieves avg-MPJPE improvements of 2.69% and 1.05% for short-term and long-term motion prediction, respectively, demonstrating its superior accuracy in predicting human motion over extended periods. The study also investigates the impact of different numbers of T-Transformers and S-GCNs and explores the specific roles and contributions of the T-Transformer, S-GCN, and cross-part components.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"788-797"},"PeriodicalIF":3.5,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Design of a Robotic System Featured With High Operation Transparency for Quantifying Arm Impedance During Ultrasound Scanning","authors":"Baoshan Niu;Dapeng Yang;Yangjunjian Zhou;Le Zhang;Qi Huang;Yikun Gu","doi":"10.1109/THMS.2024.3442537","DOIUrl":"10.1109/THMS.2024.3442537","url":null,"abstract":"Experienced sonographers can adjust their arm impedance in real-time to obtain high-quality ultrasound (US) images during US scanning. These operational skills can be captured through robot systems with multimodal data collection capabilities (position, force, and impedance). However, low operational transparency between the system (generally, a serial robot with admittance control) and its users will result in significant delays and errors, interfering with the skill acquisition process. The paper proposes a new system that adopts the parallel mechanism (Omega.7) to improve the transparency of the operation. The scanning probe and a 6-axis force sensor are attached to the end of Omega.7. When operating the probe, a zero-force drag effect can be realized through gravity and torque compensations. The arm impedance during the scanning can be measured through the force disturbance method by analyzing external forces on the device. Ultrasonic scans were conducted on phantoms of different hardness, and arm impedance was measured. Statistical analysis reveals that when scanning softer phantoms, arms exhibit higher stiffness. The transparency analysis results show that the equipment designed in this paper has a higher level of transparency than the scheme of serial robot with admittance control. The high operation transparency of the system makes it an ideal skill-acquisition device with broad applications.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"798-807"},"PeriodicalIF":3.5,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Development of a MR Training System for Living Donor Liver Transplantation Using Simulated Liver Phantom and ICP Tracking Technology","authors":"Tsung-Han Yang;Yi-Chun Du;Cheng-Bin Xu;Wei-Siang Ciou","doi":"10.1109/THMS.2024.3450689","DOIUrl":"10.1109/THMS.2024.3450689","url":null,"abstract":"Living donor liver transplantation (LT) is a curative treatment for decompensation liver cirrhosis, some metabolic diseases, and acute liver failure. For specific conditions of hepatocellular carcinoma, LT provides a better prognosis than other known treatments do. During living donor LT, recognition and preservation of the middle hepatic vein (MHV) and its main branch are extremely important and closely related to the outcomes for the donor and recipient. Currently, preoperative computed tomography (CT) scans and intraoperative ultrasound are used to evaluate the location of the MHV; however, the information from CT scans and ultrasound is two-dimensional and lacks specific perception data. To achieve better MHV tracking during surgery, this work presents a mixed-reality (MR) training system for open liver LT surgery, which uses a simulated elastic liver phantom and iterative closest point (ICP) tracking technology. We created a three-dimensional (3-D) liver reconstruction model based on CT images from 20 patients and produced a series of equal-sized elastic liver phantoms with soft vessels inside. The ICP algorithm was used to track the liver phantom with the MR system, and the 3-D reconstruction model was superimposed on the phantom. The experimental results revealed that the registration error was <4 mm. The feedback from ten novice surgeons who practiced with the proposed system was positive. It is expected that the proposed system for LT could enhance the overall effectiveness of surgeon training and serve as a reference for other applications in the future.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"678-687"},"PeriodicalIF":3.5,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Layered Modeling of Affective, Perception, and Visual Properties: Optimizing Structure With Genetic Algorithm","authors":"Shuhei Watanabe;Takahiko Horiuchi","doi":"10.1109/THMS.2024.3434573","DOIUrl":"10.1109/THMS.2024.3434573","url":null,"abstract":"To design the “Kansei value” aspect of a product, it is useful to design multilayered relationships of perceptual and affective responses via the physical or psychophysical properties of the product. However, because they are qualitative and ambiguous, designing a model is time-consuming. Moreover, the design was conducted by hypothesis and trial-and-error by the experimenter. In this article, we developed a method to automatically construct several semioptimal structures by applying a genetic algorithm to model design based on structural equation modeling, using the results of image measurement and subjective evaluation experiments on various material samples. Under set convergence conditions, the method constructed statistically optimized structures that represent the relationships among adjectives describing perception and affective, and the properties. A semantic validation was performed to determine the final model. As a result, the proposed method could be used to construct a model that can be interpreted as semantically and statistically superior compared to methods in related studies. A unique feature of this article was the use of the physical and psychophysical properties obtained by measurements in the construction of a multilayer model. Also, the advantage of this method is that it can be used to construct important structures that may be overlooked.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"609-618"},"PeriodicalIF":3.5,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"To Err is Automation: Can Trust be Repaired by the Automated Driving System After its Failure?","authors":"Peng Liu;Yueying Chu;Guanqun Wang;Zhigang Xu","doi":"10.1109/THMS.2024.3434680","DOIUrl":"10.1109/THMS.2024.3434680","url":null,"abstract":"Failures of the automated driving system (ADS) in automated vehicles (AVs) can damage driver–ADS cooperation (e.g., causing trust damage) and traffic safety. Researchers suggest infusing a human-like ability, active trust repair, into automated systems, to mitigate broken trust and other negative impacts resulting from their failures. Trust repair is regarded as a key ergonomic design in automated systems. Trust repair strategies (e.g., apology) are examined and supported by some evidence in controlled environments, however, rarely subjected to empirical evaluations in more naturalistic environments. To fill this gap, we conducted a test track study, invited participants (\u0000<italic>N</i>\u0000 = 257) to experience an ADS failure, and tested the influence of the ADS’ trust repair on trust and other psychological responses. Half of participants (\u0000<italic>n</i>\u0000 = 128) received the ADS’ verbal message (consisting of apology, explanation, and promise) by a human voice (\u0000<italic>n</i>\u0000 = 63) or by Apple's Siri (\u0000<italic>n</i>\u0000 = 65) after its failure. We measured seven psychological responses to AVs and ADS [e.g., trust and behavioral intention (BI)]. We found that both strategies cannot repair damaged trust. The human-voice-repair strategy can to some degree mitigate other detrimental influences (e.g., reductions in BI) resulting from the ADS failure, but this effect is only notable among participants without substantial driving experience. It points to the importance of conducting ecologically valid and field studies for validating human-like trust repair strategies in human–automation interaction and of developing trust repair strategies specific to safety-critical situations.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"508-519"},"PeriodicalIF":3.5,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}