Virtual Reality Intelligent Hardware最新文献

{"title":"A simple, stroke-based method for gesture drawing","authors":"Lesley Istead ,&nbsp;Joe Istead ,&nbsp;Andreea Pocol ,&nbsp;Craig S. Kaplan","doi":"10.1016/j.vrih.2022.08.004","DOIUrl":"10.1016/j.vrih.2022.08.004","url":null,"abstract":"<div><h3>Background</h3><p>Gesture drawing is a type of fluid, fast sketch with loose and roughly drawn lines that captures the motion and feeling of a subject. Although style transfer methods, which are able to learn a style from an input image and apply it to a secondary image, can reproduce many styles, they are currently unable to produce the flowing strokes of gesture drawings.</p></div><div><h3>Method</h3><p>In this paper, we present a method for producing gesture drawings that roughly depict objects or scenes with loose dancing contours and frantic textures. By following a gradient field, our method adapts stroke-based painterly rendering algorithms to produce long curved strokes. A rough, overdrawn appearance is created through a progressive refinement. In addition, we produce rough hatch strokes by altering the stroke direction. These add optional shading to gesture drawings.</p></div><div><h3>Results</h3><p>A wealth of parameters provide users the ability to adjust the output style, from short and rapid strokes to long and fluid strokes, and from swirling to straight lines. Potential stylistic outputs include pen-and-ink and colored pencil. We present several generated gesture drawings and discuss the application of our method to video.</p></div><div><h3>Conclusion</h3><p>Our stroke-based rendering algorithm produces convincing gesture drawings with numerous controllable parameters, permitting the creation of a variety of styles.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000791/pdf?md5=5d6ede6955247fdfc333f73a0cddaa0d&pid=1-s2.0-S2096579622000791-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122622709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"RADepthNet: Reflectance-aware monocular depth estimation","authors":"Chuxuan Li ,&nbsp;Ran Yi ,&nbsp;Saba Ghazanfar Ali ,&nbsp;Lizhuang Ma ,&nbsp;Enhua Wu ,&nbsp;Jihong Wang ,&nbsp;Lijuan Mao ,&nbsp;Bin Sheng","doi":"10.1016/j.vrih.2022.08.005","DOIUrl":"10.1016/j.vrih.2022.08.005","url":null,"abstract":"<div><h3>Background</h3><p>Monocular depth estimation aims to predict a dense depth map from a single RGB image, and has important applications in 3D reconstruction, automatic driving, and augmented reality. However, existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy, which leads to inferior performance.</p></div><div><h3>Methods</h3><p>To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy, we propose RADepthNet, a novel reflectance-guided network that fuses boundary features. Specifically, our method predicts depth maps using the following three steps: (1) Intrinsic Image Decomposition. We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance. Through an ablation study, we demonstrate that the module can reduce the influence of illumination on depth estimation. (2) Boundary Detection. A boundary extraction module, consisting of an encoder, refinement block, and upsample block, was proposed to better predict the depth at object boundaries utilizing gradient constraints. (3) Depth Prediction Module<strong>.</strong> We use an encoder different from (2) to obtain depth features from the reflectance map and fuse boundary features to predict depth. In addition, we proposed FIFADataset, a depth-estimation dataset applied in soccer scenarios.</p></div><div><h3>Results</h3><p>Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000808/pdf?md5=fc1d9cddf0180762f5b3a461f1d2e01d&pid=1-s2.0-S2096579622000808-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116232217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Integrating digital twins and deep learning for medical image analysis in the era of COVID-19","authors":"Imran Ahmed ,&nbsp;Misbah Ahmad ,&nbsp;Gwanggil Jeon","doi":"10.1016/j.vrih.2022.03.002","DOIUrl":"10.1016/j.vrih.2022.03.002","url":null,"abstract":"<div><h3>Background</h3><p>Digital twins are virtual representations of devices and processes that capture the physical properties of the environment and operational algorithms/techniques in the context of medical devices and technologies. Digital twins may allow healthcare organizations to determine methods of improving medical processes, enhancing patient experience, lowering operating expenses, and extending the value of care. During the present COVID-19 pandemic, various medical devices, such as X-rays and CT scan machines and processes, are constantly being used to collect and analyze medical images. When collecting and processing an extensive volume of data in the form of images, machines and processes sometimes suffer from system failures, creating critical issues for hospitals and patients.</p></div><div><h3>Methods</h3><p>To address this, we introduce a digital-twin-based smart healthcare system integrated with medical devices to collect information regarding the current health condition, configuration, and maintenance history of the device/machine/system. Furthermore, medical images, that is, X-rays, are analyzed by using a deep-learning model to detect the infection of COVID-19. The designed system is based on the cascade recurrent convolution neural network (RCNN) architecture. In this architecture, the detector stages are deeper and more sequentially selective against small and close false positives. This architecture is a multi-stage extension of the RCNN model and sequentially trained using the output of one stage for training the other. At each stage, the bounding boxes are adjusted to locate a suitable value of the nearest false positives during the training of the different stages. In this manner, the arrangement of detectors is adjusted to increase the intersection over union, overcoming the problem of overfitting. We train the model by using X-ray images as the model was previously trained on another dataset.</p></div><div><h3>Results</h3><p>The developed system achieves good accuracy during the detection phase of COVID-19. The experimental outcomes reveal the efficiency of the detection architecture, which yields a mean average precision rate of 0.94.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000183/pdf?md5=1f5a53060a043bec60b5fd3de876ef4d&pid=1-s2.0-S2096579622000183-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42600248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Digital twin intelligent system for industrial internet of things-based big data management and analysis in cloud environments","authors":"Christos L. Stergiou,&nbsp;Kostas E. Psannis","doi":"10.1016/j.vrih.2022.05.003","DOIUrl":"10.1016/j.vrih.2022.05.003","url":null,"abstract":"<div><p>This work surveys and illustrates multiple open challenges in the field of industrial Internet of Things (IoT)-based big data management and analysis in cloud environments. Challenges arising from the fields of machine learning in cloud infrastructures, artificial intelligence techniques for big data analytics in cloud environments, and federated learning cloud systems are elucidated. Additionally, reinforcement learning, which is a novel technique that allows large cloud-based data centers, to allocate more energy-efficient resources is examined. Moreover, we propose an architecture that attempts to combine the features offered by several cloud providers to achieve an energy-efficient industrial IoT-based big data management framework (EEIBDM) established outside of every user in the cloud. IoT data can be integrated with techniques such as reinforcement and federated learning to achieve a digital twin scenario for the virtual representation of industrial IoT-based big data of machines and room temperatures. Furthermore, we propose an algorithm for determining the energy consumption of the infrastructure by evaluating the EEIBDM framework. Finally, future directions for the expansion of this research are discussed.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000444/pdf?md5=77ac7ba395219ea4a1f3583a51767386&pid=1-s2.0-S2096579622000444-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132922825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Deep inside molecules — digital twins at the nanoscale","authors":"Marc Baaden","doi":"10.1016/j.vrih.2022.03.001","DOIUrl":"10.1016/j.vrih.2022.03.001","url":null,"abstract":"<div><h3>Background</h3><p>Digital twins offer rich potential for exploration in virtual reality (VR). Using interactive molecular simulation approaches, they enable a human operator to access the physical properties of molecular objects and to build, manipulate, and study their assemblies. Integrative modeling and drug design are important applications of this technology.</p></div><div><h3>Methods</h3><p>In this study, head-mounted virtual reality displays connected to molecular simulation engines were used to create interactive and immersive digital twins. They were used to perform tasks relevant to specific use cases.</p></div><div><h3>Results</h3><p>Three areas were investigated, including model building, rational design, and tangible models. Here, we report several membrane-embedded systems of ion channels, viral components, and artificial water channels. We were able to improve and create molecular designs based on digital twins.</p></div><div><h3>Conclusions</h3><p>The molecular application domain offers great opportunities, and most of the technical and technological aspects have been solved. Wider adoption is expected once the onboarding of VR is simplified and the technology gains wider acceptance.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000171/pdf?md5=c3f874da70ddd62619d89326c3770de9&pid=1-s2.0-S2096579622000171-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132137540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Novel virtual nasal endoscopy system based on computed tomography scans","authors":"Fábio de O. Sousa ,&nbsp;Daniel S. da Silva ,&nbsp;Tarique da S. Cavalcante ,&nbsp;Edson C. Neto ,&nbsp;Victor José T. Gondim ,&nbsp;Ingrid C. Nogueira ,&nbsp;Auzuir Ripardo de Alexandria ,&nbsp;Victor Hugo C. de Albuquerque","doi":"10.1016/j.vrih.2021.09.005","DOIUrl":"10.1016/j.vrih.2021.09.005","url":null,"abstract":"<div><h3>Background</h3><p>Currently, many simulator systems for medical procedures are under development. These systems can provide new solutions for training, planning, and testing medical practices, improve performance, and optimize the time of the exams. However, to achieve the best results, certain premises must be followed and applied to the model under development, such as usability, control, graphics realism, and interactive and dynamic gamification.</p></div><div><h3>Methods</h3><p>This study presents a system for simulating a medical examination procedure in the nasal cavity for training and research purposes, using a patient′s accurate computed tomography (CT) as a reference. The pathologies that are used as a guide for the development of the system are highlighted. Furthermore, an overview of current studies covering bench medical mannequins, 3D printing, animals, hardware, software, and software that use hardware to boost user interaction, is given. Finally, a comparison with similar state-of-the-art studies is made.</p></div><div><h3>Results</h3><p>The main result of this work is interactive gamification techniques to propose an experience of simulation of an immersive exam by identifying pathologies present in the nasal cavity such as hypertrophy of turbinates, septal deviation adenoid hypertrophy, nasal polyposis, and tumor.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000158/pdf?md5=56e0a645d11847e1aa7d2a1aa890e681&pid=1-s2.0-S2096579622000158-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133998648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Virtual-reality and intelligent hardware in digital twins","authors":"Zhihan Lv ,&nbsp;Gustavo Marfia ,&nbsp;Fabio Poiesi ,&nbsp;Neil Vaughan ,&nbsp;Jun Shen","doi":"10.1016/j.vrih.2022.08.002","DOIUrl":"10.1016/j.vrih.2022.08.002","url":null,"abstract":"","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000705/pdfft?md5=db4abbeed2bd0f584132707933abf5c0&pid=1-s2.0-S2096579622000705-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131609670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Balanced-partitioning treemapping method for digital hierarchical dataset","authors":"Cong Feng ,&nbsp;Minglun Gong ,&nbsp;Oliver Deussen","doi":"10.1016/j.vrih.2021.09.006","DOIUrl":"10.1016/j.vrih.2021.09.006","url":null,"abstract":"<div><h3>Background</h3><p>The problem of visualizing a hierarchical dataset is an important and useful technique in many real-life situations. Folder systems, stock markets, and other hierarchical-related datasets can use this technique to better understand the structure and dynamic variation of the dataset. Traditional space-filling(square)-based methods have the advantages of compact space usage and node size as opposed to diagram-based methods. Spacefilling-based methods have two main research directions: static and dynamic performance.</p></div><div><h3>Methods</h3><p>This study presented a treemapping method based on balanced partitioning that enables excellent aspect ratios in one variant, good temporal coherence for dynamic data in another, and in the third, a satisfactory compromise between these two aspects. To layout a treemap, all the children of a node were divided into two groups, which were then further divided until groups of single elements were reached. After this, these groups were combined to form a rectangle representing the parent node. This process was performed for each layer of the hierarchical dataset. For the first variant from the partitioning, the child elements were sorted and two groups, sized as equally as possible, were built from both big and small elements (size-balanced partition). This achieved satisfactory aspect ratios for the rectangles but less so temporal coherence (dynamic). For the second variant, the sequence of children was taken and from this, groups, sized as equally as possible, were created without the need for sorting (sequence-based, good compromise between aspect ratio and temporal coherency). For the third variant, the children were split into two groups of equal cardinalities, regardless of their size (number-balanced, worse aspect ratios but good temporal coherence).</p></div><div><h3>Results</h3><p>This study evaluated the aspect ratios and dynamic stability of the employed methods and proposed a new metric that measures the visual difference between rectangles during their movement to represent temporally changing inputs.</p></div><div><h3>Conclusion</h3><p>This study demonstrated that the proposed method of treemapping via balanced partitioning outperformed the state-of-the-art methods for several real-world datasets.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S209657962200016X/pdf?md5=3522acfc9eaaf5a0fffab76c1bc1bfad&pid=1-s2.0-S209657962200016X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123073673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Measuring 3D face deformations from RGB images of expression rehabilitation exercises","authors":"Claudio Ferrari ,&nbsp;Stefano Berretti ,&nbsp;Pietro Pala ,&nbsp;Alberto Del Bimbo","doi":"10.1016/j.vrih.2022.05.004","DOIUrl":"10.1016/j.vrih.2022.05.004","url":null,"abstract":"<div><h3>Background</h3><p>The accurate (quantitative) analysis of 3D face deformation is a problem of increasing interest in many applications. In particular, defining a 3D model of the face deformation into a 2D target image to capture local and asymmetric deformations remains a challenge in existing literature. A measure of such local deformations may be a relevant index for monitoring the rehabilitation exercises of patients suffering from Parkinson’s or Alzheimer’s disease or those recovering from a stroke.</p></div><div><h3>Methods</h3><p>In this paper, a complete framework that allows the construction of a 3D morphable shape model (3DMM) of the face is presented for fitting to a target RGB image. The model has the specific characteristic of being based on localized components of deformation. The fitting transformation is performed from 3D to 2D and guided by the correspondence between landmarks detected in the target image and those manually annotated on the average 3DMM. The fitting also has the distinction of being performed in two steps to disentangle face deformations related to the identity of the target subject from those induced by facial actions.</p></div><div><h3>Results</h3><p>The method was experimentally validated using the MICC-3D dataset, which includes 11 subjects. Each subject was imaged in one neutral pose and while performing 18 facial actions that deform the face in localized and asymmetric ways. For each acquisition, 3DMM was fit to an RGB frame whereby, from the apex facial action and the neutral frame, the extent of the deformation was computed. The results indicate that the proposed approach can accurately capture face deformation, even localized and asymmetric deformations.</p></div><div><h3>Conclusion</h3><p>The proposed framework demonstrated that it is possible to measure deformations of a reconstructed 3D face model to monitor facial actions performed in response to a set of targets. Interestingly, these results were obtained using only RGB targets, without the need for 3D scans captured with costly devices. This paves the way for the use of the proposed tool in remote medical rehabilitation monitoring.</p></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000456/pdf?md5=10f3974adc62709cdc0d135e68fc356c&pid=1-s2.0-S2096579622000456-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130890604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advances in wireless sensor networks under AI-5G for augmented reality","authors":"Muhammad Khan","doi":"10.1016/j.vrih.2022.06.003","DOIUrl":"10.1016/j.vrih.2022.06.003","url":null,"abstract":"","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2096579622000493/pdfft?md5=4539102c7e8705a74eb1beacbee6409b&pid=1-s2.0-S2096579622000493-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128356220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}