{"title":"Automatic skeleton generation using hierarchical mesh segmentation","authors":"Ryan Anthony J. de Belen, Rowel Atienza","doi":"10.1145/2992138.2992150","DOIUrl":"https://doi.org/10.1145/2992138.2992150","url":null,"abstract":"High fidelity and interactive 3D characters abound in virtual and augmented reality. However, making one usually requires manual skeleton extraction and specification on how the body reacts with the translation and rotation of the bones. While there are previous works that already solve this problem, most systems have limitations with the input or the output cannot be directly used for animation. With the goal of bringing any humanoid toys to life, a pipeline is presented to automatically generate the toy's skeletal structure and skinning weights. First, the mesh is segmented into several parts using normal characteristic value (NCV) and global point signatures (GPS) as candidate points for segmentation. Then, joint locations are generated based on the segmentation results. Further-more, the skinning weights are generated by solving the Laplace diffusion equation. Experimental results show that our pipeline is robust enough to extract skeletal structures from graphic artists' models as well as from scanned models. In addition, our pipeline is deformation-invariant as it can generate the same skeletal structure of a model having different poses. Finally, our pipeline achieves both appealing virtual realism and fast speed. The output can be directly used to setup skeleton-based animations in games as well as real-time virtual and augmented reality applications within minutes.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82017107","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sebouh-Steve Bardakjian, Serguei A. Mokhov, Miao Song, S. Mudur
{"title":"ISSv3: from human motion in the real to the interactive documentary film in AR/VR","authors":"Sebouh-Steve Bardakjian, Serguei A. Mokhov, Miao Song, S. Mudur","doi":"10.1145/2992138.2992139","DOIUrl":"https://doi.org/10.1145/2992138.2992139","url":null,"abstract":"This is a small research-creation interactive augmented and virtual reality (AR/VR) documentary prototype. Its design extends the predecessor AR mobile app component incorporating AR footage previously only available in traditional mobile screen into a VR world. The mobile app prototype runs on iOS and Android and integrates interactive 3D models and objects in the VR world, the previous app did not have in an effort to increase engagement and educational aspects appealing to a wider audience. The focus of this project is to enhance user engagement and create immersion for interactive documentaries by combining Augmented Reality and Virtual Reality in order to create virtual worlds where users can enjoy watching, hearing, and reading content. We develop Illimitable Space System v3 as inspired by v1 in early days that used both motion capture and initial lower end augmented reality setup.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81685141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Aerial interaction with femtosecond laser induced plasma","authors":"Yoichi Ochiai","doi":"10.1145/2992138.2992153","DOIUrl":"https://doi.org/10.1145/2992138.2992153","url":null,"abstract":"In this article we report case studies on the aerial interaction with laser induced plasma as computational voxels. Laser induced plasma of air can be utilized to generate light source or tactile components for aerial interactions. In current studies we transformed ordinary air into interactive visual media [Ochiai et al. 2016b] and haptic media [Ochiai et al. 2016a]. To implement these interactive media, we employed Computer Generated Hologram and rendered with laser induced plasma generated by femtosecond laser sources with Spatial Light Modulators and ultrasound generated by acoustic phased array. In article, we introduce the concept through these studies and results of our research group in aerial touch display project called Fairy Lights in Femtoseconds [Ochiai et al. 2016b].","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80763321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"LumiSpace: a VR architectural daylighting design system","authors":"Yujie Hong, P. Michalatos","doi":"10.1145/2992138.2992140","DOIUrl":"https://doi.org/10.1145/2992138.2992140","url":null,"abstract":"We built a VR daylighting design system for architects to design an indoor daylighting experience. With this system, users can import 3d models or meshes, and interact with the architecture in space. In particular users can draw the lighting areas they want and receive system-generated window opennings, given the sunlight conditions of a specific time and location. This research proposes a reversed design paradigm empowered by virtual reality technology, which puts the design of spatial experience first rather than the design of physical objects. That is, window design is determined as a result of daylighting experience.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87825321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chen Su, Yifan Peng, Qing Zhong, Haifeng Li, Rui Wang, W. Heidrich, X. Liu
{"title":"Towards VR and AR enhancement: light field display with mid-air interaction","authors":"Chen Su, Yifan Peng, Qing Zhong, Haifeng Li, Rui Wang, W. Heidrich, X. Liu","doi":"10.1145/2992138.2992151","DOIUrl":"https://doi.org/10.1145/2992138.2992151","url":null,"abstract":"We present a 360° light field display to visualize 3D objects above the table-top screen, and provide a graspable and measurable virtual or augmented reality environment. The display is constructed by a spinning flat-plate deflected diffuser screen, a high-speed light field projector, and the user tracking and interaction components. A rear projection light propagation is specially designed to reconstruct the light field. By calibrating this levitated light field to the physical dimension, and capturing user gestures, we enable multiple users mid-air interactions based on their virtual measurement perception.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74712254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Joint labelling and segmentation for 3D scanned human body","authors":"Hanqing Wang, Changyang Li, Zikai Gao, Wei Liang","doi":"10.1145/2992138.2992149","DOIUrl":"https://doi.org/10.1145/2992138.2992149","url":null,"abstract":"In this paper, we present an approach to perform 3D human body labelling and segmentation jointly. Given a 3D mesh of scanned human body with texture, our approach segments it into 5 parts: head, torso, arms, legs and feet automatically. We assume that the faces on the same part of human body share similar color features and are constrained by geometry. According to this assumption, we formulate the labelling and segmentation of 3D Mesh as an energy function optimization problem. In this energy function, a data term models the color information and a smooth term models the geometry constraint. Then a GraphCut algorithm is applied to solve the optimization problem. The experiment results show good performance of our method.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89331899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Seungjae Lee, C. Jang, Seokil Moon, Byounghyo Lee, Jaebum Cho, Byoun-gkil Lee
{"title":"See-through light field displays for augmented reality","authors":"Seungjae Lee, C. Jang, Seokil Moon, Byounghyo Lee, Jaebum Cho, Byoun-gkil Lee","doi":"10.1145/2992138.2992142","DOIUrl":"https://doi.org/10.1145/2992138.2992142","url":null,"abstract":"Augmented reality (AR) is a fascinating technology that receives considerable attention of public and industries. Here, we introduce an optical tool, holographic optical element (HOE), which can be applied for AR applications. HOEs are holographic volume gratings, which have transparency and distinctive optical characteristics. HOEs can perform the roles of optical elements by transforming an incident wavefront to a predefined wavefront via diffraction [Close 1975]. Since HOEs are usually recorded in thin film such as photopolymers, they have various advantages in weight, cost, and thickness. Thus, there have been several applications using HOEs, including see-through light field displays [Hong et al. 2014] and head-mounted displays [Kasai et al. 2001].","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77555283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A virtual reality platform for dynamic human-scene interaction","authors":"Jenny Lin, Xingwen Guo, Jingyu Shao, Chenfanfu Jiang, Yixin Zhu, Song-Chun Zhu","doi":"10.1145/2992138.2992144","DOIUrl":"https://doi.org/10.1145/2992138.2992144","url":null,"abstract":"Both synthetic static and simulated dynamic 3D scene data is highly useful in the fields of computer vision and robot task planning. Yet their virtual nature makes it difficult for real agents to interact with such data in an intuitive way. Thus currently available datasets are either static or greatly simplified in terms of interactions and dynamics. In this paper, we propose a system in which Virtual Reality and human / finger pose tracking is integrated to allow agents to interact with virtual environments in real time. Segmented object and scene data is used to construct a scene within Unreal Engine 4, a physics-based game engine. We then use the Oculus Rift headset with a Kinect sensor, Leap Motion controller and a dance pad to navigate and manipulate objects inside synthetic scenes in real time. We demonstrate how our system can be used to construct a multi-jointed agent representation as well as fine-grained finger pose. In the end, we propose how our system can be used for robot task planning and image semantic segmentation.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84476109","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SIGGRAPH ASIA 2016 Virtual Reality meets Physical Reality: Modelling and Simulating Virtual Humans and Environments","authors":"Hui Huang","doi":"10.1145/2992138","DOIUrl":"https://doi.org/10.1145/2992138","url":null,"abstract":"The SIGGRAPH Asia Workshops program aims to provide participants with the opportunity to present and discuss novel research ideas on active and emerging topics of computer graphics and interactive techniques. Our workshops are expected to foster topics or communities that do not have a \"home\" in other venues. They give the community a new, organized way of thinking about a specific topic, and ideas that suggest promising directions for future research.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87714719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating physical quantities and learning human utilities from RGBD videos","authors":"Yixin Zhu, Chenfanfu Jiang, Yibiao Zhao, Demetri Terzopoulos, Song-Chun Zhu","doi":"10.1145/2992138.2992148","DOIUrl":"https://doi.org/10.1145/2992138.2992148","url":null,"abstract":"We propose a notion of affordance that takes into account physical quantities generated when the human body interacts with real-world objects, and introduce a learning framework that incorporates the concept of human utilities, which in our opinion provides a deeper and finer-grained account not only of object affordance but also of people's interaction with objects. Rather than defining affordance in terms of the geometric compatibility between body poses and 3D objects, we devise algorithms that employ physics-based simulation to infer the relevant forces/pressures acting on body parts. By observing the choices people make in videos (particularly in selecting a chair in which to sit) our system learns the comfort intervals of the forces exerted on body parts (while sitting). We account for people's preferences in terms of human utilities, which transcend comfort intervals to account also for meaningful tasks within scenes and spatiotemporal constraints in motion planning, such as for the purposes of robot task planning.","PeriodicalId":54588,"journal":{"name":"Presence-Teleoperators and Virtual Environments","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2016-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75700134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}