{"title":"Characterizing user behavior for speech and sketch-based video retrieval interfaces","authors":"Ozan Can Altiok, T. M. Sezgin","doi":"10.1145/3092907.3122801","DOIUrl":"https://doi.org/10.1145/3092907.3122801","url":null,"abstract":"From a user interaction perspective, speech and sketching make a good couple for describing motion. Speech allows easy specification of content, events and relationships, while sketching brings in spatial expressiveness. Yet, we have insufficient knowledge of how sketching and speech can be used for motion-based video retrieval, because there are no existing retrieval systems that support such interaction. In this paper, we describe a Wizard-of-Oz protocol and a set of tools that we have developed to engage users in a sketch- and speech-based video retrieval task. We report how the tools and the protocol fit together using \"retrieval of soccer videos\" as a use case scenario. Our software is highly customizable, and our protocol is easy to follow. We believe that together they will serve as a convenient and powerful duo for studying a wide range of multi-modal use cases.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114551612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tiziano Portenier, Qiyang Hu, P. Favaro, Matthias Zwicker
{"title":"SmartSketcher: sketch-based image retrieval with dynamic semantic re-ranking","authors":"Tiziano Portenier, Qiyang Hu, P. Favaro, Matthias Zwicker","doi":"10.1145/3092907.3092910","DOIUrl":"https://doi.org/10.1145/3092907.3092910","url":null,"abstract":"We present a sketch-based image retrieval system, designed to answer arbitrary queries that may go beyond searching for predefined object or scene categories. While sketching is fast and intuitive to formulate visual queries, pure sketch-based image retrieval often returns many outliers because it lacks a semantic understanding of the query. Our key idea is to combine sketch-based queries with inter-active, semantic re-ranking of query results. We leverage progress in deep learning and use a feature representation learned for image classification for re-ranking. This allows us to cluster semantically similar images, re-rank based on the clusters, and present more meaningful query results to the user. We report on two large-scale benchmarks and demonstrate that our re-ranking approach leads to significant improvements over the state of the art. Finally, a user study designed to evaluate a practical use case confirms the benefits of our approach.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129034735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A taxonomy of motion applications in data visualization","authors":"Irene de la Torre-Arenas, P. Cruz","doi":"10.1145/3092907.3122798","DOIUrl":"https://doi.org/10.1145/3092907.3122798","url":null,"abstract":"We propose a new taxonomy that explains the roles of motion in data visualization, focusing especially on their communicative aspects. Our taxonomy clarifies the main axis in how visualization designers can employ motion in data portrayal.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122519096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Modeling go: a mobile sketch-based modeling system for extracting objects","authors":"Chun-An Lai, P. Chiang","doi":"10.1145/3092907.3122797","DOIUrl":"https://doi.org/10.1145/3092907.3122797","url":null,"abstract":"This article presents an easy to use mobile application which allows users to create 3D digital copies of their interested objects anywhere and anytime. An advanced 3-sweep modeling technique is developed to construct 3D primitives not only from generalized cylinder and cuboid, but also objects with symmetrical or non-uniformly scaled profiles. In addition, our system supports the texture and structure refinement which combine results created from multiple source images. The constructed 3D model will be the combination of our 3D primitives. The combined result can preserve more features which may not be seen from a single photo.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122004243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Interactive swept surface modeling in virtual reality with motion-tracked controllers","authors":"T. McGraw, Esteban García, Drew Sumner","doi":"10.1145/3092907.3092908","DOIUrl":"https://doi.org/10.1145/3092907.3092908","url":null,"abstract":"In this work we describe an interactive technique which enables gestural curve and surface design in an immersive virtual environment. We use a pair of motion tracked controllers to allow the user to intuitively control a Hermite spline curve which can be swept through space to create surfaces, or to define the location and orientation of cloned meshes. A head mounted display and tracked controllers replace the traditional keyboard and mouse for view selection and object interaction. Natural and expressive body motions allow the user to specify the parameters which define the curves and surfaces. Results are demonstrated using the HTC Vive VR system.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127448665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Shading with painterly filtered layers: a technique to obtain painterly portrait animations","authors":"S. Castaneda, E. Akleman","doi":"10.1145/3092907.3122800","DOIUrl":"https://doi.org/10.1145/3092907.3122800","url":null,"abstract":"In this manuscript, we describe a process that can be used to create still and/or animated portrait paintings to be shown in Expressive Art Exhibit. Our process consists of two stages: (1) Creation of control textures for a Barycentric shader by using color information gathered from photographs to provide realistic looking skin rendering; (2) Filtering and compositing the layers of images that are obtained by control textures, which correspond to effects such as diffuse, specular and ambient. To demonstrate proof-of-concept, we have created a few rigid body animations of painterly portraits under different lighting conditions.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"100 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134400137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. D. Parakkat, S. Joshi, Uday Bondi Pundarikaksha, Ramanathan Muthuganapathy
{"title":"Sketch and shade: an interactive assistant for sketching and shading","authors":"A. D. Parakkat, S. Joshi, Uday Bondi Pundarikaksha, Ramanathan Muthuganapathy","doi":"10.1145/3092907.3122799","DOIUrl":"https://doi.org/10.1145/3092907.3122799","url":null,"abstract":"We present a drawing assistant for sketching and for assisting users in shading a hand drawn sketch. The augmented reality based system uses a sketch made by a professional and uses it to help inexperienced users to do sketching and shading. The input image is converted to a set of points based on simple heuristics for providing a \"connect the dots\" interface for a user to aid sketching. With the help of a 2.5D mesh generated by our algorithm, the system assists the user by providing information about the colors that can be given in different parts of the sketch. The system was tested with users of different age groups and skill levels, indicating its usefulness.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132104584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Swarna Keshavabhotla, Blake Williford, S. Kumar, Ethan C. Hilton, Paul Taele, Wayne Li, J. Linsey, T. Hammond
{"title":"Conquering the cube: learning to sketch primitives in perspective with an intelligent tutoring system","authors":"Swarna Keshavabhotla, Blake Williford, S. Kumar, Ethan C. Hilton, Paul Taele, Wayne Li, J. Linsey, T. Hammond","doi":"10.1145/3092907.3092911","DOIUrl":"https://doi.org/10.1145/3092907.3092911","url":null,"abstract":"Design sketching is a powerful tool for expressing ideas from pen and paper effectively and becoming a more well-rounded communicator. Sketching instructors conventionally employ pen and paper in their classrooms to convey these fundamentals to students. However this traditional approach limits the bandwidth and capability of instructors to give timely and individualized feedback. An intelligent tutoring system can leverage the knowledge of domain expert design sketching instructors so that students can practice and receive real-time feedback outside of classroom hours. Our system leverages consulted instructor insights and observed pedagogical practices of an active university design sketching curriculum, and applies them in a mastery-based progression of exercises that utilize sketch recognition to give real-time feedback. An evaluation of our system's usability in a class of engineering students studying design sketching showed that it performed very well, was seen by the students as a motivating and intuitive practice tool, and allowed the students to improve the accuracy and speed of their sketches.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"282 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134437349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Flow2Code: from hand-drawn flowcharts to code execution","authors":"Jorge Iván Herrera Cámara, T. Hammond","doi":"10.1145/3092907.3092909","DOIUrl":"https://doi.org/10.1145/3092907.3092909","url":null,"abstract":"Flowcharts play an important role when learning to program by conveying algorithms graphically and making them easy to read and understand. Computer-based flowchart design requires the user to learn the software first, which often results in a steep learning curve. Paper-drawn flowcharts don't provide feedback. We propose a system that allows users to draw their flowcharts directly on paper combined with a mobile phone app that takes a photo of the flowchart, interprets it, and generates and executes the resulting code. Flow2Code uses off-line sketch recognition and computer vision algorithms to recognize flowcharts drawn on paper. To gain practice and feedback with flowcharts, the user needs only a pencil, white paper, and a mobile device. The paper describes a tested system and algorithmic model for recognizing and interpreting offline flowcharts as well as a novel geometric feature, Axis Aligned Score (AAS), that enables fast accurate recognition of various quadrilaterals.","PeriodicalId":393945,"journal":{"name":"Proceedings of the Symposium on Sketch-Based Interfaces and Modeling","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129145954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}