Keynote: Affordable and Practical Home Context Recognition with “Image as a Document” Approach

Masahide Nakamura
{"title":"Keynote: Affordable and Practical Home Context Recognition with “Image as a Document” Approach","authors":"Masahide Nakamura","doi":"10.1109/percomworkshops48775.2020.9156241","DOIUrl":null,"url":null,"abstract":"Technologies for home context recognition have been studied for many years in the field of ubiquitous computing. The traditional ubiquitous computing employs ambient sensors (e.g., temperature, humidity, presence), wearable sensors (e.g., accelerometer, heart rate), and indoor positioning systems. In more recent years, the emerging deep learning allows the system to recognize multimedia data. Since image, voice, and text data usually contain richer information than the conventional sensor data, it is promising to use such multimedia data for recognizing the home contexts. Unfortunately, however, these existing technologies are yet far from practical use in general households, since they require expensive resources at home. It is difficult for ordinary users to operate and maintain proprietary systems at home on a daily basis. One may try to recognize home contexts via image recognition based on deep learning. However, constructing a custom recognition model dedicated for a single house requires a huge amount of labeled datasets and computing resources. Thus, there is still a big gap between research and real life. To achieve more affordable and practical home context recognition, we present a novel technique that integrates image-based cognitive API and light-weight machine learning. The cognitive API receives an image from an external application, recognizes specific information within the image, and returns the information as a set of text words called tags. Our key idea is to use these tags as features of the image, and apply light-weight machine learning techniques to infer the target context. Since every image can be considered as a document, the expensive deep learning is no more needed. We call this idea the image-as-a-document approach. To demonstrate the practical feasibility, we have conducted an experiment that recognizes contexts within our laboratory. The experimental results showed that the overall accuracy of the recognition model was 0.929. We also present further approach to improve the accuracy, by exploiting multiple cognitive APIs.","PeriodicalId":250033,"journal":{"name":"2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)","volume":"59 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/percomworkshops48775.2020.9156241","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Technologies for home context recognition have been studied for many years in the field of ubiquitous computing. The traditional ubiquitous computing employs ambient sensors (e.g., temperature, humidity, presence), wearable sensors (e.g., accelerometer, heart rate), and indoor positioning systems. In more recent years, the emerging deep learning allows the system to recognize multimedia data. Since image, voice, and text data usually contain richer information than the conventional sensor data, it is promising to use such multimedia data for recognizing the home contexts. Unfortunately, however, these existing technologies are yet far from practical use in general households, since they require expensive resources at home. It is difficult for ordinary users to operate and maintain proprietary systems at home on a daily basis. One may try to recognize home contexts via image recognition based on deep learning. However, constructing a custom recognition model dedicated for a single house requires a huge amount of labeled datasets and computing resources. Thus, there is still a big gap between research and real life. To achieve more affordable and practical home context recognition, we present a novel technique that integrates image-based cognitive API and light-weight machine learning. The cognitive API receives an image from an external application, recognizes specific information within the image, and returns the information as a set of text words called tags. Our key idea is to use these tags as features of the image, and apply light-weight machine learning techniques to infer the target context. Since every image can be considered as a document, the expensive deep learning is no more needed. We call this idea the image-as-a-document approach. To demonstrate the practical feasibility, we have conducted an experiment that recognizes contexts within our laboratory. The experimental results showed that the overall accuracy of the recognition model was 0.929. We also present further approach to improve the accuracy, by exploiting multiple cognitive APIs.
主题演讲:基于“图像即文件”方法的经济实用的家居环境识别
家庭环境识别技术在普适计算领域已经研究多年。传统的普适计算采用环境传感器(如温度、湿度、存在感)、可穿戴传感器(如加速度计、心率)和室内定位系统。近年来,新兴的深度学习使系统能够识别多媒体数据。由于图像、语音和文本数据通常比传统的传感器数据包含更丰富的信息,因此使用这些多媒体数据来识别家庭环境是有希望的。然而,不幸的是,这些现有的技术还远远不能在普通家庭中实际使用,因为它们需要昂贵的家庭资源。普通用户在家中日常操作和维护专有系统是很困难的。人们可以尝试通过基于深度学习的图像识别来识别家庭环境。然而,构建一个专门针对单个房屋的自定义识别模型需要大量的标记数据集和计算资源。因此,研究与现实生活之间还有很大的差距。为了实现更实惠和实用的家庭背景识别,我们提出了一种集成基于图像的认知API和轻量级机器学习的新技术。认知API从外部应用程序接收图像,识别图像中的特定信息,并将信息作为一组称为标签的文本单词返回。我们的关键思想是使用这些标签作为图像的特征,并应用轻量级机器学习技术来推断目标上下文。由于每张图像都可以被视为文档,因此不再需要昂贵的深度学习。我们称这种想法为“图像即文档”方法。为了证明实际的可行性,我们在实验室中进行了一个识别上下文的实验。实验结果表明,该识别模型的总体准确率为0.929。我们还提出了通过利用多个认知api来提高准确性的进一步方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书