B. Baldwin, Lauren Reese, LIming Zhang, Jan Neumann, Taylor Cassidy, Michael Pereira, G. C. Murray, Kishorekumar Sundararajan, Yidnekachew Endale, Pramod Kadagattor, Paul Wolfe, Brian Aiken, Tony Braskich, Donte Jiggetts, Adam Sloan, Esther Vaturi, Crystal Pender, Ferhan Ture
{"title":"大规模模拟人类以评估电视语音接口:康卡斯特的往返系统","authors":"B. Baldwin, Lauren Reese, LIming Zhang, Jan Neumann, Taylor Cassidy, Michael Pereira, G. C. Murray, Kishorekumar Sundararajan, Yidnekachew Endale, Pramod Kadagattor, Paul Wolfe, Brian Aiken, Tony Braskich, Donte Jiggetts, Adam Sloan, Esther Vaturi, Crystal Pender, Ferhan Ture","doi":"10.1145/3539597.3575787","DOIUrl":null,"url":null,"abstract":"Evaluating large-scale customer-facing voice interfaces involves a variety of challenges, such as data privacy, fairness or unintended bias, and the cost of human labor. Comcast's Xfinity Voice Remote is one such voice interface aimed at users looking to discover content on their TVs. The artificial intelligence (AI) behind the voice remote currently powers multiple voice interfaces, serving tens of millions of requests every day, from users across the globe.In this talk, we introduce a novel Round-Trip system we have built to evaluate the AI serving these voice interfaces in a semi-automated manner, providing a robust and cheap alternative to traditional quality assurance methods. We discuss five specific challenges we have encountered in Round-Trip and describe our solutions in detail.","PeriodicalId":227804,"journal":{"name":"Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining","volume":"15 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2023-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Simulating Humans at Scale to Evaluate Voice Interfaces for TVs: the Round-Trip System at Comcast\",\"authors\":\"B. Baldwin, Lauren Reese, LIming Zhang, Jan Neumann, Taylor Cassidy, Michael Pereira, G. C. Murray, Kishorekumar Sundararajan, Yidnekachew Endale, Pramod Kadagattor, Paul Wolfe, Brian Aiken, Tony Braskich, Donte Jiggetts, Adam Sloan, Esther Vaturi, Crystal Pender, Ferhan Ture\",\"doi\":\"10.1145/3539597.3575787\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Evaluating large-scale customer-facing voice interfaces involves a variety of challenges, such as data privacy, fairness or unintended bias, and the cost of human labor. Comcast's Xfinity Voice Remote is one such voice interface aimed at users looking to discover content on their TVs. The artificial intelligence (AI) behind the voice remote currently powers multiple voice interfaces, serving tens of millions of requests every day, from users across the globe.In this talk, we introduce a novel Round-Trip system we have built to evaluate the AI serving these voice interfaces in a semi-automated manner, providing a robust and cheap alternative to traditional quality assurance methods. We discuss five specific challenges we have encountered in Round-Trip and describe our solutions in detail.\",\"PeriodicalId\":227804,\"journal\":{\"name\":\"Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining\",\"volume\":\"15 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-02-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3539597.3575787\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3539597.3575787","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Simulating Humans at Scale to Evaluate Voice Interfaces for TVs: the Round-Trip System at Comcast
Evaluating large-scale customer-facing voice interfaces involves a variety of challenges, such as data privacy, fairness or unintended bias, and the cost of human labor. Comcast's Xfinity Voice Remote is one such voice interface aimed at users looking to discover content on their TVs. The artificial intelligence (AI) behind the voice remote currently powers multiple voice interfaces, serving tens of millions of requests every day, from users across the globe.In this talk, we introduce a novel Round-Trip system we have built to evaluate the AI serving these voice interfaces in a semi-automated manner, providing a robust and cheap alternative to traditional quality assurance methods. We discuss five specific challenges we have encountered in Round-Trip and describe our solutions in detail.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
