2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD)最新文献

筛选

英文中文

Dynamic Transformer for Efficient Machine Translation on Embedded Devices 嵌入式设备上高效机器翻译的动态变压器

2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD) Pub Date : 2021-07-17 DOI: 10.1109/MLCAD52597.2021.9531281

Hishan Parry, Lei Xun, Amin Sabet, Jia Bi, Jonathon S. Hare, G. Merrett

{"title":"Dynamic Transformer for Efficient Machine Translation on Embedded Devices","authors":"Hishan Parry, Lei Xun, Amin Sabet, Jia Bi, Jonathon S. Hare, G. Merrett","doi":"10.1109/MLCAD52597.2021.9531281","DOIUrl":"https://doi.org/10.1109/MLCAD52597.2021.9531281","url":null,"abstract":"The Transformer architecture is widely used for machine translation tasks. However, its resource-intensive nature makes it challenging to implement on constrained embedded devices, particularly where available hardware resources can vary at run-time. We propose a dynamic machine translation model that scales the Transformer architecture based on the available resources at any particular time. The proposed approach, ‘Dynamic-HAT’, uses a HAT SuperTransformer as the backbone to search for SubTransformers with different accuracy-latency trade-offs at design time. The optimal SubTransformers are sampled from the SuperTransformer at run-time, depending on latency constraints. The Dynamic-HAT is tested on the Jetson Nano and the approach uses inherited SubTransformers sampled directly from the SuperTransformer with a switching time of <1s. Using inherited SubTransformers results in a BLEU score loss of 6 1.5% because the SubTransformer configuration is not retrained from scratch after sampling. However, to recover this loss in performance, the dimensions of the design space can be reduced to tailor it to a family of target hardware. The new reduced design space results in a BLEU score increase of approximately 1% for sub-optimal models from the original design space, with a wide range for performance scaling between 0.356s - 1.526s for the GPU and 2.9s - 7.31s for the CPU.","PeriodicalId":210763,"journal":{"name":"2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120996586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

首页上一页