Raphael A Meyer, Cameron Musco, Christopher Musco, David P Woodruff
{"title":"Hutch++:最优随机轨迹估计。","authors":"Raphael A Meyer, Cameron Musco, Christopher Musco, David P Woodruff","doi":"10.1137/1.9781611976496.16","DOIUrl":null,"url":null,"abstract":"<p><p>We study the problem of estimating the trace of a matrix <b><i>A</i></b> that can only be accessed through matrix-vector multiplication. We introduce a new randomized algorithm, Hutch++, which computes a (1 ± <i>ε</i>) approximation to tr( <b><i>A</i></b> ) for any positive semidefinite (PSD) <b><i>A</i></b> using just <i>O</i>(1<i>/ε</i>) matrix-vector products. This improves on the ubiquitous <i>Hutchinson's estimator</i>, which requires <i>O</i>(1<i>/ε</i> <sup>2</sup>) matrix-vector products. Our approach is based on a simple technique for reducing the variance of Hutchinson's estimator using a low-rank approximation step, and is easy to implement and analyze. Moreover, we prove that, up to a logarithmic factor, the complexity of Hutch++ is optimal amongst all matrix-vector query algorithms, even when queries can be chosen adaptively. We show that it significantly outperforms Hutchinson's method in experiments. While our theory requires <b><i>A</i></b> to be positive semidefinite, empirical gains extend to applications involving non-PSD matrices, such as triangle estimation in networks.</p>","PeriodicalId":93491,"journal":{"name":"Proceedings of the SIAM Symposium on Simplicity in Algorithms (SOSA)","volume":"2021 ","pages":"142-155"},"PeriodicalIF":0.0000,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8553228/pdf/nihms-1696966.pdf","citationCount":"73","resultStr":"{\"title\":\"Hutch++: Optimal Stochastic Trace Estimation.\",\"authors\":\"Raphael A Meyer, Cameron Musco, Christopher Musco, David P Woodruff\",\"doi\":\"10.1137/1.9781611976496.16\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>We study the problem of estimating the trace of a matrix <b><i>A</i></b> that can only be accessed through matrix-vector multiplication. We introduce a new randomized algorithm, Hutch++, which computes a (1 ± <i>ε</i>) approximation to tr( <b><i>A</i></b> ) for any positive semidefinite (PSD) <b><i>A</i></b> using just <i>O</i>(1<i>/ε</i>) matrix-vector products. This improves on the ubiquitous <i>Hutchinson's estimator</i>, which requires <i>O</i>(1<i>/ε</i> <sup>2</sup>) matrix-vector products. Our approach is based on a simple technique for reducing the variance of Hutchinson's estimator using a low-rank approximation step, and is easy to implement and analyze. Moreover, we prove that, up to a logarithmic factor, the complexity of Hutch++ is optimal amongst all matrix-vector query algorithms, even when queries can be chosen adaptively. We show that it significantly outperforms Hutchinson's method in experiments. While our theory requires <b><i>A</i></b> to be positive semidefinite, empirical gains extend to applications involving non-PSD matrices, such as triangle estimation in networks.</p>\",\"PeriodicalId\":93491,\"journal\":{\"name\":\"Proceedings of the SIAM Symposium on Simplicity in Algorithms (SOSA)\",\"volume\":\"2021 \",\"pages\":\"142-155\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8553228/pdf/nihms-1696966.pdf\",\"citationCount\":\"73\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the SIAM Symposium on Simplicity in Algorithms (SOSA)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1137/1.9781611976496.16\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the SIAM Symposium on Simplicity in Algorithms (SOSA)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1137/1.9781611976496.16","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
We study the problem of estimating the trace of a matrix A that can only be accessed through matrix-vector multiplication. We introduce a new randomized algorithm, Hutch++, which computes a (1 ± ε) approximation to tr( A ) for any positive semidefinite (PSD) A using just O(1/ε) matrix-vector products. This improves on the ubiquitous Hutchinson's estimator, which requires O(1/ε2) matrix-vector products. Our approach is based on a simple technique for reducing the variance of Hutchinson's estimator using a low-rank approximation step, and is easy to implement and analyze. Moreover, we prove that, up to a logarithmic factor, the complexity of Hutch++ is optimal amongst all matrix-vector query algorithms, even when queries can be chosen adaptively. We show that it significantly outperforms Hutchinson's method in experiments. While our theory requires A to be positive semidefinite, empirical gains extend to applications involving non-PSD matrices, such as triangle estimation in networks.