Elastic Consistency

Abstract

Machine learning models can match or surpass humans on specialized tasks such as image classification [20, 14], speech recognition [37], or complex games [39]. One key tool behind this progress has been a family of optimization methods which fall under the umbrella term of stochastic gradient descent (SGD) [35], which are by and large the method of choice for training large-scale machine learning models.