Leverage Score Ordering

Abstract: In this paper we present Leverage Score Ordering, a novel technique for determining the ordering of data in the training of deep neural networks. Our technique is based on the distributed computation of leverage scores using random projections. These computed leverage scores provide a flexible and efficient method to determine the optimal ordering of training data without manual intervention or annotations. To demonstrate the strength of our method, we present empirical results on an extensive set of experiments across image recognition tasks, language based sentiment analysis and multimodal sentiment analysis. Our method is faster compared to standard randomized projection algorithms and shows promising improvements in convergence and results.