Fast, Deterministic and Sparse Dimensionality Reduction

Abstract

We provide a deterministic construction of the sparse Johnson-Lindenstrauss transform of Kane & Nelson (J.ACM 2014) which runs, under a mild restriction, in the time necessary to apply the sparse embedding matrix to the input vectors. Specifically, given a set of n vectors in ℝd and target error ε, we give a deterministic algorithm to compute a {–1,0,1} embedding matrix of rank O((ln n)/ε2) with O((ln n)/ε) entries per column which preserves the norms of the vectors to within 1 ± ε. If NNZ, the number of non-zero entries in the input set of vectors, is Ω(d2), our algorithm runs intime O(NNZ · ln n/ε).One ingredient in our construction is an extremely simple proof of the Hanson-Wright inequality for subgaussian random variables, which is more amenable to derandomization. As an interesting byproduct, we are able to derive the essentially optimal form of the inequality in terms of its functional dependence on the parameters.