The singular value matrix decomposition plays a ubiquitous role throughout statistics and related fields. Myriad applications including clustering, classification, and dimensionality reduction involve studying and exploiting the geometric structure of singular values and singular vectors. This paper contributes to the literature by providing a novel collection of technical and theoretical tools for studying the geometry of singular subspaces using the $2\to\infty$ norm. Motivated by preliminary deterministic Procrustes analysis, we consider a general matrix perturbation setting in which we derive a new Procrustean matrix decomposition. Together with flexible machinery developed for the $2\to\infty$ norm, this allows us to conduct a refined analysis of the induced perturbation geometry with respect to the underlying singular vectors even in the presence of singular value multiplicity. Our analysis yields perturbation bounds for a range of popular matrix noise models, each of which has a meaningful associated statistical inference task. We discuss how the $2\to\infty$ norm is arguably the preferred norm in certain statistical settings. Specific applications discussed in this paper include the problem of covariance matrix estimation, singular subspace recovery, and multiple graph inference. Both our novel Procrustean matrix decomposition and the technical machinery developed for the $2\to\infty$ norm may be of independent interest.
Paper accepted July 2018.