We study the structure of the Hilbert space of gauged matrix models with a global symmetry. In the first part of the paper, we focus on bosonic matrix models with
U(2) gauge group and
SO(d) global symmetry, and consider singlets under both the gauge and global symmetry. We show how such "double-gauged'' matrix models can be described in terms of a simpler
SO(3) single-matrix model. In the second part of the paper, we consider the so-called BMN subsector of the
N=4 SU(N) super Yang-Mills theory, which is closely related to the BMN matrix model. Among the 1/16 BPS operators in this sector, "non-graviton'' operators were recently discovered, which are expected to relate to the microstates of supersymmetric
AdS5 black holes. We show that a double gauging of this model, where one projects onto
SU(3)R R-symmetry singlets, considerably simplifies the analysis of the non-graviton spectrum. In particular, for low values of
N, we show that (almost) all graviton operators project out of the spectrum, while important classes of non-graviton operators remain. In the
N=3 case, we obtain a closed form expression for the superconformal index of singlet non-gravitons, which reveals structural features of their spectrum.