A random matrix model with a σ-model like constraint, the restricted trace ensemble (RTE), is solved in the large-n limit. In the macroscopic limit the smooth connected two-point resolvent G(z,w) is found to be non-universal, extending previous results from monomial to arbitrary polynomial potentials. Using loop equation techniques we give a closed though non-universal expression for G(z,w), which extends recursively to all higher k-point resolvents. These findings are in contrast to the usual unconstrained one-matrix model.