High Dimensional Statistical Modelling with Limited Information