Semiparametric Regression Models For Between- And Within-Subject Attributes: Applications To High-Dimensional Data, Asymptotic Efficiency And Beyond