Learning Deep Models with Linguistically-Inspired Structure