Scalable Coordination of Intelligent Vehicles in Shared Markovian Dynamics