Exploring and constraining new physics in the dark sector