Correct Your Horse's Misbehavior