Driving on public roads is not a right, it is a privilege that can be revoked by the state at any time for any reason. (within the standard non-discrimination limits, race, sex, age, etc)

Yes it is BS but there really is no way to fight it.