Exceptions prove the Rule
They prove there is a rule, or there could be no exceptions; the very fact of exceptions proves there must be a rule.
“Exceptio probat regulam.” —Columella.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Exceptions prove the Rule from Infoplease: