Stolen Things are Sweet
A sop filched from the dripping-pan, fruit procured by stealth, and game illicitly taken, have the charm of dexterity to make them the more palatable. Solomon says, “Stolen waters are sweet, and bread eaten in secret [i.e. by stealth] is pleasant.”
From busie cooks we love to steal a bit Behind their backs, and that in corners eat; Nor need we here the reason why entreat; All know the proverb, `Stolen bread is sweet.'
History of Joseph, n. d.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Stolen Things are Sweet from Infoplease: