(To). To cheat in a small way, as “I diddled him out of ...” Edgar Allan Poe has an article on the art of “Diddling.” Rhyming slang is very common. (See Chivy.) Fiddle and diddle rhyme. “Fiddle” is slang for a sharper, and “diddle” is the act of a sharper. The suggestive rhyme was
Hi diddle diddle! The cat and the fiddle. A certain portion of the human race Has certainly a taste for being diddled.
Hood: A Black Job, stanza 1.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894