Love —i.e. the passion of the fantasy or imagination. A fancyman is a man (not your husband) whom you fancy or select for chaperon.
Tell me where is fancy bred, Or in the heart or in the head.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Fancy from Infoplease: