Properly, one who makes up and supplies decorations for horses. A horse-soldier more fit for the toilet than the battle-field. The expression was first used by Rowley in his Ballads of Charitie, but Sir Walter Scott revived it.
One comes in foreign trashery Of tinkling chain and spur, A walking haberdashery Of feathers, lace, and fur; In Rowley's antiquated phrase. Horse milliner of modern days.
Bridal of Triermain, ii. 3.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Horse-milliner from Infoplease: