A corruption of Milaner; so called from Milan, in Italy, which at one time gave the law to Europe in all matters of taste, dress, and elegance.
Milliner was originally applied to the male sex; hence Ben Jonson, in Every Man in his Humour, i. 3, speaks of a “milliner's wife.” The French have still une modiste and un modiste.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Milliner from Infoplease: