Oliver Wendell Holmes, in the Autocrat of the Breakfast Table (chap. viii.), speaks of Kirke White as one of the “sweet Albino poets,” whose “plaintive song” he admires. It implies some deficiency of virility, as albinism suggests weakness, and possibly is meant as a play upon the name in this particular instance.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Albino-poets from Infoplease: