Pen and Feather
are varieties of the same word, the root being the Sanskrit pat, to fly. (We have the Sanskrit pattra, a wing or instrument for flying; Latin, petna or, penna, pen; Greek, pteron; Teutonic, phathra; Anglo-Saxon, fether; our “feather.”)
Analogous examples are TEAR and LARME, NAG and EQUUS, WIG and PERUKE, HEART and COEUR, etc.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Pen and Feather from Infoplease: