the character made thus, “&” = and. In the old Hornbooks, after giving the twenty-six letters, the character & was added, and was called “Ampersand,” a corruption of “and per-se &” (and by itself, and). A B C D. ... X Y Z &.
Any odd shape folks understand To mean my Protean amperzand.
Punch (17 April, 1869, p. 153, col. 2).
The martyr Bradford, says Lord Russell, was “A per se A” with them, “to their comfort,” etc.—i.e. stood alone in their defence.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Ampersand from Infoplease: