Loungers who frequented the middle of St. Paul's, which was the Bond Street of London up to the time of the Commonwealth. (See Ben Jonson's Every Man out of his Humour, where are a variety of scenes given in the interior of St. Paul's. Harrison Ainsworth describes these “walkers” in his novel entitled Old St. Paul's.)
“The young gallants ... used to meet at the central point, St. Paul's: and from this circumstance obtained the appellation of Paul's Walkers, as we now say Bond Street Lungers.” - Moser: European Magazine, July, 1807.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Paul's Walkers from Infoplease: