on churches, tombstones, and used by pilgrims:
(1) If dedicated to James the Greater, the scallop-shell is his recognised emblem. (See James.) If not, the allusion is to the vocation of the apostles generally, who were fishermen, and Christ said He would make them “fishers of men.”
(2) On tombstones, the allusion is to the earthly body left behind, which is the mere shell of the immortal soul.
(3) Carried by pilgrims, the allusion may possibly be to James the Greater, the patron saint of pilgrims, but more likely it originally arose as a convenient drinking-cup, and hence the pilgrims of Japan carry scallop shells.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894