properly means “belonging to a village” (Latin, pagus).
The Christian Church fixed itself first in cities, the centres of
intelligence. Long after it had been established in towns, idolatrous
practices continued to be observed in rural districts and villages, so
pagan and villager came to mean the same thing. (See Heathen.)
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894