Prayers were offered on the tops of high hills, and temples built on “high places,” from the notion that the gods, could better hear prayers on such places, as they were nearer heaven. As Lucian says, And Tacitus says, “maxime coelo appropinquare, precesque mortalium a Deo nusquam propius audire.” It will be remembered that Balak (Numbers xxiii. xxiv.) took Balaam to the top of Peor and other high places when Balaam wished to consult God. We often read of “idols on every high hill.” (Ezek. vi. 13.)
The Greek gods dwelt on Mount Olympus.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Hills from Infoplease: