The placing of the east window of a church due east, that is, so that the rising sun may at noon shine on the altar. Anciently, churches were built with their axes pointing to the rising sun on the saint's day; so that a church dedicated to St. John was not parallel to one dedicated to St. Peter. The same practice prevailed both in Egypt and ancient Greece.
Modern churches are built as nearly due east and west as circumstances will allow, quite regardless of the saint's day.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Orientation from Infoplease: