It was formerly a common belief that the sun danced on Easter Day. Sir Thomas Browne combats the notion in his Vulgar Errors.
But oh, she dances such a way, No sun upon an Easter day Is half so fine a sight.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Easter-day Sun from Infoplease: