The eve of May Day, when the old pagan witch-world was supposed to hold high revelry under its chief on certain high places. The Brocken of Germany was a favourite spot for these revelries.
Walpurgis was a female saint concerned in the introduction of Christianity into Germany. She died February 25th, 779.
“He changed hands, and whisked and rioted like a dance of Walpurgis in his lonely brain.” -
J. S. Le Fanu: The House in the Churchyard, p. 109.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Walpurgis Night from Infoplease: