Horse races held in May, and lasting four days. They are held on Epsom Downs, and were instituted by Charles I. The second day (Wednesday) is the great Derby day, so called from Lord Derby, who instituted the stakes in 1780. The fourth day (Friday) is called the Oaks, so called from “Lambert's Oaks.” The “Oaks Estate” passed into the Derby family, and the twelfth Earl of Derby established the stakes.
The Derby, the Oaks, and the St. Leger (held at Doncaster) are called the Three Classic Races. N.B.—There are other races held at Epsom besides the great four-day races mentioned above—for instance, the City Suburban and the Great Metropolitan (both handicap races).
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Epsom Races from Infoplease: