Children of a parent. It is a Scripture term: “Thy wife shall be as a fruitful vine ... thy children like olive plants round about thy table” (Psalm cxxviii. 3.).
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Olive Branches from Infoplease: