The tail of a fox or squirrel, which is brushy.
He brushed by me. He just touched me as he went quickly past. Hence also brush, a slight skirmish. All these are metaphors from brushing with a brush.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894