A bit of my mind, as “Ill tell him a bit of my mind,” I'll reprove him. Same word as bite, meaning a piece bitten off, hence a piece generally. (Anglo-Saxon, bitan, to bite.)
Bit by bit. A little at a time; piece-meal.
Not a bit, or Not the least bit. Not at all; not the least likely. This may be not a morsel, or not a doit, rap, or sou. “Bit” used to be a small Jamaica coin. We still talk of a threepenny-bit. Bit, of course, is the substantive of bite, as morsel (French morceau) of mordre.
Bit (of a horse ). To take the bit in (or between) his teeth. To be obstinately self-willed; to make up one's mind not to yield. When a horse has a mind to run away, he catches the bit “between his teeth,” and the driver has no longer control over him.
“Mr. X. will not yield. He has taken the bit between his teeth, and is resolved to carry out his original measure.” —Newspaper paragraph, April, 1886.
Money. The word is used in the West Indies for a half pistareen (fivepence). In Jamaica, a bit is worth sixpence, English; in America, 12 1/2 cents; in Ireland, tenpence.
The word is still thieves' slang for money generally, and coiners are called bit-makers. In English we use the word for a coin which is a fraction of a unit. Thus, a shilling being a unit, we have a six-penny bit and threepenny bit (or not in bits but in divers pieces). So, taking a sovereign for a unit, we had seven-shilling bits, etc.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894