Meaning speech or the tongue, so called by the Psalmist because speech is man's speciality. Other animals see, hear, smell, and feel quite as well and often better than man, but rational speech is man's glory, or that which distinguishes the race from other animals.
“I will sing and give praise even with my glory.”—Psalm cviii. 1.
“That my glory may sing praise to Thee, and not be silent.”—Psalm xxx. 12.
“Awake up my glory, awake psaltery and harp.”—Psalm lvii. 8.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Glory from Infoplease: