"Covering one's head expresses one's awe and reverence of God"

"A Jewish person covers his head not only in a holy place (such as a synagogue) or when engaged in holy matters (such as learning Torah or performing a mitzvah) but wherever he may be and all the time--the reason being that the Divine Presence, the Shekhinah exists everywhere and always rests on his head.

Covering oneself (and in particular one's head) expresses one's awe and reverence of God and His glory.

For that reason, the sages wrapped themselves in their robes when they sat to administer justice or speak words of Torah.

Similarly, in most congregations, the men cover their heads with their prayer shawls when reciting the Shmoneh Esrei.

Wrapping oneself in one's garment is a sign of accepting authority.

Contrarily, an uncovered head is an expression of throwing off a yoke, of liberty.

The Aramaic phrase reish gali, 'an uncovered head,' expresses that point exactly.

Thus, the Aramaic translation of the verse 'The children of Israel went out with a high hand' (Exodus 14:8) is 'The children of Israel went forth with an uncovered head.'

A person removes articles of clothing when he feels at ease, and he covers himself when he believes that he is being judged from above.

Therefore, covering one's head is an expression of the sense that the Shekhinah is to be found everywhere and that one is always subject to the immediacy of the omnipresent Divinity."

--Rabbi Adin Steinsaltz

From Understanding the Tanya, p.191, by Rabbi Adin Steinsaltz