As anyone who lives in the north-central U.S. can tell you, clear days in winter provide a mixed blessing. On the one hand, you enjoy a respite from the long march of dim, dreary days, when you find yourself wondering if that Norse myth about a wolf eating the sun might have something to it after all. On the other hand, clear days -- and clear nights -- often mean cold weather, brought to you by a dry, continental high-pressure zone. These systems tend to have words like "cold," "frigid" or "arctic" prepended to them because, as far as Earth's surface and lower atmosphere are concerned, a cloudless night is like a cold night without blankets.
During the day, the sun's shortwave radiation is absorbed by Earth and converted into heat. When the sun sets, the planet begins radiating this heat at various rates depending upon the materials involved. Lacking clouds to capture that heat and hold it in, the surface and atmosphere grow increasingly colder through radiative heat loss.
So there you have it. "Cold is the night when the stars shine bright," and frost warnings often coincide with clear nights.