The angular distribution of the sky radiance determines the energy generation of solar power technologies as well as the ultraviolet (UV) doses delivered to the biosphere. The sky-diffuse radiance distribution depends on the wavelength, the solar elevation, and the atmospheric conditions. Here, we report on ground-based measurements of the all-sky radiance at three sites in the Southern Hemisphere across a transect of about 5,000 km: Santiago (33°S, a mid-latitude city of 6 million inhabitants with endemic poor air quality), King George Island (62°S, at the northern tip of the Antarctic Peninsula, one of the cloudiest regions on Earth), and Union Glacier (79°S, a snow-covered glacier in the vast interior of Western Antarctica). The sites were strategically selected for studying the influence of urban aerosols, frequent and thick clouds, and extremely high albedo on the sky-diffuse radiance distribution. Our results show that, due to changing site-specific atmospheric conditions, the characterization of the weather-driven sky radiance distribution may require ground-based measurements.