As part of the commissioning process for five new Atlas xenon arc Weather-ometers, the degree of uniformity within each device and consistency among devices were evaluated. We measured the surface temperatures of white and black polycarbonate plaques and their relative rates of color shift and gloss loss in nine sample positions from top to bottom in each device. Temperature gradients were found in all devices. These were small for white specimens (1°C–3°C) but larger and more variable for black specimens (5°C–8°C). Degradation rates generally followed the temperature profiles. For whites, the relative standard deviation (SD) for color shift and gloss loss rates for all devices and all positions was 0.042, suggesting that rates for 95 % of specimens should be within ±8 % of the mean. Black specimens had a higher overall relative SD of 0.061, mostly due to higher ranges for gloss loss, suggesting that rates for 95 % of the specimens should be within ±12 % of the mean. Therefore, rate differences less than approximately 15 % in light colors and approximately 25 % in dark colors should not be considered as significant unless samples are exposed literally side-by-side. The values of a property at any pull may vary much more if the property is changing rapidly around that time. Evaluations should involve taking enough data points to understand the shapes of the degradation curves. Regularly repositioning specimen holders vertically during exposure reduced the in-device variability by 50 to 60 %.