This section provides a concrete example of how a comparative performance assessment study can be conducted when a large number of models is involved. In the present case, a rather exhaustive literature survey provided a list of 35 clear – sky broadband irradiance models that can predict instantaneous or short-term (e. g., hourly) direct and diffuse irradiances from limited information on the optical properties of the atmosphere. A computer program that can use the benchmark dataset
Fig. 20.6 Comparison between meteorological (METSTAT) site-based and (Perez) satellite-based direct normal irradiance (DNI) solar radiation estimates for the 1991-2005 US National Solar Radiation Data Base update (Wilcox et al. 2007). Grayscale contours (originally in color) indicate the average 1998-2005 DNI predictions from the Perez model, and similarly color-coded circles indicate those from the METSTAT model
mentioned in Sect. 5.1 to evaluate all these models is included on the CD-ROM (file ‘Models_performance_compar. f’). Only a subset of 15 models is described in what follows, owing to the demonstration purpose of this validation exercise and space limitations. Most of the models have been described and discussed in previous studies (e. g., (Gueymard 1993, 2003b), so that only a summary is provided here, except where more details are necessary. The models are listed in alphabetical order below.
• Model 1—ASHRAE
This is the model used by engineers to calculate solar heat gains and cooling loads in buildings. It was first introduced in 1972, but new monthly coefficients have appeared recently (ASHRAE 2005), which are used here. Note that, beyond these empirical coefficients and solar zenith angle, this model does not depend on any atmospheric data.
• Model 2—Bird
This is the original Bird model (Bird and Hulstrom 1981a, b), with only one modification, required by changes in turbidity measurement practice. At the time this model was developed, the aerosol optical depth (AOD) was measured by sunphotometers with at most two channels, centered at 380 and 500 nm, hence the model’s requirement for the AOD at these two wavelengths. Since the early 1990s, networks of multiwavelength sunphotometers, with typically 5-7 aerosol channels, have expanded worldwide (see, e. g., http://aeronet. gsfc. nasa. gov). Therefore, it is now easier than ever to obtain the turbidity coefficients a and в by fitting the experimental AOD at various wavelengths, xax, to Angstrom’s Law:
таХ = в(Х/Хо)-а (20.10)
where Ao = 1000nm. When a and в are known, the specific AOD at 380 and 500 nm required by Bird’s model can be replaced by в0.38-а and в0.5-а, respectively. This respects the model’s integrity while considerably expanding its applicability.
• Model 3—CLS
The Cloud Layer-Sunshine model (Suckling and Hay 1976, 1977) is based on original work by Houghton (1954) and Monteith (1962). The CLS model has been independently validated for average sky conditions during the IEA Task IX mentioned in the previous section, but not for clear skies only—at least outside of Canada. The original expression for the aerosol transmittance, Ta = 0.95m, where m is the air mass, is used here.
• Model 4—CPCR2
This two-band model (Gueymard 1989) has already been tested extensively in various studies (Battles et al. 2000; Gueymard 1993, 2003a, 2003b; Ineichen 2006; Olmo et al. 2001). It normally requires separate values of a and в for each of the two wavebands (290-700 nm and 700-4000 nm) considered by the model, i. e., (a1, в1) and (a2, вг) with the constraint в1 = в20.7а1-а2. This information can be derived from the current sunphotometric data by appropriate application of Eq. (20.10). If not possible, the model can be accommodated with simply a1 = a2 = a and в1 = вг = в yielding only a modest degradation of performance.
• Model 5—ESRA2
The original version of this model (Rigollier et al. 2000) has been used to derive the latest edition of the European Solar Radiation Atlas (Scharmer and Greif 2000). The Linke turbidity factor, TL, was then the basis to evaluate the effect of aerosols. However, this factor cannot be measured directly, and therefore needs to be evaluated by inversion of an appropriate irradiance model, using experimental clear-sky direct irradiance as the input. This creates a problem in the context of validation studies since the measured direct irradiance cannot be used both to test the model’s predictions and to derive its inputs (see Rule #1 in Sect. 5.3). A new version of the model, which is tested here, rather calculates TL from air mass, precipitable water, and в (Remund et al. 2003).
• Models 6-8—Iqbal’s Parameterization Models A, B and C
These models are fully described in the original publication (Iqbal 1983), and have been tested previously, to some extent (Battles et al. 2000; Gueymard 1993). Scatterplots of Iqbal C’s model appear in Fig 20.3 for the benchmark dataset considered here.
• Model 9—Kasten
This classic model (Kasten 1980, 1983; Kasten and Czeplak 1980) has been expanded to provide direct and global irradiance, both as a function of TL (Davies and McKay 1989). The latter version is used here. To overcome the difficulty in using TL while respecting the model’s intentional simplicity, a simple linear function of в has been used,
TL = 2.1331 + 19.0204 в (20.11)
where the numerical coefficients have been obtained by combining the empirical determinations of в = f(TL) proposed by different authors (Abdelrahman et al. 1988; Grenier et al. 1994; Hinzpeter 1950; Katz et al. 1982).
• Model 10—METSTAT
As mentioned in Criterion #6 of Sect. 2, this model has a deterministic algorithm that can be combined with statistical features so that correct frequency distributions of hourly irradiances can be obtained despite the use of daily or monthly-average turbidity and cloud input data (Maxwell 1998). The model is used here without these statistical corrections since short-term input data are available for validation, therefore respecting Rule #4 in Sect. 5.3. A modification to the model, however, is necessary since it uses the broadband aerosol optical depth, та, to evaluate the aerosol transmittance. Like TL, та can only be obtained indirectly from an inverted model and irradiance measurements. To circumvent this problem, a convenient methodology (Molineaux et al. 1998), which uses the concept of equivalent wavelength for broadband turbidity, is used here to derive та from a and в through
Ta = в [0.695 + (0.0160 + 0.066в0.7~“)да]~“. (20.12)
• Model 11—MAC
The McMaster (MAC) model has evolved slightly between its original derivation (Davies et al. 1975) and the latest performance assessment results (Davies and McKay 1989). The version described in the IEA Task IX report (Davies et al. 1988) is used here, with a Rayleigh transmittance formula corrected for its typographic error. In the absence of specific information on the most appropriate aerosol transmittance to be used here, the original formula Ta = 0.95m (Davies et al. 1975; Davies and Hay 1979), as for the BCLS model, is selected.
• Model 12—MRM5
This new, version 5, of the Meteorological Radiation Model, is described in Chap. 14. It contains important changes from the previous version 4 (Muneer 2004), which used incorrect numerical coefficients that considerably affected the model’s irradiance predictions (Gueymard 2003a, 2003b).
• Model 13—REST2
This two-band model (Gueymard 2008) is based on CPCR2, but incorporates completely revised parameterizations, which have been derived from the SMARTS spectral model (Gueymard 2001, 2005b). REST2 uses the same inputs as CPCR2, with the addition of the amount of nitrogen dioxide in a vertical atmospheric column, which can be defaulted if unknown. See Fig. 20.3 for scatterplots involving this model and the benchmark dataset.
• Model 14—Santamouris
This model’s algorithm (Santamouris et al. 1999) is in essence similar to that of the Bird model, except that a fixed turbidity is considered.
• Model 15—’Yang
The direct irradiance predictions of this model (Yang et al. 2001; Yang and Koike 2005) have been shown to perform very well (Gueymard 2003a, b). Its diffuse irradiance predictions have not been evaluated so far.
All radiation models require at least one input variable, namely the solar zenith angle, Z. The simplest models (e. g., ASHRAE) do not need more inputs (besides some empirical coefficients). The most sophisticated models (e. g., REST2) may require as many as 10 more inputs, mostly atmospheric data. For clarity, all the inputs (besides Z) required by the 15 models considered here are compiled in Table 20.1.
Using the specially-developed Fortran program contained on the CD-ROM (file ‘Models_performance_compar. f’), all these models (and more) have been run using the 30-point benchmark dataset recently proposed (Gueymard 2008) and previously mentioned in Sects. 3 and 5.1. This dataset is contained in file ‘Models_perf_exp. dat. txt’. Of course, to the benefit of the reader, this Fortran program can also
Table 20.1 Inputs required by the 15 clear-sky models under scrutiny
Key: m, air mass; En0, distance-corrected extraterrestrial irradiance; pg, far-field ground albedo; p, site pressure;T, dry-bulb temperature; Uo; total ozone in the vertical column; Un, total nitrogen dioxide in the vertical column; w, precipitable water; TL, Linke turbidity coefficient; Ta, broadband aerosol optical depth; a, Angstrom wavelength exponent; в, Angstrom’s turbidity coefficient, a>„, aerosol single-scattering albedo.
Table 20.2 Performance statistics, and their associated ranking (in bold italics), for direct irradi – ance predicted by the 15 clear-sky models under scrutiny, relative to a benchmark dataset
accommodate other, more voluminous datasets, if prepared with the same format. The present example is purposefully limited to a small dataset, so that the conclusions reached here should not be considered of general or “universal” validity. However, owing to the benchmark status of this dataset, it can be said that a model should not be considered as universal if it does not perform well under the conditions of this dataset.
A statistical analysis has been conducted from the differences between predicted and measured direct, diffuse and global irradiances; the summary performance results appear in Tables 20.2, 20.3 and 20.4, respectively. These tables include the
Table 20.3 Performance statistics, and their associated ranking (in bold italics), for diffuse irradi – ance predicted by the 15 clear-sky models under scrutiny, relative to a benchmark dataset
Table 20.4 Performance statistics, and their associated ranking (in bold italics), for global irradi – ance predicted by the 15 clear-sky models under scrutiny, relative to a benchmark dataset
MBE, RMSE, R2, t-statistic, Willmott’s index of agreement d, and Muneer’s accuracy score AS. For each of these performance indices, the corresponding model ranking is indicated. As could be expected, the models that use detailed atmospheric information (particularly on aerosols and water vapor) perform better than those with little or no such inputs. The main disturbing fact, however, is that the ranking methods disagree widely, particularly for diffuse and global irradiance. This confirms the need for more in-depth investigations on this issue.
The primary focus of this chapter has been to emphasize to the newcomer as well as the experienced solar radiation model developer, tester, or user, the nuances of model validation and performance evaluation. Section 2 addressed seven criteria describing typical solar radiation model approaches or types. Sections 3 and 4 described the principles of model validation and uncertainty analysis required for both measured data and uncertainties in model estimates. Sections 5.1 and 5.2 addressed some aspects of qualitative and quantitative model performance. Section 5.3 emphasized seven constituent elements of model validation that must be addressed in any evaluation. These include validation and input data quality, independence, and consistency of temporal and spatial extent, and validation limits. Section 6 discussed evolution and validation of model component parts, the importance of, and difficulties associated with, interpreting independent model validation, as well as demonstrated the practice (and difficulties) of comparing the performance of many models).
As the solar energy industry and scientific research into the detailed energy balance of the Earth continues to grow and evolve, it is critical that computational models be validated and tested as stringently as possible to provide decision makers, and the scientific community in general, with the most accurate, comprehensive, and well documented information possible.
 Inference. According to the rule-base from Table 7.1, six rules are set-up. At this step the fuzzy inputs are combined logically using the operator AND (Eqn. 7.14a) to produce the output values:
 Fuzzyfication. Crisp inputs are transformed into confidence levels of input linguistic variable attributes, being computed with the equations (7.18) and (7.20 a, b). For At = 14°C, the linguistic variable air temperature amplitude is characterized by three attributes with the corresponding confidence levels:
T4: mAt,4 = 0.2
T5: mAt,5 = 0.7
Te. mAt,6 = 0.8
Julian day j = 90 have both attributes SUMMER and WINTER:
S: mS = 0.4
W: mW = 0.6
It can be appreciated that the formulation is similar to the one proposed by Perez in his all-weather model. Nevertheless, some differences must be emphasized. On the one hand, a new term (exp(dn/2)) appears in the scattering indicatrix function. It was introduced by Kittler (1994) as a necessary correction according to the concept of relative scattering indicatrix.
On the other hand, six groups of a and b coefficients and other six groups of c, d and e coefficients are considered instead of employing continuous parameters for establishing the type of sky and the value of the coefficients of Eq. (17.9). Subsequently, six different gradation and indicatrix functions exist whose combination may produce up to thirty six different types of sky. Notwithstanding, the standard only includes those fifteen considered to be of more interest, although some work in low latitudes seems to make clear the need of increasing the current type of skies under consideration (Wittkopf and Soon 2007).
In order to choose the type of sky to be used in each case, one of the following procedures may be employed:
 By comparison of the theoretical functions of gradation, g(0), and indicatrix, f (£), with their observed counterparts. The observed gradation function may be determined from luminance measurements in different points of the sky located on the plane of the solar meridian and on another one perpendicular to it. In contrast, for establishing the observed indicatrix function, it is necessary to know the luminance in different points of almucantars of different altitudes.
 By the analysis of the ratio between the zenith’s luminance and the diffuse illuminance (Dv) on a horizontal plane. In this respect, Kittler et al (1997) graphically