Quantitative Performance of ARIMA, kNN, ANN, GA/ANN

Models built on the historical data of 2009 and 2010 (the shaded area in Figure 15.7) are applied to the 2011 data (unshaded area) without modifications or retraining. Given that, as seen in Figure 15.1, there is a strong seasonality in power-output variability, we expect a strong seasonality in the accuracy of the predicted values as well. To study the influence of this factor, we consider three solar-variability seasons, or periods, that are subsets of the total error- evaluation dataset. The three periods are defined based on the solar- variability study summarized in Figure 15.1 as

• High variability, from January 1, 2011, to April 30, 2011 (P1)

• Medium variability, from May 1, 2011, to June 30, 2011 (P2)

• Low variability, from July 1, 2011, to August 15, 2011 (P3)

All of the statistical metrics for the error are calculated for the three periods. Table 15.1 lists them for the 1 h and 2 h forecasting horizons, respectively.“P1,” “P2,” and “P3” and “Total” identify the error values for the 3 subsets and for the entire validation dataset, respectively. The boldfaced values identify the best model for a given error metric and a given dataset.

The scatter plots for 1 h ahead forecasting are depicted in Figure 15.9; those for 2 h ahead are shown in Figure 15.10. In these figures, each row corresponds to a different forecasting model and each column corresponds to a different variability period. Morning and afternoon values in the scatter plots are iden­tified by different symbols. The dispersion of the forecast values about the identity line is uniform for morning or afternoon. This is clear evidence that the models are free of systematic errors related to daily solar variation.

Table 15.1 shows that the two ANN-based methods, ANN and GA/ANN, clearly outperform the others. Only in terms of MBE is the GA/ANN worse

than the ARIMA for certain periods. The table also shows how strongly the accuracy of the methods depends on season. For all models, the error metrics for P3 are substantially better than those for the other two periods, as expected. The table shows that, for its simplicity, kNN performs very well for low – variability situations. This is not surprising given that in those cases the mapping pattern/forecasting becomes “almost” deterministic. However, for the periods of medium and high variability kNN performs the worst for most error metrics.

The results also show that the GA/ANN represents a large improvement with respect to the results from the ANN predictor for both forecasting hori­zons. Notably, this improvement is more substantial for the periods of higher variability, P1 and P2. Scatter plots m/n from Figure 15.9 show a clear clus­tering of the data close to the unity line when compared to plots j/k. For the 2 h ahead forecasting, the improvement is even greater, as seen in Figure 15.10. Table 15.2 compares the ARIMA, kNN, and ANN models with respect to the persistence model in terms of RMSE for the entire validation period. A positive value indicates a decrease in RMSE relative to the persistence model; a nega­tive value indicates an increase. The table shows that overall only kNN performs worse than the persistence model. The ARIMA model shows substantial improvement for the 2 h time horizon. Both ANN models outper­form the others, with the GA/ANN hybrid yielding consistent improvements of more than 30% in relation to persistence for a wide range of conditions.

FIGURE 15.9 Scatter plots for 1 h ahead forecasts (kW). Each row corresponds to a different model. Row 1: clear-sky index persistence; row 2: kNN; row 3: ARIMA; row 4: ANN; row 5: GA/ ANN. Each column corresponds to forecasting for a different variability period. Left: January – April 2011 (high variability); middle: May-June 2011 (medium variability); right: July-August 2011 (low variability). Squares identify morning values; circles identify afternoon values.