Zone 3 CRPS

This report of solar forecast accuracy was automatically generated using the Solar Forecast Arbiter. Please see our GitHub repository for known issues with the reports or to create a new issue.

Report Metadata
Data

Observations and Forecasts
Data Validation

Metrics

Total Analysis
Year Analysis
Month Of The Year Analysis
Hour Of The Day Analysis
Date Analysis

Versions

Report Metadata

Name: Zone 3 CRPS
Start: 2020-01-01 08:00:00+00:00
End: 2020-06-30 23:59:00+00:00
Generated at: 2020-07-07 01:19:04+00:00

Data

This report includes forecast and observation data available from 2020-01-01 08:00:00+00:00 to 2020-06-30 23:59:00+00:00.

Observations and Forecasts

The table below shows the observation, forecast, and reference forecast triplets analyzed in this report. The reference forecast is listed as None if it was not selected during the report configuration. The table includes the unprocessed observation, forecast, and reference forecast interval label (beginning, ending, instantaneous) and interval length. If these quantities do not match, the Solar Forecast Arbiter must align and/or resample the data before computing error statistics. The Solar Forecast Arbiter aligns the observation data to the forecast data. Reference forecasts must have the same interval label, length, and type as the associated forecasts. The aligned and resampled parameters are also shown below, including a unique name for the aligned and resampled observation/forecast pair that appears in the metrics plots, tables, and CSV download.

Table of data alignment parameters
Aligned Pairs		Observations		Forecasts		Reference Forecasts
Name	Interval Label Interval Length	Name	Interval Label Interval Length	Name	Interval Label Interval Length	Name	Interval Label Interval Length
Desert Rock NV Day Ahead GEFS ghi	ending 60 min	Desert Rock NV ghi	ending 1 min	Desert Rock NV Day Ahead GEFS ghi	ending 60 min	None
Albuquerque New Mexico Day Ahead GEFS ghi	ending 60 min	Albuquerque New Mexico ghi	ending 1 min	Albuquerque New Mexico Day Ahead GEFS ghi	ending 60 min	None
Hanford California Day Ahead GEFS ghi	ending 60 min	Hanford California ghi	ending 1 min	Hanford California Day Ahead GEFS ghi	ending 60 min	None
University of Arizona OASIS Day Ahead GEFS ghi	ending 60 min	University of Arizona OASIS ghi	ending 1 min	University of Arizona OASIS Day Ahead GEFS ghi	ending 60 min	None
University of Nevada Las Vegas Day Ahead GEFS ghi	ending 60 min	University of Nevada Las Vegas ghi	ending 1 min	University of Nevada Las Vegas Day Ahead GEFS ghi	ending 60 min	None

The plots below show the realigned and resampled time series of observation and forecast data as well as the distribution of forecast vs observation data.

Controls to pan, zoom, and save the plot are shown on the right. Clicking on an item in the legend will hide/show it.

Data Validation

The Solar Forecast Arbiter applied its data validation toolkit to the observation data. The table below shows the selected filters and the number of observations that were removed from each forecast/observation pair.

Table of data validation results
Aligned Pair	Desert Rock NV Day Ahead GEFS ghi	Albuquerque New Mexico Day Ahead GEFS ghi	Hanford California Day Ahead GEFS ghi	University of Arizona OASIS Day Ahead GEFS ghi	University of Nevada Las Vegas Day Ahead GEFS ghi
Observation	Desert Rock NV ghi	Albuquerque New Mexico ghi	Hanford California ghi	University of Arizona OASIS ghi	University of Nevada Las Vegas ghi
NIGHTTIME	133058	133092	133075	133240	133071

These intervals were removed from the raw time series before resampling and realignment. For more details on the data validation results for each observation, please see the observation page linked to in the table above.

Data providers may elect to reupload data to fix issues identified by the validation toolkit. The metrics computed in this report will remain unchanged, however, a user may generate a new report after the data provider submits new data. The online version of this report verifies that the data was not modified after the metrics were computed.

Data Resampling and Alignment

The Solar Forecast Arbiter's preprocessing applies the following operations to the data:

Apply the data validation tests and exclude the matched data.
For deterministic forecasts with interval lengths that are longer than the observations interval lengths,
1. Resample the observations to the forecast using the mean.
2. Remove resampled observation data if more than 10% of the points in the resampled interval are missing. For example, if 1-minute observations are resampled to 60-minute means, then a 60 minute period must have no more than 6 minutes of missing data.
Remove times that do not exist in both the resampled observations, the forecasts, and, if selected, the reference forecasts.

The table below summarizes the data preprocessing results.

Table of data preprocessing results
Preprocessing Description	Desert Rock NV Day Ahead GEFS ghi Number of Points	Albuquerque New Mexico Day Ahead GEFS ghi Number of Points	Hanford California Day Ahead GEFS ghi Number of Points	University of Arizona OASIS Day Ahead GEFS ghi Number of Points	University of Nevada Las Vegas Day Ahead GEFS ghi Number of Points
TOTAL FLAGGED VALUES DISCARDED	133429	133128	135454	133240	133072
Total Forecast Values Dropped	0	0	0	0	0
ProbabilisticForecast Values Discarded by Alignment	1832	1901	1889	1868	1834
Observation Values Discarded by Alignment	220	149	201	159	206
ProbabilisticForecast Undefined Values	0	0	0	0	0
Observation Undefined Values	0	0	0	0	0

Users may wish to fix the data issues by having new or missing data uploaded. The metrics computed in this report will remain unchanged, however, a user may generate a new report after the data provider submits new data.

Metrics

The table below shows the normalization, deadband, and cost parameters used in calculating the metrics. The normalization factor is applied when calculating the metrics NMAE, NMBE, and NRMSE. The normalization is in the same units as the forecast and observation. By default, AC power forecasts are normalized by AC capacity and DC power forecasts are normalized by DC capacity. Normalization for all other forecasts is undefined, and the metric values are set to nan. The deadband accounts for observation uncertainty by setting the error (forecast - observation) equal to 0 for any point that is within the deadband. The error is unchanged for any point that is outside the deadband. The deadband is specified as a percentage of the observation value at each time. A value of None indicates that no deadband was applied for that observation/forecast pair. The deadband is accounted for in the following metrics: MAE, MBE, RMSE, MAPE, NMAE, NMBE, NRMSE, Skill, Cost. It is ignored for all other metrics. The Cost metric is calculated using the set of cost models defined by cost parameters. See the solarforecastarbiter-core documentation for a description of how these cost parameters are used to calculate Cost.

Table of metrics metadata
Name	Normalization	Deadband (%)
Desert Rock NV Day Ahead GEFS ghi	nan	None
Albuquerque New Mexico Day Ahead GEFS ghi	nan	None
Hanford California Day Ahead GEFS ghi	nan	None
University of Arizona OASIS Day Ahead GEFS ghi	nan	None
University of Nevada Las Vegas Day Ahead GEFS ghi	nan	None

A table of metrics over the entire study period and figures for the selected categories are shown below. Metrics may be downloaded in CSV format by clicking here.

Table of total metrics
Forecast	CRPS
Desert Rock NV Day Ahead GEFS ghi	21.7
Albuquerque New Mexico Day Ahead GEFS ghi	29.6
Hanford California Day Ahead GEFS ghi	19.3
University of Arizona OASIS Day Ahead GEFS ghi	17.5
University of Nevada Las Vegas Day Ahead GEFS ghi	18.3

Total Analysis

Metric totals for the entire selected period.

Year Analysis

Metrics per year.

Month Of The Year Analysis

Metrics per month.

Hour Of The Day Analysis

Metrics per hour of the day.

Date Analysis

Metrics per individual date.

Versions

This report was created using open source software packages. The relevant packages and their versions are listed below. Readers are encouraged to study the source code to understand exactly how the data was processed.

Package	Version
solarforecastarbiter	1.0rc1
pvlib	0.7.2
pandas	1.0.3
numpy	1.18.2
scipy	1.4.1
statsmodels	0.11.0
plotly	4.5.3
bokeh	1.4.0
netcdf4	1.5.3
xarray	0.15.0
tables	3.6.1
numexpr	2.7.1
bottleneck	None
jinja2	2.11.2
python	3.7.8
platform	Linux-3.10.0-957.5.1.el7.x86_64-x86_64-with-debian-10.4