Validation of Sitewind Version 4

8 August 2020

Dr. Philippe Beaucage, Lead Research Scientist

Akila Gothandaraman, Research Scientist
Satish Kasibhatla, Global Head Information Services

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential.
Outline

I. Introduction to coupled mesoscale-

microscale models
II. Description of Sitewind
III. Wind project sites for validation
IV. Sitewind validation –
• Part 1: Raw Sitewind outputs
• Part 2: Adjusted Sitewind outputs

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 2
 Wind is part of a complex weather system

3
UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential.
Challenges in Wind Flow Modeling
• Spatial resolution must be ~ 100 m or better over large domains
• Must simulate a wide range of atmospheric conditions
» Difficult to capture all
» With limited onsite observations, which conditions are most relevant is a matter of judgment
• Advanced models require much more computer time and expertise than simpler models

http://www.jma.go.jp/jma/jma-eng/jma-center/nwp/nwp-top.htm
UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 4
Brief History of Commercial Wind Flow Models

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 5
Brief History of Sitewind

Version 1: MASS w/ NCEP/NCAR + WindMap (~ 1999)

Version 2: MASS w/ NCEP/NCAR + WindMap (~ 2004)

Version 3: WRF w/ ERA-Interim + WindMap (2016)

Version 4: WRF w/ ERA5 + WindMap

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 6
The Sitewind Process

Mesoscale Numerical Weather

Step 1
Prediction (NWP) Model

Step 2 Microscale Wind Flow Model

Adjustments to Onsite
Step 3
Measurements

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 7
Numerical Weather Prediction Modeling

Weather Research and Forecasting (WRF) Δx = 27 km

• WRF is built with state-of-the-art data
assimilation, dynamic and physics schemes Δx = 9 km
• WRF is open-source
» large community of developers
» updated twice a year
• WRF is fast
Δx = 3 km

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 8
Mesoscale Modeling Flowchart
Initial and Lateral
Terrain Land
Soil Type NDVI … Boundary Conditions
Elevation Cover e.g. ERA-Interim, ERA5

Mesoscale NWP Model

Sample of days.
Day 1 Day 2 … Day N
Typically, N= 365

Mesoscale Statistics Files (STAT)

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 9
WindMap Microscale Model

• Mass consistent (mass conserving)

• Perturbs initial NWP wind field by the smallest
possible amount to achieve zero divergence
• Surface roughness changes modeled by Internal
Boundary Layers (IBL).
• Preserves as much information as possible from the
mesoscale while capturing local topographic and
roughness effects – no CFD-type homogeneous “inlet
condition”

R.B. Stull (1988), “An introduction to boundary layer meteorology”

UL and the UL logo are trademarks of UL LLC © 2018. Proprietary & Confidential. 10
WindMap Flowchart

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 11
Sitewind Version 3 vs. Version 4 (Part 1)
Mesoscale Modeling
Characteristics Sitewind version 3 Sitewind version 4

Model Version WRF 3.5.1 WRF 4.1.3

Reanalysis Data ERA-Interim ERA5
Terrain Elevation Data ~ 90-m resolution SRTM ~ 90-m resolution SRTM
~ 90-m resolution in-house ~ 90-m resolution in-house
Land Cover Data
dataset [1] dataset [1]
Sea Surface Temperature Data From reanalysis From reanalysis
Physics Parameterization [2] Same as version 3
Dynamical Downscaling 27-9-3-1 km 27-9-3-1 km
365 random days out of the 365 random days out of the
Sampling size
2001-2015 period 2001-2015 period

[1] NLCD 2001 in the US, Corine in Europe, GeocoverLC otherwise

[2] Y:\ResearchandDevelopment\WRF\WRF_DOCS\WRF_Mesoscale_Modeling_Specs.docx 12
ERA-Interim vs. ERA5

ERA-Interim ERA5
Period 1979 – present 1950 – present, produced in 2 phases

Availability behind real time 2 - 3 months 2 – 3 months (final product)

2 – 5 days (ERA5T)

Assimilation system 2006 (31r2), 4D-Var 2016 (41r2), 4D-Var, hybrid EDA providing B

Model input (radiation and surface) As in operations, (inconsistent SST and sea Appropriate for climate, e.g., evolution greenhouse
ice) gases, volcanic eruptions, sea surface temperature and
sea ice

Spatial resolution 79 km globally 31 km globally

60 levels to 10 Pa 137 levels to 1 Pa

Uncertainty estimate From 10-member EDA at 62 km

Output frequency 6-hourly analysis fields Hourly (three-hourly for the ensemble),
Extended list of parameters
9 petabytes (1950 – timely updates)

Extra observations Mostly ERA-40, GTS Various reprocessed CDRs, latest instruments

Variational bias control radiosondes Satellite radiances, RAOBCORE Also ozone, aircraft, surface pressure, RISE

Land downscaling product ERA-Interim land, 79 km ERA5L, 9 km (forced by ERA5)

Table taken from Hersbach, H. and co-authors (2019). “Goodbye ERA-Interim, hello ERA5”. ECMWF Newsletter, No. 159, p. 17-24 13
Sitewind Version 3 vs. Version 4 (Part 2)
Microscale Modeling
Characteristics Sitewind version 3 Sitewind version
4 - automated
Model Version WindMap 4.0.0 WindMap 4.9.6
Mesoscale Data WRF STATs WRF STATs
~ 10-m resolution NED in the US,
Terrain Elevation Data Same as Version 3
~ 30-m resolution SRTM otherwise
~ 30-m resolution NLCD 2001 in the
US, ~ 90-m resolution in-
Land Cover Data
~ 30-m resolution GeocoverLC house dataset [1]
otherwise
Land cover to surface
Manual edits Global look-up table
roughness conversion

[1] NLCD 2001 in the US, Corine in Europe, GeocoverLC otherwise

14
Methodology for Validation

15
Validation Sites
ID Country State Terrain # Towers

1 USA Oklahoma Simple 8

2 USA New York Simple 6
3 USA Colorado Moderate 3
4 USA California Moderate 7
5 Spain Galicia Moderate 3
6 Spain Castilla-La Mancha Moderate 7
7 USA Wyoming Moderate 6
8 Canada Quebec Complex 9
9 Spain Basque Country Complex 4

10 USA California Complex 5

11 Canada Ontario Complex, coastal 3

16
Sitewind Validation Part 1: Raw
Outputs

UL and the UL logo are trademarks of UL LLC © 2018. Proprietary & Confidential. 17
Sitewind Validation Part 1: Raw Outputs

UL and the UL logo are trademarks of UL LLC © 2018. Proprietary & Confidential. 18
Sitewind Validation Part 1: Raw Outputs
Error statistics for (raw) WRF and Sitewind version 3 and version 4

WRF WRF Sitewind Sitewind

version 3 version 4 version 3 version 4 -
Error metrics automated

-0.310 -0.201 -0.056 -0.001

Mean bias (m/s)
0.864 0.837 0.775 0.634
RMSE (m/s)

• The mean bias is close to 0 m/s for both Sitewind version 3 and version 4.
• On average, Sitewind version 4 performs better than Sitewind version 3
• Sitewind version 4 yields a lower RMSE at 8 out of 11 sites (see previous slide)
• On average, downscaling WRF with the microscale model WindMap improves both the bias and
RMSE than relying on WRF alone

UL and the UL logo are trademarks of UL LLC © 2018. Proprietary & Confidential. 19
Sitewind Validation Part 2: Adjusted
Outputs

UL and the UL logo are trademarks of UL LLC © 2018. Proprietary & Confidential. 20
Sitewind Validation Part 2: Round-Robin Validation with Adjustment
Red → reference mast
Blue → target masts

• Adjust WRG to reference

mast
• Calculate bias at remaining
target masts
• Repeat using each mast as
reference mast
• Compute RMSE from set of
biases

21
Validation: Sitewind Version 2 vs. Version 3
Adjusted
Sitewind version 3 yields a lower RMSE at 7 out of 11 sites

SW v2

SW v3

22
Validation: Sitewind Version 3 vs. Version 4 Automated

Adjusted

Automated Sitewind version 4 yields a lower RMSE at 7 out of 11 sites

23
Validation: Sitewind Version 3 vs. Version 4

Error statistics for (adjusted) Sitewind version 3 vs. version 4

Sitewind Sitewind
version 3 version 4
Error metrics
automated

Mean bias (m/s) -0.003 -0.003

RMSE (m/s) 0.397 0.374

• Sitewind version 4 performs better than Sitewind version 3 in terms of the

RMSE
• The bias remains practically the same for Sitewind version 3 and version 4.
24
 Questions?
Digitalsolutions.renewables@ul.com
Thank you

UL and the UL logo are trademarks of UL LLC © 2020. Proprietary & Confidential. 25