Introduction
Several weeks ago, I published a summary of storm-force sustained winds, hurricane-force gusts, and a drop in water level approaching six feet across Lake Erie’s western basin, all courtesy of the passage of a deep low pressure system on November 27, 2019 (click here to read the summary). This article examines how the weather forecast models performed during the event.

Weather Forecast Model Basics
In order to evaluate how the forecast models performed, it’s helpful to understand a few basics.

Meteorologists use some of the world’s fastest computers running a suite of sophisticated software programs (forecast models) to forecast the strength and movement of low pressure systems and their hazards, such as precipitation, thunderstorms, and high winds. Because supercomputers capable of performing quadrillions of calculations per second are needed to crunch the numbers, running weather forecast models is almost exclusively the business of national government agencies, such as the National Weather Service (NWS).

Nearly all weather forecast models are grid-based, meaning the region covered by the model (not all models are global) is divided into a series of boxes. The size of each box is considered the model’s resolution with higher resolution models having smaller boxes. In order to save precious computing power, a forecast – such as wind speed — is typically only prepared for the corners of each grid box. Figure 1 shows a 56 kilometer grid over Lake Huron while Figure 2 shows the wind barbs placed at the corners of each box.

All forecasts start with the current state of the atmosphere, which the forecast model determines by ingesting millions of observations from surface-based weather stations, weather balloons, and satellite-based instruments. Because the observation locations don’t neatly correlate with the model’s grid boxes, the forecast model must first laboriously conform the observations to the grid in a process known as data assimilation. Once this step is complete, the forecast model begins predicting the future weather conditions. Most forecast models take two to three hours to finish.

The ability of a model to accurately forecast small-scale features, such as near-shore wind and waves, improves as model resolution increases (boxes get smaller). However, the demands on computing power increase dramatically as the grid boxes get smaller.  This results in the overall length of the forecast period decreasing as model resolution increases. For example, a high resolution forecast (small boxes) is only available for three to four days into the future, while a low resolution forecast (large boxes) might extend ten to fourteen days (or more). Forecasters generally use the highest resolution model covering the period for which they are preparing a forecast. Figure 3 shows examples of wind forecasts for the southern portion of the Chesapeake Bay at three model resolutions.

So, How Did the Models Perform on November 27, 2019?
There are several forecast models covering the Great Lakes, however this review is limited to the forecasts from the Global Forecast System (GFS), the North American Model (NAM), and the ensemble version of the GFS, called the Global Ensemble Forecast System (GEFS). The data used in the following comparisons were downloaded on the morning of November 27, 2019 and represent forecasts during the first twenty-four hours of each model run. My home port of Lorain, Ohio will be used for the comparison.

The sustained wind forecasts from the models is shown in Figure 4. The models were in agreement on the overall trend – a slight increase in sustained winds during the morning hours, followed by a steady decrease beginning around 7:00 pm. The GFS and NAM ranged from 30 to 35 knots, with the GFS slightly higher than the NAM. The GEFS was considerably lower, with maximum sustained winds around 25 knots.

The gust forecasts are shown in Figure 5 (the GEFS does not calculate gust speed, therefore it’s not included). Similar to the sustained winds, the gust forecasts from the NAM and GFS were in close agreement. The gusts also followed the same trend as the sustained winds – a steady increase during the morning (to 45 to 50 knots) followed by a steady decrease beginning in the afternoon.

So how did the models do? The sustained wind forecasts from the three models and the observations at 3-hour intervals are shown in Figure 6. While the models were on track during the early morning, all three missed the dramatic increase in sustained winds — from 30 to 45 knots — that occurred from mid-morning through early afternoon. Also, in contrast to the steady decline suggested by the models during the early evening hours, the sustained winds remained above 40 knots into Thursday morning – more than twice the speed that was forecast for the period. A similar pattern emerges from a comparison of the forecasts for gust speed and observations (Figure 7).

As these comparisons show, the forecast models did very poorly on November 27, 2019. Figure 8 shows the values and trend in sustained wind forecast errors during the period. The NAM and GFS were quite accurate at 7:00 am and were slightly higher than the observations at 10:00 am. However, the errors increased dramatically by late morning and reached 25 to 30 knots for most of Wednesday. The errors in gust forecasts were equally significant (click here for graph).

Why did the forecast models do so poorly? The answer lies in a review of the barometric pressure forecasts and observations at South Bass Island (Figure 9). The barometric pressure forecasts from the NAM, GFS, and GEFS were very similar and suggested a steady increase from 7:00 am on Wednesday to 7:00 am on Thursday. However, the observations confirm that instead of increasing over the period, the pressure fell by nearly 2 millibars between 7:00 am and 10:00 am before steadily increasing. It’s possible that the approaching low pressure system quickly strengthened as it passed over the relatively warm water of Lake Michigan.

The primary controller of wind speed is the pressure gradient force – the rate of change in barometric pressure across a region. The unexpected drop in barometric pressure on Wednesday morning increased the pressure gradient across the lower Great Lakes and the wind speed quickly increased in response. Since the forecast models didn’t anticipate the drop in pressure and corresponding increase in pressure gradient, it was inevitable that they would significantly miss the wind forecast.

The Added Value of a Forecaster
Although November 27, 2019 isn’t a good example, forecast model accuracy has improved over the past several years due to faster computers and coding advancements in the models. But even as forecast models have improved, experienced meteorologists regularly produce more accurate forecasts by combining data from the forecast models, pattern recognition, and their knowledge of the atmosphere. Such was the case on November 27, 2019.

Figure 10 shows the wind speed forecasts from the forecast models, the observed wind speeds, and the forecast produced by meteorologists at the National Weather Service (NWS). While the NWS forecasts also fell below the observed wind speeds, their forecasts had the smallest errors, particularly from Wednesday afternoon through Thursday morning.

Although the NWS gust forecasts fell below the observations, the overall trend in their forecast was on target and far more accurate than either the NAM or the GFS (Figure 11).

The knowledge of an experienced forecaster almost always pays a dividend. If your goal is to improve the accuracy of your wind forecasts, it pays to learn a little meteorology and hone your pattern recognition skills. And most importantly – treat the forecast model output with a bit of skepticism instead of accepting it as absolute fact.