Expert Comment: Why does the OBR keep making large forecasting errors?

Persistent forecasting errors in growth, productivity and inflation have put the Office for Budget Responsibility (OBR)—the UK Government’s official forecaster — under scrutiny.

MPs have launched a Treasury Committee Inquiry - 'The OBR: 15 Years On' - to look into the OBR’s performance on its economic and fiscal forecasts, which have time and again tended to overestimate the UK’s real GDP growth, overestimate productivity, and underestimate inflation volatility.

In our submission to this inquiry, we argue that significant improvements to the OBR’s forecasting model can and should be made.

The introduction of the OBR (which formally became a statutory body in 2011) was intended to improve transparency, accountability and credibility in policy making.

Why does this matter?

The OBR is tasked with producing twice yearly independent economic forecasts for the government, which are adopted as government forecasts by the Chancellor of the Exchequer. As such, they wield a lot of power, being used to assess whether the Chancellor’s fiscal rules on borrowing and debt are likely to be met, and thus shaping spending and borrowing decisions.

If the OBR forecasts suggest that the fiscal rules will be breached, the Chancellor adjusts the fiscal plans over the budget period.

Ever tighter finances, urgent and competing demands, and restrictive fiscal rules, have seen Budgetary decisions becoming more marginal, with Chancellors waiting for and relying upon OBR official forecasts when making spending and taxation decisions.

The fiscal headroom - the extra money the Chancellor can spend or use for tax cuts without breaking the fiscal rules - is dependent on how accurate the forecasts are over the medium to long-run (3-5 years), and small revisions to the forecasts can lead to large policy adjustments, which may be based on erroneous assumptions, poor models, or inaccurate data.

Inaccuracy of OBR forecasts can therefore have negative consequences for the credibility of fiscal policy making, as forecasts that substantially deviate from the outturns question the very premise of the specified policy.

The dawning realisation within the OBR that its forecasts are inaccurate have started to cause their own disruption during the budget-setting process, with the Autumn Budget’s last minute revisions likened to a points deduction in football. 

What's going wrong?

The central component of the OBR's forecasting approach is a large-scale macroeconomic model, originally developed by the Treasury in the 1970s and handed to the OBR in 2010. It describes the economy through accounting identities, behavioural equations, and technical relationships, and incorporates judgmental adjustments. 

Historically, the OBR, under this model, has tended to overestimate real GDP growth and productivity, and has underestimated inflation volatility.

We attribute these persistent errors to how the model handles what are called ‘structural shifts’. Structural shifts occur when something like an external shock causes a change in the relationship between economic variables that a model relies upon, putting the model out of kilter with the new reality.  Such shocks have been common in recent years, with Covid-19 lockdowns, roll-out of vaccines, global supply chain issues, and Russian invasion of Ukraine to name a few.

Why current models fail

If a major shock—such as a financial or energy crisis—shifts the economic mean or trend, and the model is not updated, forecast errors will converge on the previous mean or trend. 

When the mean falls for example, the model will repeatedly forecast upwards to "correct" its trajectory back to the old mean or trend.

In the UK we have seen this occur many times with productivity and growth forecasts, which consistently fail to match up to targets, but continue to be projected to return to trend in the long-run.

Because the model relies on a large interdependent system, a shift can alter many equations, making it difficult to update quickly. It can take many periods to collect enough information to determine which parameters changed and to estimate their new values.

Our solution

To address this, we propose an approach that corrects large forecast errors following these unexpected structural shifts. This approach is called the Smooth Random Walk (SRW) method.

The SRW generates robust mean forecasts by averaging over the previous two years while accounting for the dynamics of the system. 

The advantage of this approach is that it can be implemented rapidly across alternative models and systems to produce improved accuracy, without needing to fully re-estimate or update a complex forecasting system.

Take productivity for example, a particularly important measure to forecast, as it is a key factor driving economic growth, the government’s stated number one economic priority.  The OBR’s model has consistently over-estimated productivity growth in its forecasts due to its inability to quickly adapt to consecutive structural shifts.

Prior to the pandemic, our SRW-based method demonstrated substantially better performance than the OBR after roughly 5 quarters ahead. Extending the results through 2025, there are increasingly large differences between the OBR and SRW for longer-ahead forecasts. While multi-period forecasts naturally become more uncertain further into the future, the longer-ahead OBR forecasts exhibit increasing forecast failure compared to our approach.

To ensure exact comparisons, our analysis utilised real-time historical data vintages. The OBR changed its measure of productivity in November 2022 from Real Gross Value Added to Real Gross Domestic Product. Our real-time forecasts strictly followed the OBR's definition of productivity at each respective point in time. We also accounted for historical revisions to total hours worked and output. 

To illustrate, we compare the OBR's forecasts of productivity against our SRW forecasts, shown in the figure below with the OBRs forecasts in panel A and the SRW forecasts in panel B.

Implications for policy

At a moment when budget and policy decisions sit on the edge of a knife, the importance of accurate forecasting cannot be overstated. Restoring credibility to the national forecaster is of paramount importance, and our research shows that it can be done.

Implementing these changes to its model and reducing the occurrence of mis forecasts will give the Chancellor more clarity over which policies to pursue, as the Government searches for growth.

Persistent mis-forecasts are a cause for concern and need to be addressed. 

The OBR should evaluate what features of their model lead to poor long-run forecasts, and compare their forecasts to a smooth robust procedure to identify where their model's inbuilt corrections are counterproductive. 

Not only can our approach avoid badly inaccurate forecasts, it can help improve the underlying forecasting model. Forecast accuracy can be rapidly improved even when facing a sequence of shifts, such as the Covid-19 lockdowns, supply chain disruptions, and recent energy crises.  

Improving forecast robustness is essential for restoring confidence in fiscal policy at a time of heightened economic uncertainty.

For more information about this story or republishing this content, please contact [email protected]