The value of forecasting in an uncertain world

A speech delivered by Dr John McDermott to the New Zealand Manufacturers and Exporters Association (NZMEA) in Christchurch

"We are all forecasters." Tetlock and Gardner, Superforecasting.¹

It has often been said that there is no escaping forecasting. Every decision we make in life involves forming a view about what the future might hold. Likewise, there is no escape for the Bank. We have a Policy Targets Agreement that requires us to keep future inflation between 1 and 3 percent and therefore we need to form a view of where we currently are, how future developments might influence the path of output and inflation, and thus how we should respond.²

We use our forecasting to establish a clear and well-founded plan for what monetary policy settings are necessary to achieve our inflation target, to communicate this plan to the public and to support the effectiveness and transmission of monetary policy. We present our forecasts in the quarterly Monetary Policy Statement. A great deal of effort goes into producing these forecasts.

More often than not, the world does not turn out as we forecast. However, forecasting is still a valuable and necessary part of the monetary policy process. Producing forecasts and allowing them to be publically subject to challenge enables us to build a solid foundation for policy decisions, learn from developments, and improve our policy outcomes. Forecasting is not supposed to be prophecy; rather, it is about being precise about our thinking.

Uncertainty and the challenges for forecasters

A major challenge for forecasters is ‘radical uncertainty’, that is, we fundamentally don’t know what the future holds. Examples of this uncertainty are ‘black swan’ events that fall outside any existing experience. The term is derived from the northern hemisphere belief that all swans were white, based on the empirical evidence that every swan ever observed was white.³ For the first European migrants to Australia, discovering black swans was a completely unexpected event, lying outside of all evidence and experience up until that point.

For forecasters, such unpredictable events are a particular challenge. We have a wide range of information about the past and current state of the economy, and are setting policy to influence the future direction of the economy. But the economy is populated with thousands of households and businesses responding to their own particular circumstances and opportunities, so it is never certain how the economy will evolve over time. Even without black swan events, the chain reactions of economic activity are so complex and so changeable that the range of possible – let alone probable – outcomes is vast.

Tetlock and Gardner argue that the best forecasters understand that the world is uncertain and therefore react to it flexibly.⁴ Good forecasters are careful, curious, and comfortable with numbers, and, crucially, when the facts change they change their minds. And that is how we strive to be. Conversely, the worst forecasters remain dogmatically attached to their forecasts, unchanging in the face of the evidence.

Forecasting in uncertainty: dynamic programming and conditional forecasts

So how do we make policy for an uncertain future? The approach that the Bank takes is to develop forecasts that are highly conditional on the information currently available. That is, our forecasts are subject to revision when additional information comes to light. These forecasts are then used in what Alan Blinder would describe as a ‘dynamic programming’ plan. ⁵ This means thinking about what you want to achieve in the future and working backwards to form a plan today that will deliver that goal. In our case the goal is inflation near 2 percent in two or three years’ time, and the plan is given by the forecast of the Official Cash Rate (OCR). If events broadly unfold as envisaged then we carry out that plan, if not, then we adjust the plan.

A feature of our inflation forecasts are that they always end up at the mid-point of the target band. When planning our actions we necessarily plan to succeed in achieving this target. If we thought we would not achieve this we would modify our OCR forecasts.

As such, the forecasts show our judgement of the highest probability path to achieve the goal, but this path is not fixed. As new information becomes available, we assess it and where appropriate incorporate it into our understanding and our plans. Consequently, our policy settings are, and should be, responsive to any developments that might occur, and should move rapidly when required. For example, the sharp fall in inflation expectations in March 2016 (amongst other developments) led to us substantially revise our 90-day interest rate projection⁶ and move away from a neutral stance to an easing bias (figure 1).⁷

Figure 1: Inflation expectations and selected 90 day interest rate projections

The value of producing quantitative forecasts: internal discipline and the learning process

So why do we produce specific forecasts given that our forecasts are conditional and subject to radical uncertainty? Being numerically precise about our view of the future allows us to test ideas, which in turn accelerates our ability to learn and understand what is going on. Over time, this helps to ensure that our forecasts are as robust as possible and form a solid basis for monetary policy decisions, minimising potentially costly forecast errors and supporting good policy outcomes.

Testing is an integral part of robust analysis. As Nate Silver puts it, "The more eagerly we commit to scrutinising and testing our theories, the more readily we accept that our knowledge of the world is uncertain, the more willingly we acknowledge that perfect prediction is impossible, the less we will live in fear of our failures, and the more freedom we will have to let our minds flow freely." ⁸

When an outturn is different from that forecast, the Bank seeks to discover why, what it means for the outlook for monetary policy, and what we could do to make a better forecast next time.⁹ We use our depth of experience to examine the data, distinguish the signal from the noise, and identify where to adapt our approach or where to maintain our existing views. 

For example, residential investment at the end of 2016 was considerably weaker than the Bank had forecast. In seeking to understand this forecast error, the Bank considered the composition of residential investment. We found that a sharp fall in transfer costs (based on decreasing house sales) was the cause of weak residential investment, rather than weakness in actual construction activity. Accounting for this information should support better forecasts for this part of the economy in the future (figure 2).

Figure 2: Residential building activity and house sales

Producing forecasts: avoiding biases and supporting robust outcomes

Forecasting can also help reduce human biases that are often present in decision making. Our brains are designed to react to surprises by making sense of them. In seeking to do so, we often rewrite our perceptions from prior to these events.¹⁰ Even black swans may feel retrospectively predictable. This is known as ‘hindsight bias’ or the ‘knew it all along effect’.¹¹

As forecasters, we need to be particularly aware of this bias. To learn from our errors, we need to recognise that they are errors. By recording our quantitative forecasts and accompanying reasoning, we prevent ourselves from rewriting our original position and ensure we take on the full insights from any forecast errors we make. Over time the Bank has built an extensive institutional knowledge base of New Zealand economic data and experience of monetary policy formulation. Each time we forecast, we examine the latest data, identify where new information should lead us to change views, and incorporate lessons learned to ensure that our forecasts keep evolving and are as accurate as possible.¹² To use Kahneman’s terminology, our forecasts force us to think more slowly.

Forecasting is not the only essential part of monetary policy formulation. The Bank has a well-defined process for monetary policy decisions, based on a Monetary Policy Committee (MPC) that provides advice to the Governing Committee. ¹³ The MPC includes two external members to add to the Committee’s diversity of backgrounds and views. These external members provide a ‘reality check’ on the Bank’s thinking, and subject us to valuable independent scrutiny.¹⁴

The MPC structure is supported by a set of working practices that ensure that colleagues closest to the data and forecasting processes present their work to the MPC, and that subsequent discussions are broad-ranging and unconstrained in order to avoid groupthink or other biases.¹⁵ This means that the analytics of our Economics and Financial Markets Departments are blended with the experience of senior staff and Committee members, achieving better results through incorporating judgement in the forecasting process.

How do we create forecasts at the Reserve Bank?

The Bank monitors a wide range of data to gauge the current state of the economy, distilling the underlying trends and signals from the noise and assessing what the emerging picture means for the stance of monetary policy. In addition to formal model-based analysis, monetary policy formulation is supported by the feedback we receive from regular meetings with private businesses, industry associations and government agencies. We also undertake longer-term research projects to improve our understanding of specific features of the economy. ¹⁶

At the heart of our forecasting is our structural model, the New Zealand Structural Inflation Model (NZSIM). This is a theoretically rich framework that describes key behavioural relationships in the New Zealand economy and how they influence inflation. NZSIM generates the core forecasts for informing monetary policy decisions, and helps us identify the fundamental economic drivers and empirical relationships to explain the observed economic data.¹⁷

The information collated over the forecast round is fed into NZSIM, and the theoretical structure of the model ensures that our forecasts are consistent. For example, the national income accounting discipline is preserved, we can distinguish between domestic and imported inflation which sum up to CPI inflation, and we are able to trace the forces driving economic growth.

During the monetary policy decision-making process, the NZSIM forecasts are cross-checked with other modelling, and where appropriate modified to take into account additional information not directly represented in the model. That is, our forecasts are ‘model-assisted’ rather than ‘model-produced’.¹⁸ NZSIM also can consider alternative scenarios, meaning that we can test competing ideas and assess how monetary policy should respond if circumstances develop differently.

NZSIM necessarily abstracts from reality, and (as with any modelling) requires some difficult tradeoffs to be made, balancing the amount of detail that can be incorporated without sacrificing the ease of interpretation and resulting policy recommendations.  The combination of tractable model outputs and the expert judgement of Bank colleagues and Committee members forms the basis for building nuanced and detailed policy recommendations.

Over the last few decades, increases in computing power have enabled the Bank to vastly expand its modelling and research capability, including making NZSIM possible. However, one significant limiting factor to forecast improvement is data: macroeconomic statistics have not fundamentally changed over the 30 years we have been using inflation targeting.¹⁹ For instance, New Zealand is one of only two advanced countries that use a quarterly consumer price index rather than monthly. This is likely to be a fruitful avenue for future improvement that would greatly improve the Bank’s forecasting ability.

The benefits of publishing our forecasts: improving the effectiveness of monetary policy

The RBNZ Act²⁰ requires the Bank to publically set out its monetary policy stance and underlying reasoning, including a review of previous decisions. This information is published in our quarterly Monetary Policy Statement, along with our economic forecasts and accompanying explanation. The Bank was the first central bank to publish its planned interest rate path, starting in 1997, and we continue to rank as one of the most transparent central banks.²¹ Publishing these macroeconomic forecasts and our planned OCR path provides a clear ‘line in the sand’ as to the Bank’s thinking. It also helps synthesise our views in a more approachable format, enabling commentators, market participants, and others to understand the Bank’s monetary policy stance.

Communication is a key part of the monetary policy transmission mechanism as it helps people to understand the Bank’s thinking and expected policy direction.²² By building people’s understanding of how the Bank is likely to react to news, the predictability of monetary policy decisions is enhanced and policy uncertainty is reduced. We are a long way from the days when central bankers cultivated their ability to “mumble with great incoherence”.²³

To inform the market about the uncertainty inherent in projections, some central banks publish fan charts showing the range of possible outcomes around the highest probability scenario. While this approach shows the extent of uncertainty, it does not help people understand how the central bank’s forecasts may evolve if some of the contingencies change. Consequently, in recent Monetary Policy Statements we have published a range of potential scenarios to help people understand how the forecasts would change should the economy develop differently. We believe that this approach is potentially more helpful.

Figure 3 illustrates how market participants understand the way the Bank is likely to incorporate economic and financial information into its projections. The market revisions (red line) are generally good at anticipating changes to our 1-year ahead 90-day interest rate projection (blue line), which suggests that they understand the conditional nature of the forecasts and the Bank’s likely response (or ‘reaction function’) to new information.

Figure 3: MPS and market revisions

_{Source: RBNZ estimates.}

Case Study: the increase in the OCR in 2014

The increase in the OCR in the first half of 2014 provides a good case study of our conditional approach to forecasting in a ‘radically uncertain’ world. At the start of 2014 the global economy appeared to be recovering from the 2008/9 global financial crises, with the IMF asking whether the “tide was rising”. ²⁴ The New Zealand economy was growing strongly: the terms of trade had reached their highest level in 40 years; construction activity was robust; immigration was boosting housing and consumer demand; and monetary policy was providing substantial stimulus to the economy. Under these circumstances, we judged that there was a high probability that inflationary pressures were rising, contingent on the global and domestic economy developing as forecast and our knowledge of the structure of economic relationships. This view about rising inflation pressures was shared by the private sector forecasters and the IMF. Consequently, the OCR, which was set at an accommodative level of 2.5 percent, was raised by a total of 1.0 percentage points at successive OCR reviews in the first half of 2014.

However, by September 2014 it had become apparent that the forecast contingencies were not developing as expected. Firstly, uncertainty around the central scenario had risen: the global economy was not strengthening as anticipated and New Zealand terms of trade had fallen by about 7 percent since the start of 2014. Secondly, uncertainty about the structure of the domestic economic relationships was becoming apparent as CPI inflation remained moderate despite the inflationary pressures. Under these circumstances the Bank judged it appropriate to put a hold on interest rates, await the flow of new economic data, and investigate the evolution of domestic economic relationships.

In the event, this decision proved correct. Global economic growth continued to be sluggish through 2015 and 2016. The Bank’s analysis of the domestic economic relationships concluded that resource pressures were not as strong as previously estimated, in part due to the record level of migration that the New Zealand economy was experiencing.²⁵ Low inflation had also changed the price-setting behaviour of New Zealand businesses, who were placing a greater weight on recent low inflation (rather than expectations of future inflation) in their wage-setting and pricing decisions.²⁶ Incorporating these lessons into our forecasting framework meant that by mid-2015 the most probable scenario had changed to a continuation of low inflation (figure 4). The Bank commenced a new cycle of reducing the OCR, to an eventual low of 1.75 percent in November 2016. Today, in May 2017, the most likely scenario is for the OCR to remain stable for some time, although uncertainty remains high.

Figure 4: selected 90-day forward rates

 
_{Source: RBNZ estimates.}

Conclusion

Forecasting is a valuable part of the monetary policy process, helping the Bank to plan for the future, make well-founded policy decisions, communicate its understanding and intentions, and accommodate new information as it appears. It requires the Bank to be rigorous, unbiased, and open to new ideas in formulating and implementing monetary policy.

Because we operate in a world of radical uncertainty, we do not know what might happen in the future; we are not fortune tellers. However, because we want our forecasts to be relevant and useful, we present them numerically, an approach that comes with risks. As Tetlock and Gardner put it, “use the number and you risk being unfairly blamed. Stick with phrases as fuzzy as a puff of smoke and you are safe”.²⁷ We do not want to be safe in this context as we welcome scrutiny, both internally and externally, as it builds understanding and fosters accountability.

Finally, good forecasting happens when the numbers are supported by evidence and reasoning, and subject to public scrutiny. When things – inevitably – do not turn out as projected we seek to learn from those episodes and continually work to improve our forecasts and policy-making.

References