The projection process and accuracy of the RBNZ projections

Because of the lags between policy actions and inflation outcomes, economic projections play an important role in monetary policy formulation. For this reason, it is important that the Bank have a projection process that draws on all available sources of information, knowledge, and experience to provide a good assessment of the current economic situation, the likely outlook for the New Zealand economy, and the risks around that outlook.

Introduction

  1. This note describes the process by which the economic projections are generated within the Bank and how those projections perform relative to other forecasting institutions. The paper covers the type of information brought to bear on the assessment and policy decision on each occasion, and discusses how that information is processed in forming a view on the economic outlook and risks associated with it. This paper complements the paper "The monetary policy decision-making process", while the wider issue of whether projections should be published is discussed in "Publication of projections".
  1. The key points from this paper are:
  • The Bank generates its central economic projection using a two-step process to produce, first, a near-term assessment of the economy, and then, secondly, a medium-term outlook. This process allows us to provide, separately, for the changes in our view of the current situation, and for the medium- and long-term relationships within the economy.
  • In producing economic projections, the Bank draws as heavily on the knowledge and experience of internal and external sources as it does on economic models.
  • The projection published in the Monetary Policy Statement is just one of many components that contribute to the overall assessment and policy decision.
  • Comparisons of inflation forecasts (across all horizons) reveal that no other forecasting agency has consistently produced better inflation forecasts than the Bank. The Bank's forecasting of the real economy is similar, on average, to most external agencies.
  1. The remainder of the paper summarises the projection process at the Bank, the process of presenting and discussing the projections and policy issues within the forecasting team and at MPC, and the forecasting track record of the Bank relative to other New Zealand forecasting institutions.[1]

Forecasting

  1. While the projection published in the Monetary Policy Statement is just one component of the assessment and policy-formulation process, it is no doubt an important one. Generating a central economic projection and analysing the bounds of uncertainty around that projection is a two-step process, with each step drawing heavily on internal and external information sources, and economic models.
  1. The first step in the forecasting process is to provide analysis and interpretation of the current state of the economy, and to develop short-term projections for key economic variables. These projections provide the starting point for the Bank's medium-term projection of the economy. The second step is the development of the medium-term projection, through an iterative process with the short-term view.

Short-term forecasting

  1. Although policy actions are now seen as affecting inflation mostly one to two years from now (the "policy horizon"), it is still important to understand the current state of, and near-term outlook for, the economy. Short-term movements in activity can have important implications for the evolution of the economy over the medium-term. For example, a shock to business confidence now can affect real decisions, such as investment and employment intentions, and these, through normal lags, can affect demand and inflation pressure over the policy horizon.
  1. Moreover, as discussed in the paper "Publication of projections", our experience has been that a significant portion of the quarter-to-quarter change in policy settings reflected in our medium-term policy track is associated with changes in our view of the current situation of the economy.
  1. As a result, a very wide range of data from official and other sources and information and opinion from internal and external parties are analysed to understand the current state of the economy and its near-term direction.[2] Economic models and judgement are then used to provide estimates of activity and inflation for the current quarter and first quarter of the projection period. These two quarters are termed the "monitoring quarters".

Medium-term forecasting

  1. Once projections for the monitoring quarters are complete and assumptions about exogenous variables, such as domestic fiscal policy and trading partner output, inflation, and interest rates, are included, the medium-term projections are developed utilising a structural macro-model called the Forecasting and Policy System (FPS).[3]
  1. FPS is a useful tool for generating economic projections as it incorporates a long-run equilibrium, based on historical relationships and economic theory, and medium-term dynamics that allow the economy to develop and move towards the long-run equilibrium. In a sense, FPS is calibrated to "average" business cycle behaviour. However, FPS, like most macro-models, struggles to capture the short-term dynamics inherent in current economic data. To incorporate the information from current data and the important starting point assessment, FPS uses the projections of the monitoring quarters as if they were actual outturns.
  1. The first solution for the medium-term projection that initiates the process of iteration towards a satisfactory medium-term projection is produced using a minimum of judgement. However, as no model can capture all the forces at work in the economy, it is unlikely that the "no-judgement", mechanical projections, generated by FPS, would provide a good indication of the most likely outlook for the economy. Rather, the no-judgement projection provides an initial indication of the issues to be addressed over the projection round and a starting point to which judgement is added.
  1. To this end, the knowledge and experience of Bank economists and Governors play an important role in adjusting the no-judgement projection to get a more reasonable central projection.
  1. Adjustments to the no-judgement projection can account for special circumstances and information not included in the model (for example, house prices, bilateral exchange rates, and trading partner developments), and can incorporate circumstance-specific dynamics and known shocks (for example, tax cuts). In addition, as FPS reflects average business cycle behaviour, adjustments are made where the evidence suggests that the current cycle is different from the average economic cycle.
  1. Also, long-run equilibria, such as equilibrium exchange rates, may well be changing and, if ignored, pose risks to the projections and monetary policy decisions. Between recalibrations of the model (when the long-run relationships within the model are reviewed and adjusted if necessary), judgement can be added to the central projection to account for changes that are apparent in the long-run relationships.

Risk Assessment

  1. Uncertainty surrounds any forecast. The relevance of uncertainty in setting monetary policy is discussed in more detail in the paper "Monetary policy in an uncertain world".
  1. One of the benefits of FPS is that it allows the Reserve Bank to address uncertainty within a well-defined conceptual framework, and examine the key risks and judgements around any projection in a quantitative sense. Where a specific judgement surrounding a shock has been made, alternative assumptions can be made, and simulations can be run to assess their impact and the corresponding range of possible outcomes. Importantly, because FPS is a general-equilibrium model, the inclusion of external shocks, changes to judgements, or changes to relationships have a total, rather than partial, impact on the projection profile of the economy. That is, they affect the projection paths of all variables in the model rather than just those immediately affected by the adjusted assumption.
  1. Moreover, at present our model-driven projections feature endogenous monetary policy. See the papers "The monetary policy decision-making process" and "Publication of projections" for further discussion on the benefits of projections using endogenous monetary conditions vis-à-vis using exogenous projections. By adjusting assumptions and running different model simulations, the presence of endogenous monetary policy in the model framework produces a range of interest rate projections that could be chosen instead of the central projection, all of them consistent with the maintenance of price stability.

Communicating the projections within the Bank

  1. The formal projection process at the Reserve Bank spans around six weeks, from the initiation of the projection round to the final agreement of the central economic projection that is presented in the Monetary Policy Statement. See the paper "The monetary policy decision-making process" for a description of the full decision-making process.
  1. The communication of information, analysis, and assessment flows in many directions between staff in the Economics Department involved in producing the projections and the Monetary Policy Committee (MPC) (see the paper "The monetary policy decision-making process" for a description of the make-up of MPC). In addition, a number of meetings are held with other staff from the Economics, Banking System, and Financial Markets Departments to fully utilise the wider knowledge and experience in the Bank.
  1. The following is a summary and description of the papers in the order they are presented to MPC during the projection process. A timeline is attached in Appendix 1

Intra-quarter Update

  1. The intra-quarter update provides MPC with an update of data since the previous projections were completed. Although it is used primarily as the main source of information for the "interim" OCR review, forecasters take the opportunity in the paper to outline the key risks and policy issues that are likely to be addressed in the next forecast round.

Summary of information gained from Business Information Contacts

  1. Business consultations (BICs) provide the Bank with a direct link to the business sector. At a qualitative level, business consultations provide an indication of the current state of the business cycle and, to some extent, fill the lag between actual economic activity and the release of official data. A summary of the business talks is presented to MPC one to two weeks after they are completed, and to the Bank's Board at the completion and release of the Monetary Policy Statement. Appendix 2 explains in more detail the BIC process and lists the type of companies seen during business talks.

Data pack

  1. The aim of the data pack is to give MPC members background information to assist in the analysis and the testing of the projections to be presented later in the process. Information in the data pack includes:
  • Alternative "hawkish" and "dovish" scenarios written by four (nominated) members of MPC or the wider Economics and Financial Markets Departments (described in the paper "The monetary policy decision-making process"). These pieces are intended to tease out prior assumptions about the upcoming projections and policy decision, and to sketch out the range of possibilities before the formal projections and more systematic analyses are presented for discussion to MPC.
  • A summary of projections by external forecast agencies.[4] Comment, opinion and forecasts by external economic agencies are monitored closely. In particular, the assumptions and judgements underlying their medium-term projection profiles are analysed to ensure the Bank has not missed or ignored potentially important issues. Also, the Bank regularly hears private sector forecasters present their perspective on the current economic situation and medium-term outlook.
  • Graphs of key economic indicators.
  • A summary of MPC members' projections of key economic variables in quantitative form. The summary forecast survey requests from committee members their medium-term projections for key projection variables and the issues they think are pertinent to the upcoming round. The summary is intended to stimulate thought at the outset of the projection process and to inform the forecasting team of the issues that MPC members think deserve particular focus in the projection and analysis to be presented.

Primary sector update

  1. The primary sector update reviews the current situation and outlook for the primary sector. It includes information on and projections of production, prices, and climatic conditions.

World economic outlook

  1. Variables taken as exogenous to the FPS model need to be projected for the three years of the projection period. Examples of these variables are trading partner economic growth, and government income and expenditure. Where there are external agencies with more expertise than the Bank at producing forecasts for some variables, the Bank, in most cases, takes these forecasts directly from these outside agencies. For example, forecasts for trading partner growth, world inflation, and world interest rates are taken from international Consensus Economics, which provide projections, by international forecasting agencies, of our major trading partners. Appendix 2 explains in more detail how forecasts from Consensus Economics are used, and lists our major trading partners.

Presentation of the central projection

  1. Four papers discussing the central projection are presented to MPC. The presentation of the papers to MPC allows for a full discussion of the assumptions and judgements made around the starting point and central projection.

Current economic situation and near-term outlook

  1. This paper discusses developments in the data and indicators, and the implications for the current state of the economy and near-term macroeconomic outlook. The aim is to discuss the starting point uncertainties inherent in the projection and to reach consensus on the starting point for the medium-term projections – primarily the GDP growth projections for the near term.

Recent inflation developments and CPI components forecasts

  1. This paper discusses the near-term CPI projections, and information received since the previous projections on more general inflation trends. The projections are of CPI components rather than being based on behavioural relationships and therefore are not directly relevant for policy assessment unless a weakness in our understanding of the inflationary process is revealed. The aim is to reach agreement on the CPI inflation projections over the near term.

Medium-term projection

  1. After recapping on the near-term and world outlook, this paper presents the forecasting team's medium-term projection of the real economy, and the implied path for inflation and monetary conditions.

Risks and policy issues regarding the projections

  1. This paper discusses the risks and issues around the projection that have come up during the development of the central projection. These issues can then be incorporated into the deliberations around the immediate policy action, and into the content and tone of the Monetary Policy Statement.

Adjusted central projection

  1. No immediate decisions are made at the initial presentation of the projections. Rather, a "harvest" meeting is held the following day at which opinions can be canvassed and decisions made. This allows MPC members to digest the information from the previous day and assess the judgements behind the initial central projection. From the harvest meeting, decisions around the near- and medium-term projections are made, and adjustments to be made by the forecasting team are agreed upon. New projection runs are often commissioned and the adjusted projection is then presented in the "Adjusted Central Projection" paper.

Forecasting performance

  1. The two most important indicators that the Bank projects are CPI inflation and GDP (see Appendix 2). CPI inflation is important as it is the measure of inflation the Bank must target under the Policy Targets Agreement (PTA). GDP is important as it provides a measure of economic activity. When combined with an estimate of potential economic activity in the economy, GDP also provides an indication of the broad balance of supply and demand in the economy and the price pressures that may result.
  1. Analysis of the Reserve Bank's forecasting performance indicates that the size of forecasting errors increases over the projection horizon. For example, the Bank's short-term projections of annual inflation are typically within 0.3 percentage points of the annual inflation outturns, while projections of annual inflation one to two years ahead are typically within 0.7 percentage points of actual annual inflation outturns. Similarly, the Bank's projections of the annual average change in GDP one to two quarters ahead are typically within 0.5 percentage points of actual outturns, while projections of the annual average change in GDP one to two years ahead are typically within 0.8 percentage points of actual outturns.
  1. The Bank's forecasts can be compared with those of other forecasting agencies. Typically forecasts are compared using standard statistical measures such as the root mean squared error (RMSE), where a low value means better forecasting performance. Figures 1 and 2 provide such an evaluation of the Bank's forecast performance relative to twelve other (anonymous) forecasters, using available data from the end of 1994. Comparisons of inflation forecasts (across all horizons) reveal that no other forecasting agency has consistently produced better inflation forecasts than the Bank. And at short horizons (1 to 2 quarters ahead) the Reserve Bank is better at forecasting CPI inflation than any other forecasting agency. Comparisons of real GDP forecasts reveal that the Bank's forecasting performance is around the average of other forecasters over most horizons, although the difference between the best and the worst is not very large.

Figure 1

Evaluation of Inflation Forecasts Errors (RMSE)

Figure 2

Evaluation of Real GDP Forecasts Errors (RMSE)

  1. No matter how much effort goes into making projections, forecast errors will continue to occur because unforeseeable events (strikes, dramatic shifts in government policy, and financial crises) will continue to occur. While forecasting helps the Bank to identify and articulate various pressures that are at large in the economy, and thus forewarns us of emerging risks, they are not the totality of what goes into policy-making. The key is to understand the risks and be in a position to recognise when the world is changing. As discussed in the paper "Publication of projections", making policy on the basis of forecasts improves the ‘efficiency' of monetary policy compared with using only current information. Provided forecasting takes policy in the right direction more often than it takes policy in the wrong direction, it still pays to look forward. Thus the real litmus test of our forecasts is whether they assist us in carrying out our primary objective of controlling inflation. On this basis the Bank's forecasting system has performed well. Aided by its forecasting approach the Bank has been able to keep inflation close to or within the target range prescribed in the PTA (see figure 3).

Figure 3

Inflation as targeted by the Reserve Bank

(annual percentage change)

Appendix 1. Timeline of projection process

The complete projection process lasts around six weeks from the start of the business talks to the presentation of the adjusted central projection to MPC. The following timeline shows the main outputs for each week, and when the forecasting group presents papers to MPC.

Time

MPC Paper

Explanation

Week 1


Complete BIC discussions

Week 2

Intra-quarter update

Main information source for interim OCR review.

Week 3

BIC discussions

A summary of the key messages from the BIC round.

Week 4

Data pack

Chart pack, external agency forecasts, MPC priors and issues, key events since May, and hawk/dove scenarios from selected MPC members.


Primary sector update

An update of the current situation and outlook of primary sector output, weather, and prices.


World economic outlook

Presentation of the trading partner outlook from consensus forecasts.

Week 5

Presentation of forecasts

Current economic activity and near-term outlook

CPI components and near-term inflation outlook

Medium-term projection

Risks and policy issues



Harvest session



Adjusted central projection


Week 6

Drafting begins

Drafting of the Monetary Policy Statement begins

Appendix 2: Information sources used when projecting economic activity and prices

Data sources

In preparing short-term projections, the forecasting team relies on a wide range of data available from Statistics New Zealand and other organisations, including the New Zealand Institute of Economic Research, Quotable Value New Zealand, Real Estate Institute of New Zealand, various private bank surveys, National Institute of Water and Atmospheric Research, Ministry of Agriculture and Forestry, and the Meat and Wool Economic Service.

In total there are around 6000 series (a significant proportion are disaggregated components of aggregate data such as the CPI and GDP) available in the Reserve Bank's database. The following table outlines the most important data used, split into quarterly and monthly frequency. As a general rule, indicator data at the quarterly frequency are more important than those at the monthly frequency as they are less subject to ‘noise', making them more reliable indicators of economic activity and prices. However, in many cases, some of the quarterly data are released too late to be included into our formal projections, but are taken into account before a decision on the OCR is reached.

Table 1: Data sources

Monthly data

Quarterly data

Retail trade survey

CPI

Building consents issued

GDP

Merchandise trade values

Household labour force survey

REINZ house sales and median prices

Balance of payments

Food price index

Overseas trade indices

National Bank business outlook survey

Quarterly employment survey - wages

Colmar Brunton consumer confidence

Quotable Value New Zealand – house prices

External migration

Real building work put in place

Reserve Bank money and credit aggregates, credit card billings, and derived series

Real retail trade survey

ANZ commodity price index

New Zealand Institute of Economic Research quarterly survey of business opinion

Barfoot and Thomson house sales

Westpac-McDermot-Miller consumer confidence

Milkfat production

Producer price index

Livestock slaughter

Quarterly survey of manufacturing

Motor vehicle registrations

Capital goods price index

Cement sales


Steel sales


Hydro electricity generation


Thermal electricity generation


ANZ job ads


Road user charges


In addition to the data listed above, we also closely monitor data available on a daily basis, in particular domestic and international interest rates, exchange rates, and oil prices. Some data are also collected from newspaper reports, including council rates, tertiary fees, petrol prices, and electricity prices.

Business information contacts

Roughly six weeks before the publication of each Statement, staff from the Bank's forecasting team spend three or four days visiting senior executives of around 50 businesses and business organisations across the country. The sample of businesses (several hundred in total) is chosen to reflect broadly the different sectors and industries that make up the economy. Our sample includes large firms and industry leaders, as well as banks, Chambers of Commerce, and industry federations, who provide insight into how smaller firms or businesses in more fragmented industries are faring. Table 2 provides a summary of the number of firms visited by region and by industry over the past two years (some of these firms were visited more than once).

The aim of our business visits is to discuss current trends and the outlook over the next year or so, as they relate to individual firms. Discussions cover trends in sales, output, capital expenditure, employment, costs and pricing, as seen in recent months and as expected or planned over the coming year or so. Particular points of interest include whether recent outcomes have matched budget or expectation, and whether the factors driving prospects are specific to the businesses visited, or are a general feature of their industries.

These business visits add richness to the context within which the official data can be interpreted, and assist the Bank in discerning the relevant economic trends in those data for the purposes of forecasting and policy assessment. Our discussions also give the Bank a sense of how the business sector is responding to changes in economic conditions, including the impact of monetary conditions.

Table 2: Business contacts by region and by industry

Region

Number

Industry

Number

Auckland

120

Manufacturing

150

Wellington

Hamilton

70

20

Housing market and house construction

15

Palmerston North/Taranaki

Hawkes Bay

15

15

Non-residential construction

Communications

20

20

Tauranga

10

Transport

20

Christchurch

80

Tourism

20

Dunedin

30

Primary sector

40

Nelson

Invercargill

10

10

Retail & Wholesale trade

50



Importers/distributors

15



Accountants, Chambers of commerce, industry boards/reps, Govt. organisations

30

In addition to the business visits carried out by the forecasting team, other Reserve Bank staff visit major banks regularly to discuss issues such as credit conditions. Furthermore, the Governors of the Bank have regular contact with business people, including the Reserve Bank's Non-executive Directors, both formally and informally. Their information is then compared to the information gathered by the forecast team.

Information gained on business visits is also supplemented with regular monitoring of relevant media articles.

External agency forecasts

The Reserve Bank uses forecasts from Consensus Economics in forecasting world GDP, interest rates and CPI inflation. Consensus Economics surveys more than a dozen forecasting agencies (primarily financial institutions) and provides a central forecast by taking an arithmetic mean of the forecasts obtained from the survey respondents. The Reserve Bank's ‘world-GDP' forecast, for example, is then obtained by taking a weighted average of the central Consensus GDP forecasts for 14 different countries (listed in table 3). The weights are determined by the relative importance of each country as a trade destination for New Zealand exports. For example, Australia receives a much higher proportion of New Zealand exports than does France. Consequently, the Consensus forecast for Australia is given a higher weight than is the Consensus forecast for France in the Reserve Bank's forecast of ‘world-GDP'.

Although the Reserve Bank uses Consensus forecasts to guide our view of future international developments, the Bank will deviate from this central view when deemed appropriate. For example, during the Asian financial crisis the central Consensus forecast was deemed to be too optimistic, and so a more pessimistic subset of the survey forecasts was used by the Reserve Bank.

Table 3: Major trading partners

Country

Weight

Country

Weight

Australia

28.3

China

3.7

United States of America

18.3

Hong Kong

3.4

Japan

16.9

Indonesia

1.3

United Kingdom

8.2

Malaysia

2.5

Germany

3.5

South Korea

5.2

Italy

2.3

Taiwan

3.2

France

1.7

Thailand

1.4

We also collect a wide range of domestic agency projections in order to see where our projections are placed compared with the market. Their analysis also plays a role in testing the range of information we have considered and our interpretation of the data.

Appendix 3. Documentation on FPS and related forecasting systems

The following is a comprehensive list of papers by Bank staff and others on FPS and papers related to forecasting more generally.

I. FPS: The Core Model

Black, R, V Cassino, A Drew, E Hansen, B Hunt, D Rose, and A Scott (1997), "The Forecasting and Policy System: an introduction," Reserve Bank of New Zealand Bulletin, Vol 60, No 3.

Black, R, V Cassino, A Drew, E Hansen, B Hunt, D Rose, and A Scott (1997), "The Forecasting and Policy System: the core model," Reserve Bank of New Zealand Research Paper No 43.

II. FPS and Forecasting

Drew, A and M Frith (1998), "Forecasting at the Reserve Bank of New Zealand," Reserve Bank of New Zealand Bulletin, Vol 61, No 4.

Drew, A., and B Hunt (1998), "The Forecasting and Policy System: Preparing Economic Projections," Reserve Bank of New Zealand Discussion Paper G98/7.

McDermott, C J, and R St. Clair (2000), "The Reserve Bank of New Zealand's Model and its Role in Forecasting and Policy: A Review," Paper prepared for the Conference on Macroeconometric Modelling and Public Policy, Singapore, 2 February 2000.

Robertson, J (2000), "CentralBank Forecasting: An International Comparison Federal Reserve Bank of Atlanta Economic Review, second quarter 2000.

III. Output Gap Measurement

Claus, I, P Conway, and A Scott (2000), "The Output Gap: Measurement, Comparisons and Assessment," Reserve Bank of New Zealand Research Paper No 44.

Conway, P, and B Hunt (1997), "Estimating Potential Output: semi-structural approach," Reserve Bank of New Zealand Discussion Paper G97/9.

Conway, P, and B Hunt (1998), "Estimating Potential Output of the New Zealand Economy," Reserve Bank of New Zealand Bulletin, Vol 61, No 3.

IV. Policy Analysis and FPS

Conway, P, A Drew, B Hunt and A Scott (1999), "Exchange Rate Effects and Inflation Targeting in a Small Open Economy: A Stochastic Analysis Using FPS," Reserve Bank of New Zealand Discussion Paper G99/4.

Drew, A, and B Hunt (1998), "The Forecasting and Policy System: stochastic simulations of the core model," Reserve Bank of New Zealand Discussion Paper G98/6.

Drew, A, and A Orr (1999), "The Reserve Bank's role in the recent business cycle: actions and evolution," Reserve Bank of New Zealand Bulletin, Vol 62, No 1.

Drew, A, and B Hunt (2000), "Efficient simple policy rules and the implications of the potential output uncertainty," Journal of Economics and Business, Vol 52, 143-160.

Hunt, B (1999), "Inter-forecast monetary policy implementation: fixed-instrument versus MCI-based strategies," Reserve Bank of New Zealand Discussion Paper G99/1.

Ha, Y (2000), "Uncertainty about the length of the monetary policy transmission lag: implications for monetary policy," Reserve Bank of New Zealand Discussion Paper DP2000/01.

[1] A more complete discussion of the technical side of the projection process can be found in Drew, A and M Frith "Forecasting at the Reserve Bank of New Zealand," Reserve Bank of New Zealand Bulletin, Vol 61, No 4

[2] A more complete discussion on information sources can be found in Appendix 2

[3] Appendix 3 includes a comprehensive list of papers that provide additional information about the FPS model.

[4] External agencies whose projections are monitored include the National Bank, New Zealand Institute of Economic Research, WestpacTrust Bank, ANZ Bank, UBS Warburg, Bank of New Zealand, Ord Minnett, Merrill Lynch, Deutsche Bank, Infometrics, New Zealand Treasury, Credit Suisse First Boston, and BERL.