Data challenges in the monetary policy process

This paper discusses some of the issues the Bank faces in the area of monetary policy with respect to macroeconomic data.

We address the subject at two levels.

  • We discuss challenges arising from the changing structure of modern economies for our capacity to measure the economic concepts that are central to monetary policy, including not least, our capacity to measure the level of prices.
  • We identify more specific areas where we think data improvements may be feasible and would assist the Bank in formulating monetary policy.

Although compiling macroeconomic data is not without its challenges, and we see some areas where additional, well-targeted, resources could yield some useful data improvements, overall we consider that the data currently available for monetary policy are broadly adequate. There is, however, a need on the part of both data providers and users to be vigilant in ensuring the existing core statistical frameworks remain relevant in a rapidly-changing world.

Introduction

1. A separate paper "The projection process and accuracy of the RBNZ projections" addresses the categories and sources of information used by the Bank in its economic forecasting, and the forecasting processes used.

2. This paper discusses some of the challenges and issues we currently face with respect to data. It also seeks to provide some perspective on the issues raised by considering the inherent nature of macroeconomic data, and some of the implications for economic measurement of changes in the structure and character of modern economies.

3. Our overall assessment is that the data available for monetary policy purposes in New Zealand currently are broadly adequate. While there are, and always will be, areas where there could be more or better data, and we identify some of those areas later in this note, we do not consider monetary policy has been significantly handicapped by major data inadequacies. Nor are we suggesting that the improvements proposed will result in obviously better monetary policy. Better data can help in making better judgements at the margin, or in making the right judgement a little sooner, but it will always be the case that good monetary policy will be achieved mostly through analytical excellence. In other words, no matter how good the data, the big challenge for monetary policy-makers remains that of analysing and interpreting the data appropriately.

4. Perhaps as important as anything else regarding macroeconomic data is that the statistical frameworks being used should be kept under continuous review by data providers and users. Maintenance of good data requires a certain amount of on-going "replacement investment" in the statistical frameworks and data collection methods that are employed. This is especially the case when the economy is going through a period of structural change.

5. A consequence of revising data methodologies and sources to take account of structural change is breaks in data series. Hence, there is an inherent tension between maintaining the quality of current data, and maintaining continuity in data series over the long term. We do not propose that one dimension should prevail over the other, but rather that when structural breaks occur, the new data should be spliced with the old (or at least that new and old series should overlap).

6. We would like to record here our appreciation for the assistance and collaboration we receive from Statistics New Zealand and from numerous other government department and private sector data providers. We consider ourselves to have a good working relationship with Statistics New Zealand at all levels and, in the last few years, have worked collaboratively on some matters of common interest. These include participation in the CPI Review in 1997, and collaborative work on compiling data on the hedging of foreign currency debt and on New Zealand's International Investment Position. We see scope for on-going collaborative efforts of this nature.

Some contextual background

7. It is useful to recall at the outset that most of the macroeconomic data we work with are indicators of underlying economic concepts, and typically no more than estimates of those indicators. That is, the numbers we use generally are two steps removed from the underlying economic concepts we are interested in. In an important sense, macroeconomic data are always approximate measures of abstract concepts. This highlights that macroeconomics, including monetary policy, is a social science, not a physical science where more precise definition and measurement are possible.

8. In considering whether there is a need for better data, we need to consider two aspects of the question.

  • Do we need more appropriate indicators of, or proxies for, the underlying concepts we want to quantify? In other words, can the data being compiled be better aligned with the concept being measured, and more generally, with the analytical framework employed by data users?
  • Do we need better estimates of the indicators we already have? That is, can improvements be made in, for example, survey techniques, survey coverage, and response rates?

These are the "what" and "how" of macroeconomic statistics.

9. A complicating factor is that modern economies are evolving in ways that are making macroeconomic measurement ever more challenging. Most of the macroeconomic statistical frameworks we use were developed when primary and manufacturing production represented larger shares of aggregate output, and the production processes more obviously involved transformation of physical inputs into physical outputs - hence terminology such as the "volume" of production. By contrast, in modern economies today, the service sector, in which much output is intangible, dominates.

10. A related challenge arises from the fact that, increasingly, more output means better output, not increased quantities. For example, Federal Reserve Board Chairman Greenspan has suggested that US GDP today probably weighs no more than it did 100 years ago. This gives rise to difficulties in separating nominal variables into their price/real components, given that quality change has become such an important element of what is "real", and that quality change is difficult to measure. It follows that measuring the "general level of prices", and thus inflation, is getting no easier. These measurement uncertainties can be material when inflation is at a very low level.

11. Similar points apply to the measurement of the capital stock (given that, increasingly, it is human and IT capital that matters) and the balance of payments (globalisation of business is making it more difficult to compile economic data on the basis of residency concepts). And if the capital stock and the balance of payments are difficult to measure, then so too are national investment and saving.

12. There are also well-known difficulties in measuring financial quantities. In a deregulated financial system, and with global financial markets, it is no longer easy to fit everything into the neat boxes that make up the traditional monetary aggregate accounting frameworks.

13. Another - more New Zealand-specific - factor to be borne in mind when considering data requirements for monetary policy is the size and nature of the New Zealand economy. New Zealand is a small and in some respects un-diversified economy, such that firm/project/event/sector-specific factors can impact on the macro data. But New Zealand is also too large for policy-makers to be able to form a macro view from micro observation. In other words, the macro data are inherently volatile, and understanding what they indicate calls for a good degree of familiarity with the economy's sector-specific idiosyncrasies. But, equally, care is needed to avoid macro assessments coming to be overly influenced by those idiosyncrasies.

14. Despite these challenges the System of National Accounts as documented in the SNA93 manual remains in our view the most relevant core statistical framework for economic analysis. We note the work done by Statistics New Zealand to adapt New Zealand's accounts to meet current international standards.

Some principles

15. Against the background sketched, what attributes are we looking for in macroeconomic data?

16. First and foremost, macroeconomic data needs to be relevant for the purpose. By this we mean that the measurement frameworks developed by statisticians should align reasonably well with the analytical frameworks employed by the data users. That, in turn, suggests that both analytical and statistical frameworks should be well-grounded on conceptual foundations, and that there should be a philosophy of "economic substance" over "accounting form". The importance of these principles is underscored by the nature of the structural and other change that is taking place in all modern economies. There is a risk that statistical frameworks that are not strongly focussed on delivering quantification of the substantive concepts that policy-makers work with end up delivering data that may be accurate, but of diminishing relevance.

17. Second, data should represent what they purport to represent. In some respects, this is another way of expressing the "substance over form" point just made. But it goes beyond that. Getting the conceptual framework right is one thing. But equally important is that the data collected in the field accurately reflect the concepts that the data seek to quantify.

18. Data also need to be available with sufficient frequency. Most New Zealand macroeconomic data are available either monthly or quarterly, and we think the existing frequency is appropriate in most if not all cases. We are conscious that some New Zealand macroeconomic data are available with less frequency than in some larger countries, for example, New Zealand has a quarterly rather than monthly CPI. But, as a general observation, we do not think that moving to higher frequency data would necessarily result in our having better data. The small size and still significant primary sector orientation of the New Zealand economy mean that New Zealand macroeconomic data is subject to a degree of inherent short-term volatility. This being the case, it is possible that the "noise to signal" ratio in higher frequency data would be quite high.

19. Data also need to be timely to be useful. However, generally, more timely data will not be useful - and may even be unhelpful - if achieving timeliness means materially compromising quality in terms of paras 16 and 17. What is needed is good quality data that are also timely.

20. But it is acknowledged that there will always be some trade-offs between quality and timeliness. As monetary policy decision-making occurs mostly in a medium-term framework, whether data are available a few weeks sooner or later generally is a second-order issue relative to data quality. Marginal improvements in timeliness of accurate data can result in marginal benefits for the Bank, but early release of inaccurate data can be positively unhelpful.

21. This raises the issue of data revisions. These, on occasion, have been material, and in some cases have had a bearing on policy judgements, albeit at the margin. For example, on three occasions in the three years 1994 to 1996, final GDP quarterly figures varied from the initial estimate by more than 0.5 percent. While a desirable rebase of the series accounted for much of this result, revisions of that magnitude mean that policy judgements made on the basis of the initially available data turn out to have been less than ideal.

22. For much data, there will be a tension between achieving timeliness of release, and ensuring that the data released are complete and accurate. The former objective can sometimes be furthered by release of provisional data (which are subject to revision), while the latter puts a premium on data being released only when quality standards have been satisfied. We do not think there is any single, hard and fast, rule for determining how this trade-off should be managed, other than that:

a) the longer the delay until final data are available, the more desirable it is to have provisional data; but
b) the larger are the revisions to provisional data, the more desirable it is for release to await the availability of final data.

We are aware that Statistics New Zealand has in mind the development of criteria for managing these trade-offs and we welcome that initiative.

23. Where the timeliness of data availability can be a particular issue is where the lags are long, and in scheduling Monetary Policy Statements and Official Cash Rate reviews. However, the issue in the latter case is more one of presentation than substance. On occasion, significant data become available (or are revised) shortly after monetary policy judgements have to be made and announced, and that can make those policy judgements appear less than fully informed. However, this problem is largely inherent where economic data are released progressively as they become available, as they should be, rather than being held back for, say, end-of-month release dates. Moreover, in most cases, single pieces of new information will influence our policy judgement only at the margin, and then there is only, generally at most, a six-week lag before new information is factored into policy (given that the policy interest rate is reviewed about every six weeks).

24. Nonetheless, from the stand-point of presentation, it is not ideal that important data should be released shortly following a Monetary Policy Statement (MPS) or OCR review. For this reason, the Bank endeavours to schedule MPS and OCR review dates to follow important data releases. Statistics New Zealand's practice of publishing a forward calendar of data releases is particularly helpful in the latter regard. Given that the Bank also publishes a forward schedule of its MPS and OCR release dates, generally for about a year ahead, and faces market constraints in changing dates that have already been announced, it is important to the Bank that the data release calendar is adhered to.

Some specific issues concerning data availability and quality

25. The Bank's core data needs comprise data series indicative of the level of activity/aggregate demand in the economy, and of price level developments. Our work-horse analytical framework is the national income/expenditure accounting framework. Thus our key needs are for good quality real and nominal quarterly national expenditure and income (as well as production) data, and good quality measures of price level developments.

National accounts data

26. Relative to our need for expenditure and income data in our analytical frameworks, it is the production-based national income accounts which are currently regarded as best quality. The quarterly expenditure-based data are more volatile and viewed as of lesser quality, while quarterly real and nominal income data, in a national income and expenditure accounting framework, are not available. From the Reserve Bank's standpoint, improvements in the quality of the expenditure GDP data, and the development of quarterly national income data, would be particularly welcome.

27. In so far as the weaknesses in New Zealand's expenditure GDP data reflect weaknesses in expenditure source data, that would reinforce our view on the desirability of strengthening the measurement of expenditure GDP. The various monthly and quarterly expenditure indicator series, which comprise the source data for expenditure GDP, are central to our near-term tracking of the level of activity/demand in the economy. The more traditional expenditure indicators (eg, for retail sales, building permits issued, and building work put in place) remain important in this regard. However, the services sector of the economy is less well covered by comparable indicators. Also, some categories of investment, such as in communications infrastructure, and in the information processing sector, are not well covered. There is a question whether indicators of investment in buildings and plant and machinery any longer provide adequate coverage of investment expenditure.

28. Perhaps the aspect of the New Zealand economy, in a national income/expenditure/production accounting framework, for which data are most sketchy, is income data. Comprehensive national income data are available only on an annual basis (and then with a lag of 6-7 months). While higher-frequency data are available on labour costs, few such data, other than what are available anecdotally and from business opinion surveys, are available on business/entrepreneurial incomes and margins. This represents a material gap in our information base, in so far as information on pressures on business income margins can be informative with respect to likely price-setting behaviours.

Other income data

29. Another question concerns whether official data on wages and salaries (Quarterly Employment Survey data and Labour Cost Index data) have adequately kept pace with structural changes in the labour market over the last decade or so. The structural changes include contracting out work that was previously performed by employees, more flexible working hours, and structural changes in (and perhaps less) overtime remuneration. All these developments will have had implications for how labour costs need to be measured, as well as for how income is classified in the national accounts as between "compensation of employees" and "operating surplus". The question of what is, and is not, being measured, and how it is being measured, might benefit from review by data providers and users.

Price level data

30. In its submission to the most recent CPI Review Committee, the Bank recommended, inter alia, that (a) the interest element in consumer credit service charges (notably residential mortgage interest costs) should be removed from the CPI regimen and (b) new house construction costs/purchase prices and section prices should similarly be removed. It was recommended by the Bank that these items in the CPI regimen should be replaced by a rental equivalence measure of housing occupancy costs. The Committee agreed to (a) and to the removal of section prices, but not to the removal of the cost of new housing in favour of a rental equivalence approach. Instead it was concluded that a separate, supplementary, CPI which measures the cost of housing occupancy on a rental equivalence basis should be developed.

31. We understand that development of the alternative, rental equivalence-based, measure has not yet occurred, and accordingly have taken this opportunity to review the matter. We have concluded that the removal of residential mortgage interests costs and section prices from the CPI regimen, which took effect from September 1999, substantially addresses our previous concerns, and that taking the extra step to construct a separate CPI, with housing occupancy costs based on a rental equivalence measure, may yield only marginal benefits. Accordingly, construction of such an index is not now seen by the Bank as a current priority.

32. A higher priority issue, in the Bank's view, is that Statistics New Zealand continue to actively manage potential sources of bias in the CPI. These potential sources of bias require methods of treatment that fully reflect the improvements in the quality of consumer goods and services, and respond to changes in the pattern of retail distribution in a timely manner. In an increasingly complex economy it is important that Statistics New Zealand's practices reflect international developments and make effective use of available information.

33. The degree of stability in New Zealand's terms of trade data since the early 1990's is surprising. In light of this, there have been some questions about whether Customs-based export and import price data are best. Import or export categories can be quite broad, so they will often include goods of different sizes or quality. This scope for compositional change will make the Customs-based import and export price data prone to distortion, as quarter-on-quarter price changes for a category may be more the result of a different mix of goods than price change. In the Overseas Trade Price indices, prices for categories that are not sufficiently narrow to allow calculation of meaningful unit-value price indicators are imputed from those that are. This results in high levels of imputation of price movements that may not be appropriate. We understand that the Australian Bureau of Statistics uses its own (survey) sources, and regimen, at least for its import price index, and that Statistics New Zealand is planning to supplement the Import and Export Price Indices with surveyed prices where customs-based price indicators for imports and exports are not appropriate.

34. To gauge inflation, we should be looking at a battery of price and cost measures. We generally have the data to do this: consumer prices, producer (input and output) prices, labour costs, capital goods prices, real estate (residential, commercial and rural) prices to name some. We are also generally satisfied with the quality and timeliness of these data. However, there may be scope to align the various measures of prices more closely, to facilitate the tracing of cost and price adjustments through the various phases of the production process. Also, we have already mentioned labour cost data as an area for possible attention. We see these data as important given that labour costs represent perhaps the single largest element underpinning "non-tradable" costs and prices.

35. We further observe that the GDP deflator in New Zealand, perhaps the most comprehensive single measure of the general level prices that we have, is little referred to and little used. There seems to be a perception (whether justified or not) that this measure of the price level is not of high quality. The fact that the GDP deflator becomes available only quite some time after most other price level data may be another factor.

Financial sector data

36. The Bank has made a significant commitment in recent years to improving the quality of financial sector data. The work is on-going, but has already resulted in new and revised data series, principally focused on borrowing and lending by the household sector. This priority has reflected the relevance of trends in household borrowing and lending to understanding aspects of New Zealand macro balances, including, for example, pressures in the housing market, the low level of saving, the current account deficit, and the financing of that deficit (much of which has been intermediated through the banking system).

Business and consumer opinion survey data

37. The Bank draws on information from many sources beyond official data sources. Particularly significant are the business and consumer opinion data published by a range of private organisations. Information on business and consumer expectations and opinion are used by many central banks and policy- makers to help them assess the current and near-term prospective position of the economy in the business cycle. These data, though often thought of as "soft" data, are becoming more important to policy-makers as the measurement challenges in compiling "hard" data increase.

38. Just as the "hard" data that measure actual out-turns (for expenditure, incomes, production and prices) need to be subject to on-going critical evaluation in the light of structural change in the economy, so too do the surveys from which expectation and opinion data are sourced. For example, central in some business opinion surveys are questions on capacity utilisation. An issue here concerns whether changes in management and production processes (eg, more efficient stock management techniques, multi-shifting, contracting out) mean that firms' conception of what constitutes "normal capacity" is different today from what it was a decade ago. Similarly, does one need to think about how to measure inflation expectations differently when inflation is low and reasonably well anchored, from when inflation was persistently at a double-digit level? If at very low rates of inflation most observed movements in the CPI reflect relative price movements, do we need to think of better ways to gauge inflation expectations than merely asking respondents for their forecast of the movement in the CPI?

External data

39. While balance of payments data are generally not viewed as being as critical for monetary policy as the data considered so far, they do provide important information on the overall macro (income/expenditure) imbalances in the economy. In turn, how those imbalances get resolved can have implications for the price level over the longer term, notably as the result of exchange rate adjustment. These aspects of New Zealand's current macroeconomic position are the focus of particularly close attention at this time.

40. For these reasons, we think balance of payments data are important. It is also an area where the measurement challenges are no less than elsewhere. We list some specific questions concerning the balance of payments data here.

  • Is the coverage and timing of the Customs data for merchandise exports and imports adequate? The coverage issue arises given the $1000 threshold for Customs documentation, with Internet shopping and use of couriers and airfreight to deliver smaller customised orders growing rapidly. Delays in processing export consignments can introduce volatility to monthly trade statistics. There is also difficulty in valuing exports and imports like computer software, which is a good with a service component. We would encourage Statistics New Zealand to address these issues.
  • Are data on tourism receipts and payments being captured adequately? By some classifications, tourism is now New Zealand's single largest export industry.
  • Are imports and exports of non-tourism services being adequately captured? It seems likely that international trade in such services (for example, consultancy services, software development, and financial services, to name some) will become relatively more important over time.
  • Is the investment income account information being captured as well as it could be? We are aware that this is a particularly difficult area, given that cross-border financing relationships are not as simple as they used to be, the complexity of multinational corporate structures, and possibly also a diversity of accounting practices used by the respondents who provide the source information. We are also conscious of the likelihood that investment income earned directly by New Zealand residents offshore may not be captured in the investment income account.