The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.
Updated Historical Forecast Errors (4/9/2014)1 Table 2 of the FOMC’s Summary of Economic Projections (SEP) reports average historical projection error ranges. These figures are the average of the root mean squared forecast error (RMSE) of six public and private forecasters over the past 20 years, as described in more detail in Reifschneider and Tulip (2007). 2 Our latest estimates are shown in table 1 of this memo. In the first quarter of each year, we update these estimates by rolling forward the sample one year. In 2014, we also updated our procedures to assess the performance of the GDP growth forecasts against the latest available BEA estimates, adjusted for the effect of major methodological revisions, instead of against the BEA’s third estimates. Greater detail on this change is provided in the appendix. 3 Updating the sample period—from 1993–2012 to 1994–2013—did not have much effect on the RMSEs. However, the change in our performance benchmark for GDP growth increased the RMSEs of those forecasts, as shown in the top panel of figure 1. 1 Jeremy Nalewaik and Adam Scherling updated the errors and the methodology in 2014. David Reifschneider and Peter Tulip (2007), “Gauging the Uncertainty of the Economic Outlook from Historical Forecasting Errors,” Finance and Economics Discussion Series 2007-60 (Washington: Board of Governors of the Federal Reserve System, November), www.federalreserve.gov/pubs/feds/2007/200760/200760abs.html. 3 We have made some other minor changes to our database of forecast errors since Reifschneider and Tulip (2007). For example, in 2013, the timing of the SEP forecasts shifted from the January, April, June, and November forecast rounds to the March, June, September, and December forecast rounds, and we modified our databases that year to align the timing of the forecasts as closely as possible to the new SEP timing. 2 -2Table 1: Root Mean Square Prediction Errors; 1994-2013 A. Real GDP growth (Q4/Q4) – Latest GDP Estimates ex. methodological changes Projection Year Current year Second Third Fourth Spring 1.6 2.1 2.0 -- Summer 1.4 2.0 2.1 -- Fall 1.3 1.9 2.1 2.1 Winter 0.9 1.8 2.1 2.1 B. Unemployment Rate (Q4) Projection Year Current year Second Third Fourth Spring 0.6 1.2 1.7 -- Summer 0.4 1.2 1.8 -- Fall 0.3 1.0 1.6 1.9 Winter 0.1 0.8 1.4 1.8 C. Consumer Price Index (Q4/Q4) Projection Year Current year Second Third Fourth Spring 0.9 1.0 1.1 -- Summer 0.7 1.0 1.0 -- Fall 0.8 1.0 1.1 1.0 Winter 0.2 0.9 1.0 1.0 -3- -4Appendix A: Using Latest GDP (excluding methodological changes) to Construct Forecast Errors Previously, table 2 of the SEP assessed forecasts of Q4/Q4 real GDP growth against a time series of the BEA’s third GDP growth estimates (i.e. the Q4/Q4 growth estimate for each year that is released in late March of the subsequent year). 4 However, those third estimates are based on source data that are incomplete, while the most up-to-date BEA estimates incorporate more complete annual survey data that are presumably more accurate. The purpose of the forecast error table is to give a sense of the accuracy of forecasts evaluated against the best available measure of output growth, so using the better-measured latest time series should provide a more pertinent uncertainty measure. The magnitude of the revisions from the third estimates to the latest estimates can be nontrivial. For instance, the third BEA estimate of real GDP growth over the four quarters of 2008 has revised from minus 0.8 percent to a much more substantial decline of 2.8 percent in the latest estimates. More generally, the third estimates ended up revising by 0.7 percentage point or more in 8 out of the 16 years from 1994 to 2009 (see the first two columns of table A1 below). On the other hand, while the latest real GDP growth time series is based on a consistent methodology, different methodologies have been employed in the past, with forecasters trying to predict GDP growth as measured by the methodology that prevailed at the time of their projections. For example, forecasters were predicting fixed-weight GDP prior to 1995, GDP ex-software investment prior to 1999, and GDP ex-investment in other intangibles prior to this year, so it does not seem appropriate to count the effect of those methodological changes on GDP growth as a forecast error because forecasters likely would have made different predictions had the current methodologies been in place back then. These three methodological revisions likely had the largest effect on Q4/Q4 GDP growth over the past 20 years, and, fortunately, their effect can be estimated and removed 4 Since we now update these RMSEs in early March rather than April, the last GDP estimate in the sample is the second rather than the third estimate. -5for the relevant years. The effects of these methodological revisions on GDP growth were generally small (see table A1). 5 Table A1 (Estimates of Q4/Q4 Real GDP Growth): Year 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 5 GDP (3rd) GDP (latest) 4.1 1.3 3.1 3.7 4.3 4.6 3.4 0.5 2.9 4.3 3.9 3.2 3.1 2.5 -0.8 0.1 2.8 1.6 1.6 2.5 4.1 2.3 4.5 4.4 5.0 4.9 2.9 0.2 2.0 4.3 3.1 3.0 2.4 1.9 -2.8 -0.2 2.8 2.0 2.0 2.5 Effect of Methodological revisions GDP (latest, ex meth. rev.) Intangibles Software Chain-wgt 0.1 0.1 -0.1 4.0 0.1 0.1 2.0 0.2 0.2 4.1 0.1 0.3 4.0 0.1 0.2 4.6 0.1 4.7 0.1 2.8 0.0 0.2 0.0 2.0 0.0 4.3 0.1 3.0 0.1 3.0 0.2 2.3 0.1 1.8 0.0 -2.8 0.0 -0.2 0.1 2.7 0.0 2.0 0.0 1.9 2.5 The effect of including software as a component of business investment (rather than as an intermediate input) is its contribution to real GDP growth, estimated as real software growth weighted by its lagged share of nominal GDP. The same approach is taken for investment in other intangibles such as R&D. The effect of the switch to chain weighting in the 1995 benchmark revision is approximated as the effect of that benchmark revision on the GDP price deflator. -6The three panels in Table A2 show root mean square prediction errors for the Q4/Q4 growth rates of real GDP, computed using the third BEA estimates, the latest BEA estimates, and the latest BEA estimates excluding the effect of the methodological revisions described above. As indicated by the similarity of the estimates in panels E and F, stripping out the effect of methodological changes does not matter very much. In contrast, the RMSEs increase substantially when moving from the third to the latest estimates, particularly for the current year forecasts. In particular, the RMSE of the current year forecasts made in winter roughly doubles when making this switch from third to latest. As such, we decided to switch to the latest real GDP growth ex. methodological adjustments time series in computing RMSE. Going forward, we plan on continuing to make adjustments for major methodological changes as the BEA implements them. For the unemployment rate and CPI inflation, we had previously evaluated forecasts against prior-year estimates pulled from April/May Tealbooks, and, similar to GDP, we are now switching to use the latest estimates. For these variables, the switch has virtually no effect on the RMSEs, as revisions to these measures have been small. (Seasonal factors do revise but these obviously do not have much effect on Q4/Q4 CPI inflation, and they do not seem to have much effect on the Q4 level of the unemployment rate, either.) Furthermore, when methodological changes have been made to the CPI and the unemployment rate in the past, the BLS generally has not revised previous years to be consistent with the new methodology, so we do not need to make any methodological adjustments as we did for real GDP. -7Table A2: Root Mean Square Prediction Errors; 1994-2013 D. Real GDP growth (Q4/Q4) – Third GDP Estimates Projection Year Current year Second Third Fourth Spring 1.3 1.7 1.9 -- Summer 1.0 1.6 1.8 -- Fall 0.8 1.6 1.8 2.0 Winter 0.5 1.4 1.8 1.8 E. Real GDP growth (Q4/Q4) – Latest GDP Estimates Projection Year Current year Second Third Fourth Spring 1.7 2.1 2.1 -- Summer 1.5 2.0 2.2 -- Fall 1.4 2.0 2.2 2.2 Winter 1.0 1.8 2.2 2.1 F. Real GDP growth (Q4/Q4) – Latest GDP Estimates ex. methodological changes Projection Year Current year Second Third Fourth Spring 1.6 2.1 2.0 -- Summer 1.4 2.0 2.1 -- Fall 1.3 1.9 2.1 2.1 Winter 0.9 1.8 2.1 2.1
@FedFRASER