The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.
Search MACROBLOG December 5, 2016 Using Judgment in Forecasting: Does It Matter? Many professional forecasters use statistical models when making their near-term projections for real gross domestic product (GDP) growth. A 2013 special survey on the forecasting methods of the Survey of Professional Forecasters found that 18 out of 21 respondents featured a statistical model prominently in their current-quarter economic projections. Nevertheless, there is fairly compelling evidence that many professional forecasters incorporate judgment in their forecasts of the first estimate of real GDP growth for a quarter—even when much of the source data used to construct the GDP estimate are available. In the October 2016 Wall Street Journal Economic Forecasting Survey (WSJ), the most common panelist projection for annualized third-quarter real GDP growth was 2.5 percent, and the second most common one was 3.0 percent. The first digit after the decimal point, or tenths digit, of these two numbers are "5" and "0." Of the 58 individual forecasts of third-quarter growth in the survey, 21 had a tenths digit of "0" or "5," a total that is almost twice as large as we would expect if all tenths digits were equally likely to be submitted. This pattern isn't unique to the most recent quarter's GDP forecast. The following chart shows the historical frequency of the tenths digit in past WSJ surveys for first estimates of real GDP growth over the period from the first quarter of 2003 to the third quarter of 2016, made about three weeks before the release. Almost 40 percent of these 2,390 forecasts have a tenths digit of "0" or "5." In contrast, the historical distribution of published first estimates of real GDP growth from the fourth quarter of 1991 to the third quarter of 2016 and real gross national product (the most common measure of U.S. production in an earlier era) growth from the third quarter of 1965 to the third quarter of 1991 has a tenths digit of either "0" or "5" only 18 percent of the time. The historical Atlanta Fed's GDPNow forecasts have a "0" or a "5" tenths digit only 15 percent of the time. More formally, one easily can reject the hypothesis at the 1 percent significance level that the tenths digit of the WSJ panelist forecasts are either uniformly distributed or follow the Benford distribution for tenths digits after rounding to the nearest tenth (see this paper by economists Stefan Gunnel and Karl-Heinz Todter , who found similar relative frequencies of "0s" and "5s" in A A A professional forecasts of German GDP growth and consumer price index inflation). If we assume that near-term GDP growth forecasts with a tenths digit of "0" or "5" typically involve more judgment than forecasts with another tenths digit, a natural question is whether these more judgmental forecasts are less accurate than others. Of the 2,390 WSJ growth forecasts mentioned above, the ones with a tenths digit of "0" or "5" (after rounding to the nearest tenth) had an average error of 0.786 percentage points without regard to sign, and the others had an average error of 0.743 percentage points. These accuracy metrics are not statistically different at even the 10 percent significance level. Moreover, because of the panel nature of WSJ forecasts, we can measure how often a forecaster has a tenths digit of "0" or "5" (after rounding). Of the 44 panelists who submitted at least 30 three-week-ahead GDP forecasts during the period of the first quarter of 2003 through the third quarter of 2016, the correlation of the panelists "0" or "5" tenth digit frequency and their average error without regard to sign is only 0.13 and not significantly different from 0. Although at least some professional forecasters appear to make judgmental adjustments to their near-term GDP projections, the evidence presented here does not suggest it comes, on average, at the cost of accuracy. By Pat Higgins, an associate policy adviser and economist in the Atlanta Fed's research department January 7, 2016 in Education, Employment, Labor Markets | Permalink
@FedFRASER