The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.
A Funny Thing Happened on the Way to the Data Bank Dean Croushore and Tom Stark Understanding Asset Values: Stock Prices, Exchange Rates, And the “Peso Problem” Keith Sill S ometimes, the present depends on the future: people carry umbrellas when there is a forecast for stormy weather; football teams in the lead play zone defense late in the game, since they expect their opponents to pass; advancepurchase airfares are higher for holiday-travel times, when passenger traffic is expected to be heavy. In each of these cases, and many others *Keith Sill is a senior economist in the Research Department of the Philadelphia Fed. we can think of, what people expect will happen affects how they behave today. Exchange rates and prices of assets such as stocks and bonds depend not only on the most likely future outcomes but also on possible but less likely outcomes. Sometimes a possible outcome can be so different from today’s conditions that asset prices, which incorporate such extreme possibilities, make financial markets look flawed, even if they are not. Economists call such a condition a “peso problem.” No one knows the precise origin of the term 3 BUSINESS REVIEW peso problem, but it is often attributed to Nobel laureate Milton Friedman in comments he made about the Mexican peso market of the early 1970s. At that time, the exchange rate between the U.S. dollar and Mexican peso was fixed, as it had been since 1954. At the same time, the interest rate on Mexican bank deposits exceeded the interest rate on comparable U.S. bank deposits. This situation might seem like a flaw in the financial markets, since investors could borrow at the low interest rate in the United States, convert dollars into pesos, deposit the money in Mexico and earn a higher interest rate, then convert the proceeds back into dollars at the same exchange rate and pay off their borrowings, making a tidy profit. Friedman noted that the interest rate differential between Mexico and the United States must have reflected financial market concerns that the peso would be devalued. Otherwise, the interest-rate differential would soon disappear as investors increasingly took advantage of it. In August 1976, those concerns were justified when the peso was allowed to float against the dollar and its value fell 46 percent. The difference in return on comparable U.S. and Mexican assets—which looked like an anomaly to analysts who thought the exchange rate would remain fixed because it had been fixed for 20 years—could be explained once investors’ recognition of the possibility of a large drop in the value of the peso was factored in. More generally, peso problems can arise when the possibility that some infrequent or unprecedented event may occur affects asset prices. The event must be difficult, perhaps even impossible, to accurately predict using economic history. Peso problems present a serious difficulty for economists who like to build and estimate models of the economy and financial markets, then use them to interpret economic data. Empirical economic models are designed to match features of the economy. They are calibrated or estimated using current and historical data on economic variables.1 If the historical data used to calibrate or estimate models do not accurately reflect the 4 SEPTEMBER/OCTOBER 2000 probabilities of bad (or good!) things happening, model-based forecasts can prove inaccurate and the policy advice that rests on them can suffer. PESO PROBLEMS, ECONOMIC FORECASTS, AND EXPECTATIONS Expectations are often an important ingredient in our everyday actions and decision-making. For example, grocery stores become more crowded than usual when the weather forecast calls for a severe snowstorm. Firms may make additional investments in plant and equipment today in order to meet projections of strong future demand for their products. In the financial realm, prospects for variables like economic growth and inflation help determine asset prices and exchange rates. The most useful forecasts give the best approximations of what actually ends up happening in the economy. We judge the “goodness” of forecasts by the properties of their forecast errors, which are the differences between a sequence of forecasted values of a variable and its actual, or “realized,” values. Good forecasts have forecast errors that are zero, on average. If forecast errors aren’t zero on average, the forecast is biased: the forecast is more often too high than too low, or vice versa. The presence of bias means that the forecaster is repeatedly making the same mistake, a mistake we would expect to be eliminated as the forecaster learns from his past misses. Good forecasts also have errors that aren’t predictable. If they were, the forecast could obviously be improved by correcting those predictable errors. 1 Calibrating an economic model is a two-step process. First, the economist must construct a set of measurements on economic variables that are consistent with the variables that appear in the model. Second, values must be assigned to the model’s parameters so that the behavior of the model economy matches as many features of the constructed data set as possible. FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Asset Understanding Happened Values: on Stock the Way Prices, to the Exchange Data Bank Rates, and the “Peso Problem” When the economy is stable, forecasters using historical data have a hope of predicting the future with some accuracy. “Stable,” in this context, doesn’t mean unchanging. Rather, it means that the future is similar to the past in that the likely occurrence of any future economic outcome is about the same as what we observed in the past. For example, if people conclude, based on an analysis of economic history, there’s a 1 percent chance the stock market will crash in any given year, they can confidently extrapolate that analysis into the future if the economy is stable. But if the economy is unstable, such an extrapolation may not work well. If the economy is not stable, people’s beliefs about the likelihood of future events may, correctly, be different from what was observed in the past. Peso problems may occur when the economy faces this sort of instability. In this environment, using historical data to predict the future is difficult because the future may be much different from the recent past.2 Wars, nationalizations of industries, and severe political turmoil are examples of unusual events that are extremely difficult to predict. But when markets think there is a chance such events may occur, that perception can have a dramatic impact on forecasts and forecast errors. Forecasts may capture the possibility of unusual events, but until the events actually occur, forecasters may seem to make persistent errors and their forecasts may look biased to someone who is not aware of the possi- 2 More technically, peso problems can be interpreted as a failure of the methodology of rational expectations econometrics, which requires that the ex post distribution of economic variables be equal to the expected ex ante distribution of the same variables. Another way to interpret the peso problem is as a small-sample problem in statistics. If the sample of data is large enough in the sense that the occurrence of rare events in the data coincides with their true likelihood of occurrence, then the ex post and ex ante distributions will be the same and analysts will have more success modeling investors’ expectations. Dean Croushore and Tom KeithStark Sill bility of an unusual or unprecedented outcome. Indeed, forecasts may look bad despite the fact that forecasters make their estimates using the best information at their disposal and the best practices. When peso problems are present, forecasts that look bad may actually be good. The Forward Premium Puzzle. Economists have examined whether peso problems can account for some apparent anomalies in the behavior of asset returns. One such anomaly is the forward premium puzzle in foreign-exchange markets.3 This puzzle is closely related to the forecasting issues we have been discussing. In the foreign-exchange market, investors can purchase forward contracts on currencies. A forward contract is an agreement to buy or sell a currency on a certain future date for a certain future price, called the forward rate. We might think that the forward rate would be a good predictor of what the spot exchange rate will turn out to be on the day the forward contract matures, since the forward rate is a price that embodies financial market participants’ beliefs about the future value of the spot rate. (The spot rate is the price at which a currency can be bought or sold for immediate delivery. If you go to your local bank and convert dollars to francs, the conversion takes place at the spot exchange rate.) In an efficient market, the forward rate will equal the market’s expectation of what the spot exchange rate will be when the forward contract matures. The forward rate prediction may be high or low in any given month, but on average, it ought to be correct.4 Economists would then say that the forward rate is an unbiased predictor of the future spot rate. However, when we look at the data, the forward rate is not an unbiased predictor of the fu- 3 See the 1994 article by Gregory Hopper for more on the forward premium puzzle and the efficiency of the foreign-exchange market. 4 Assuming investors are risk-neutral. 5 BUSINESS REVIEW ture spot rate. Statistical analysis shows that the forward rate tends to stay above or below the spot exchange rate for extended periods. One contributing explanation for this finding is the peso problem.5 If foreign-exchange markets think there is some chance the exchange rate will fall, then until it actually does, the forward exchange rate will remain below the spot value of the exchange rate, since the forward rate embodies the market’s expectation. Research by Martin Evans and Karen Lewis shows that the peso problem is potentially an important component in an explanation of the forward premium puzzle.6 Let’s look at a simple example. Suppose the spot exchange rate has been fixed at 20 cents per peso, and investors think there is a 95 percent chance it will remain at 20 cents but a 5 percent chance that the exchange rate will fall to 10 cents per peso. Then the expectation, or expected value, of the future exchange rate is 19.5 cents ([0.95 x 20] + [0.05 x 10]). As long as the exchange rate remains fixed at 20 cents per peso, a forecast error of 1/2 cent per peso will persist— it will occur every period until either the peso is devalued or markets revise their expectations about devaluation. Someone casually evaluating these forecasts might conclude that market participants are irrational, since they seem to be making the same mistake over and over. An economist is more likely to think that the market is getting things about right. We can then turn the problem around and use the market’s fore- 5 Another possible explanation is risk premiums in the foreign-exchange market. Risk premiums represent compensation to the asset holder for taking on the risk of holding the asset. See the survey article by Karen Lewis for an in-depth discussion of risk premiums and the peso problem as explanations for the predictability and variability of excess returns. 6 However, Evans and Lewis find that the peso problem by itself cannot resolve the forward premium puzzle. They do show that the bias introduced by peso problems can be economically significant. 6 SEPTEMBER/OCTOBER 2000 casts to infer beliefs about the future value of the peso or the probability of a devaluation. REGIME SWITCHING AS AN EXPLANATION OF PESO PROBLEMS One approach economists have used to model peso problems is to suppose that the economy goes through changes in regime.7 In general, regimes represent different environments. A simple example of changes in regime involves political parties and control of the legislature. Sometimes Democrats are in control, sometimes Republicans, and over time, control of the legislature switches back and forth between the two parties. Legislation and fiscal policy might be different under each of these regimes, and the overall performance of the economy could be regime-dependent as well. While politics offers a good example of a regime-switching process, we want to think more generally about the economy’s shifting randomly between two (or more) regimes. Examples of regimes might include periods of high or low inflation, periods of rising or falling exchange rates, or economic recessions and expansions. The key is that in one regime the disturbances to the economy are different from what they are in another regime.8 These disturbances affect economic variables, so the behavior of variables such as inflation, interest rates, or real output growth could be different in the different regimes. Regime switches are irregular events for the 7 This view is somewhat different from the view that peso problems are due to small probabilities of catastrophic events that may happen only once. The problem with one-time events is that they are very hard to model. If an event is repeated, even infrequently, there is a possibility of its being described statistically. See the 1996 article by Martin Evans for a survey of research on the regime-switching view of the peso problem. 8 A disturbance is an unpredictable event that affects the economy. Examples include dramatic changes in oil prices, weather-related catastrophes, or technological innovation. FEDERAL RESERVE BANK OF PHILADELPHIA A Funny ThingAsset Happened on Stock the Way to the Data Bank Understanding Values: Prices, Exchange Rates, and the “Peso Problem” Dean Croushore and Tom KeithStark Sill economy: they happen repeatedly but infre- gime-switching behavior.9 quently. We can easily see how this regimeAnother variable that seems to undergo reswitching instability could give rise to peso prob- gime-switching is the exchange rate. Research lems. Suppose the economy has been in one re- by Charles Engel and James Hamilton and by gime for a long time, but people now think there Martin Evans and Karen Lewis found that, from is a sizable chance that it will switch to another regime. Their behavior, which reflects their be9 lief that the economy may switch regimes, could See the article by James Hamilton for technical debe hard to interpret if we looked just at recent tails on fitting regime-switching models to data. history and falsely assumed the economy would always stay in the FIGURE 1: Regime Switching and the Exchange Rate current regime. How prevalent is this Marks per Dollar Exchange Rate regime-switching instability? We see it in many economic and financial variables. As an example, consider output growth over the business cycle. Recessions are repeated but infrequent events— there have been nine recessions since World War II. The economy behaves differently in recessions than in expansions. In Source: Author's calculations recessions, unemployment rises, real output falls, and investment and consumption drop. In exProbability That Dollar Is in Appreciating Regime pansions, unemployment falls, real output rises, and investment and consumption increase. We can think of recessions as one regime for the economy and expansions as another, different regime. Indeed, economists such as James Hamilton have successfully modeled real output growth in the United Source: Author's calculations States as following re7 BUSINESS REVIEW the early 1970s to the late 1980s, the U.S. dollar went through roughly three appreciating and two depreciating regimes against the German mark. My own calculations, using similar methods, confirm their results (Figure 1). The figure’s upper panel shows the pattern of the mark/dollar exchange rate over the sample period. The figure’s lower panel shows the probability that the dollar was in the appreciating regime. The closer the probability is to 1, the more likely that the dollar was in the appreciating regime. The closer the probability is to zero, the more certain that the dollar was in the depreciating regime. The bottom panel indicates that the dollar was in the depreciating regime twice and in the appreciating regime three times over the sample period. Many economic variables display behavior that looks like regime switching. Properly interpreting economic forecasts and forecast errors can be difficult when this type of instability is present. In the mark/dollar example, just as we saw in the simple dollar/peso exchange-rate example, if forecasters expect a regime switch to occur and it does not, their forecasts may appear to be biased. Further, the bias will persist until the regime switch occurs or expectations are revised. But persistent bias doesn’t necessarily mean that forecasters are doing something wrong. It may be that we don’t have enough information to see the full range of possible outcomes that forecasters are considering. If we did, we could approximate how often regime switches are likely to occur, then use that information to help us interpret forecasts. If we had enough data to correctly assess the full range of possible outcomes, forecast errors that appear biased when we look at an incomplete sample would look unbiased when we used all the data. Good forecasts would look good. PESO PROBLEMS, ASSET VALUES, AND FUNDAMENTALS Exchange rates are not the only financial variables that can be influenced by peso problems. 8 SEPTEMBER/OCTOBER 2000 Any asset whose current price depends on uncertain future payments could be affected. Take the case of stock prices. A standard model of stock prices relates the current price of a share to the stream of future dividends that the stockholder expects to receive from owning the share. All else equal, when expectations of future dividends are revised up, the price of stock goes up. When expected future dividends are revised down, the price of stock goes down. But in an economy where peso problems can occur, the link between stock prices and information about future dividends becomes more complicated. Let’s suppose the economy goes through regime changes that affect stock prices. In other words, the behavior of dividends over time depends on which regime the economy is in. In the “good” regime, dividend growth tends to be high. In the “bad” regime, dividend growth tends to be low. It can still be the case that in some years dividend growth is low in the good regime or high in the bad regime, but these are unusual outcomes. Since stock prices depend on expected future dividends, stock prices will also depend on which regime the economy is in. If the regime is good, stock prices will be high. If the regime is bad, stock prices will be low. However, because dividend growth can be low in the good regime or high in the bad regime—although it doesn’t happen very often– investors can’t be certain which regime the economy is in at any given time. Therefore, investors must form a belief about the current regime based on their observations of the economy. New information on economic variables may strengthen or weaken investors’ belief that the economy is in a particular regime. They may believe fairly strongly that the economy is in one particular regime, but it is unlikely they are ever absolutely certain. This uncertainty means that stock prices will be a weighted average of the good-regime dividends and the bad-regime dividends. The weights reflect the strength of investors’ beliefs about which regime the economy is in. FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Asset Happened on Stock the Way to the Data Bank Understanding Values: Prices, Exchange Rates, and the “Peso Problem” In this environment, news about future dividends can affect stock prices in three ways: (1) there may be new information about good-regime dividends; (2) there may be new information about bad-regime dividends; (3) there may be new information that changes investors’ beliefs about which regime the economy is in. Suppose investors believe the economy is currently in the good regime and news arrives that suggests future dividends in the good regime will be higher. As we have already stated, this news will lead to an increase in stock prices. Alternatively, the economy might be in a good regime when news arrives that leads investors to believe that, were the economy to switch to the bad regime, dividends would be even lower than they previously thought. Dividends in the good regime may be unaffected by this news, but the stock price drops right away because the stock price is a weighted average of the good-regime dividends and the bad-regime dividends. So stock prices could change even if expected dividends in the current regime don’t change.10 Finally, new information may affect investors’ beliefs about which regime the economy is in. For example, they may become more confident that the economy is in the good regime. Recall that stock prices are a weighted average of the share price in each regime and that the weights depend on the strength of people’s beliefs about the economy’s current regime. If those beliefs change, stock prices will change.11 10 More technically, when peso problems are present, financial markets take account of future capital gains and losses associated with regime switches. See the 1996 article by Martin Evans for a detailed discussion. 11 Researchers have fit regime-switching models to stock dividends and fundamentals. See the papers by Evans (1993); Shmuel Kandel and Robert Stambaugh; and Steve Cecchetti, Pok-Sang Lam, and Nelson Mark for examples. These papers illustrate the conditions under which peso problems can influence the behavior of stock returns. Dean Croushore and Tom KeithStark Sill Thus, in an environment where peso problems may be present, new information about dividends can affect stock prices in complicated ways. Stock prices may jump around even if there is no new information about dividends in the current regime. Stock-price models may also be affected by peso problems. If dividends do indeed depend on which regime the economy is in, an investigator may falsely conclude that a particular model performs poorly if he fails to account for regime switching when evaluating the model’s performance using historical data. Peso problems can lead to stock-price behavior that appears inconsistent with the view that stock markets are efficient and investors are rational (see Stock Market Bubbles). An Equity Return Puzzle. A striking fact about the U.S. economy relative to the economies of other industrialized nations is the extent of the real appreciation of the stock market. The empirical facts are well laid out in a paper by Philippe Jorion and William Goetzmann: the real return on equities has averaged about 4.7 percent for the United States, compared to a median real return of 1.5 percent for a sample of 39 other countries.12 No country has a higher real return than the United States over the period 1921 to 1995 (Figure 2), even though many other countries’ stock markets have long histories of continuous operation. Why does the United States have this uniquely high real return to equities? Jorion and Goetzmann conjecture that it may be due to the fact that disasters have largely bypassed the U.S. economy. For example, at the beginning of the 1920s there were active stock markets in many countries, including France, Russia, Germany, Japan, and Argentina. But the stock markets of all of these countries were interrupted by war, 12 The real return on equities is measured as the realized return from capital gains and dividends minus the inflation rate. 9 BUSINESS REVIEW SEPTEMBER/OCTOBER 2000 Stock Market Bubbles The text focuses on the case where stock prices are determined by their dividends. But stock prices may have another component: a bubble. The bubble theory of stock prices suggests that stocks might go through long periods of under- or overvaluation relative to the value implied by their fundamentals.a One type of bubble is a rational bubble. Rational bubbles reflect investors’ self-fulfilling beliefs that the price of a stock (or other asset) depends on variables unrelated to fundamentals. When bubbles are rational, there are no obvious profit opportunities to exploit; investors are efficiently using all relevant information to assess the asset’s value.b Whether rational bubbles can be found in asset prices is a matter of ongoing research for economists. Some statistical tests give results consistent with the presence of bubbles. Others show that bubble components seem to be unimportant.c One difficulty in testing for bubbles is that peso problems can give rise to the appearance of bubbles, even if they are not really there. Take the case where dividends are either in a good or bad regime. Suppose the economy is in a good (high-dividend) regime and positive news arrives about future dividends in the bad (low-dividend) regime. As described in the text, the current price of stocks will adjust to this news, even though dividends and fundamentals in the current regime may be unchanged. The change in the price of stocks might therefore be unrelated to the observed fundamentals in the current regime: it looks like a bubble. Thus, environments in which peso problems are present may make it look as if there is a bubble component to asset prices even when asset prices are actually being driven only by their fundamentals. a Fundamentals are the factors that economic theory suggests are important determinants of stock prices. They include such variables as profits, interest rates, and dividends. b See the article by Lee Ohanian for more on rational bubbles in asset prices. c See the article by Ohanian for a review of the literature on testing for bubbles. The evidence is mixed. One major problem in testing for bubbles is that it involves a joint test of a particular model of asset prices and the presence of a bubble. That is, researchers may find evidence for bubbles simply because their model of fundamentals is wrong. 10 hyperinflation, or political turmoil. Presumably investors thought there was some probability that the U.S. market would be disrupted as well. But this event has not occurred, so historical equity returns have not reflected it. The large realized return in the United States may be tied to investors’ recognition of the possibility of economic disruption and stock market interruption that never materialized—it may be, in part, a peso problem. Investors generally do not like risk. Risk-averse stock market investors want a high return on investment in normal times to compensate them for the risk of the extreme losses they would incur if the stock market crashed or was interrupted by war or political turmoil. The United States has not experienced the extreme financial market disruptions that many other countries have. Perhaps, by the luck of the draw, U.S. stockholders have been rewarded for catastrophic events that happened not to occur.13 13 The real return on equities can be interpreted as an equity premium. A very influential statement of the equity premium puzzle in the U.S. data is the paper by Rajnish Mehra and Edward Prescott. Building on Mehra and Prescott, the paper by Thomas Rietz attempts to explain the equity premium puzzle using a model that has a peso problem environment. More recently, the paper by Jean-Pierre Danthine and John Donaldson investigates peso problem implications for the equity premium in a production economy. Other research has tried to explain the equity premium puzzle in a model where stock market fundamentals follow a regime-switching process. These efforts have been less FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Asset Happened on Stock the Way to the Data Bank Understanding Values: Prices, Exchange Rates, and the “Peso Problem” Dean Croushore and Tom KeithStark Sill FIGURE 2: Real Returns on Global Stock Markets Source: Adapted from Philippe Jorion and William N. Goetzmann, “Global Stock Markets in the Twentieth Century,” Journal of Finance, 54, June 1999, Figure 1. Used with permission. Peso Problems Spell Trouble for Value At Risk Models. Value at risk (VAR) models estimate the largest loss a portfolio of assets is likely to suffer under relatively normal circumstances. Financial institutions use VAR models to determine the potential losses on their portfolios.14 For example, a bank may want to know the maximum loss it might incur on its portfolio over a specific period. A VAR calculation might show that, on average, in 95 trading days out of 100, the maximum loss on the bank’s portfolio is not expected to exceed $10 million in a day. VAR models are constructed using historical successful. See the 1996 survey article by Martin Evans for a technical discussion and references. 14 See the 1996 article by Greg Hopper for more on VAR models. returns on the assets that make up an institution’s portfolio. Because VAR models rely so heavily on the historical pattern of asset returns, they may be unreliable in environments where peso problems are present. Suppose that the day-to-day change in asset returns switches between small-change and large-change regimes. If a VAR model is constructed using historical data from only the smallchange regime, it would understate the maximum loss a portfolio would suffer should asset returns switch to the large-change regime. A VAR model would be less likely to understate potential losses if the historical data used to construct it included both small-change and large-change regimes. In practice, though, VAR models tend to weight the most recent observations on historical returns most heavily. As we have seen, when peso problems are present, the recent past is not a good guide to the true underlying distri11 BUSINESS REVIEW bution of asset returns. Thus, VAR models may not accurately estimate the maximum loss a portfolio may suffer. CONCLUSION Peso problems can arise when people assign a positive probability to events that might occur but that are not well-represented in historical data. Because asset prices embody the financial market’s perceived probabilities about possible future values of economic variables, they are particularly sensitive to peso problems. Peso problems do not reflect a market failure or a market inefficiency. Rather, peso problems reflect economic analysts’ difficulties in using historical data to properly model people’s expectations about the future. While the peso problem most commonly comes up when analyzing foreignexchange markets, we have seen that it may affect any asset market where expectations determine prices. The principal consequence of the SEPTEMBER/OCTOBER 2000 peso problem is that it makes it more difficult to correctly interpret the predictions of economic forecasts and asset-pricing models. Whether peso problems contribute to assetpricing anomalies is largely an empirical issue. We have discussed mechanisms by which peso problems can potentially affect asset prices. The principal difficulty in studying peso problems is how to model people’s expectations when the economic environment is unstable. Small changes in expectations can often lead to large changes in people’s behavior and, thus, in the behavior of economic variables such as asset prices. The literature on testing for the presence of peso problems and the literature on building economic models that incorporate peso-problem explanations of asset-price behavior are promising but still new. Nonetheless, the literature makes clear that it can be dangerous to base forecasts about the future behavior of financial variables solely on their recent behavior. REFERENCES Cecchetti, Steve, Pok-Sang Lam, and Nelson C. Mark. “The Equity Premium and the Risk-Free Rate: Matching the Moments,” Journal of Monetary Economics, 31 (1993), pp. 21-46. Danthine, Jean-Pierre, and John Donaldson. “Non-Falsified Expectations and General Equilibrium Asset Pricing: The Power of the Peso,” PaineWebber Working Paper Series in Money, Economics and Finance, Columbia Business School, PW-97-19 (August 1998). Engel, Charles, and James Hamilton. “Long Swings in the Dollar: Are They in the Data and Do the Markets Know It?” American Economic Review, 80 (1990), pp. 689-713. Evans, Martin. “Dividend Variability and Stock Market Swings,” manuscript, New York University (1993). Evans, Martin. “Peso Problems: Their Theoretical and Empirical Implications,” in G.S. Maddala and C.R. Rao, eds., Statistical Methods in Finance. Amsterdam: Elsevier, 1996, pp. 613-46. Evans, Martin, and Karen Lewis. “Do Long-Term Swings in the Dollar Affect Estimates of the Risk Premia?” Review of Financial Studies, 8 (1995), pp. 709-42. 12 FEDERAL RESERVE BANK OF PHILADELPHIA A Funny ThingAsset Happened on Stock the Way to the Data Bank Understanding Values: Prices, Exchange Rates, and the “Peso Problem” Dean Croushore and Tom KeithStark Sill Hamilton, James. “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle,” Econometrica, 57 (1989), pp. 357-84. Hopper, Gregory. “Is the Foreign Exchange Market Inefficient?” Business Review, Federal Reserve Bank of Philadelphia, May/June 1994. Hopper, Gregory. “Value at Risk: A New Methodology for Measuring Portfolio Risk,” Business Review, Federal Reserve Bank of Philadelphia, July/August 1996. Jorion, Philippe, and William Goetzmann. “Global Stock Markets in the Twentieth Century,” Journal of Finance, 54, June 1999. Kandel, Shmuel, and Robert Stambaugh. “Expectations and Volatility of Consumption and Asset Returns,” Review of Financial Studies, 3 (1990), pp. 207-32. Lewis, Karen. “Puzzles in International Financial Markets,” NBER Working Paper No. 4951 (December 1994). Mehra, Rajnish, and Edward Prescott. “The Equity Risk Premium: A Puzzle,” Journal of Monetary Economics, 15 (1985), pp. 145-61. Ohanian, Lee. “When the Bubble Bursts: Psychology or Fundamentals?” Business Review, Federal Reserve Bank of Philadelphia, January/February 1996. Rietz, Thomas. “The Equity Premium: A Solution,” Journal of Monetary Economics, 22 (1988), pp. 117-31. 13 A Funny Thing Happened on the Way to the Data Bank Dean Croushore and Tom Stark A Funny Thing Happened On the Way to the Data Bank: A Real-Time Data Set for Macroeconomists Dean Croushore and Tom Stark* I n October 1999, the U.S. government dramatically revised its data series on real gross domestic product, the best measure of the economy’s total output. The new data showed that the economy had been growing somewhat faster over the previous decade than had been reported earlier. When data are revised, econo*Dean Croushore is an assistant vice president and economist in the Research Department of the Philadelphia Fed. He is head of the department’s Macroeconomics section. Tom Stark is an economic analyst in the Research Department. mists face unique problems when forecasting, studying the economy, and analyzing economic policy. For example, economists are constantly trying new methods of forecasting the economy. An economist develops a new forecasting method by taking data about the economy, such as real output, unemployment, interest rates, and inflation rates, then relating those variables to each other through a set of equations that make up an economic model. The economist then looks at how well the model explains movements of the data in the past and how well it forecasts future 15 BUSINESS REVIEW movements of the data. But substantial data revisions, like those in October 1999, throw a monkey wrench into the development of economic models. A key problem is that the data now being used to develop forecasting models can differ from the data used prior to October 1999. Data revisions also cause problems when economists analyze past decisions about changes in policy, especially monetary policy. Many economists write articles about how monetary policy has been conducted in the past. They look at today’s economic data and argue that monetary policy was tightened or loosened, that is, interest rates were increased or reduced, in response to, say, changes in real output or changes in the inflation rate. But often the data they’re looking at have been revised dramatically and look nothing like the data that monetary policymakers were confronted with at the time the decision about interest rates was made. Because of problems like these, economists need a data set containing only the observations that were known at each point in time. Such a data set would answer questions such as: What data were available to the Federal Reserve when it met to discuss monetary policy issues in February 1974? If an economist were to prepare a forecast of output growth or inflation using a new model and using data that were known in October 1987, what would the forecast be? These types of questions can be answered only by constructing a data set that shows what the data looked like at different points in time. Doing so has been the subject of a project that the Federal Reserve Bank of Philadelphia has undertaken over the past seven years. The project required a painstaking collection of data series as they appeared in printed documents from the past. The result is a real-time data set for macroeconomists. The data set is quite large, as you might expect, and will get larger over time as we add variables to it. Research using the real-time data set is in its preliminary stages, but it generally shows that: (1) the results of certain types of fore16 SEPTEMBER/OCTOBER 2000 casting methods are very sensitive to revisions in the data, while other methods are more stable; (2) estimates of how monetary policymakers react to data are sometimes quite different when real-time data are used; and (3) the results of empirical research in macroeconomics sometimes change significantly when revised data are used. In addition, the data set can be used to study the process of data revision, which may itself be important. THE DATA SET The real-time data set was constructed to reflect, at each date, exactly what the macroeconomic data looked like at that date. We use the term vintage to describe each different date for which we have data as they looked at the time. For example, suppose we were to look at the growth rate of real output for the first quarter of 1977. The first time real output for that quarter was reported, the national income and product accounts showed that real output grew 5.2 percent—that’s the reading in our May 1977 vintage of the real-time data set. Today, when we look at the national income and product accounts, the growth rate of real output for the first quarter of 1977 is listed as 5.0 percent.1 You can pick any vintage between May 1977 and today and look in our data set to see the value of real output for the first quarter of 1977 as recorded in that vintage. Currently, the data set consists of 23 quarterly variables, including quarterly observations of 10 monthly variables. The variables include nominal output, real output and all of its components, measures of the money supply, measures of bank reserves, and the unemployment rate; for a complete list, see Variables Included in the Real-Time Data Set. There is a new vintage of the data set every three months, beginning in November 1965. The 1 When we say “today,” we mean May 2000, when this article was written. FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank data included in each vintage are those an economic analyst would have had available in the middle of each quarter. Thus, the vintages correspond to data as they existed on November 15, 1965; February 15, 1966; May 15, 1966; August 15, 1966; November 15, 1966; and so on. For most variables, each vintage contains all the historical data (back to the first quarter of 1947) available at the time. The data set is posted on the Internet at www.phil.frb.org/econ/forecast/reaindex. html. The web page contains links to the data itself, research papers that describe the data in more detail and use the data in a variety of empirical exercises, a bibliography of research papers that deal with real-time data issues, complete documentation on the data set, a description of changes in the data set, and a note on data we need to complete the data set, in case anyone can tell us of their whereabouts. As you can imagine, this type of data set would have been easy to create if only someone had collected the data as time went on. We’ve been collecting some of these data since 1991. But gathering the bulk of the data set required us to go back into historical documents (mainly the Survey of Current Business for data from the national income and product accounts) and manually enter the data into a computer spreadsheet.2 Two major problems occurred in constructing this data set. First, the historical documentation sometimes did not make clear the exact date on which the data were available. Since we want this data set to include only those observations that would have been available to someone on a particular date, it’s especially important not to include observations that were published after 2 Much of this work was done by interns from Princeton University and the University of Pennsylvania, as well as research assistants at the Federal Reserve Bank of Philadelphia. We thank all of them for their hard work and dedication to this monumental task. We especially wish to acknowledge one student, Bill Wong, whose contributions were particularly notable. Dean Croushore and Tom Stark Variables Included in the Real-Time Data Set Quarterly Observations: Nominal GNP (before 1992) or GDP (1992 and after) Real GNP (before 1992) or GDP (1992 and after) • Consumption and its components: - Durables - Nondurables - Services • Components of investment: - Business fixed investment - Residential investment - Change in business inventories (Change in private inventories after August 1999) • Government purchases (government consumption and gross investment, 1996 and after) • Exports • Imports Chain-Weighted GDP Price Index Monthly Observations: (quarterly averages of these variables are also available in the quarterly data sets) Money Supply Measures: • M1 • M2 Reserve Measures: (data from Board of Governors): • Total reserves • Nonborrowed reserves • Nonborrowed reserves plus extended credit • Monetary base Civilian Unemployment Rate Consumer Price Index 3-month T-bill Rate 10-year T-bond Rate 17 BUSINESS REVIEW the date in question. Consequently, we spent a lot of time trying to determine exactly when data were available. Whenever there was doubt about the timing, we didn’t include the data until we were sure about the date on which it had been made available to the public. We have prepared complete documentation, describing in detail all the source data, what was included, and what wasn’t. The second major problem was verifying the accuracy of the data that we typed into our spreadsheets. With such a huge data set, the opportunity for data-entry errors is large. To minimize the chance of errors in the data set, we did a large number of checks to ensure that components added up to totals; for example, total consumption spending should equal consumption spending on durables plus consumption spending on nondurables plus consumption spending on services.3 We also plotted many of the variables to see if there were numbers that didn’t make sense or that contained typos. We’re confident that the data set contains few errors; any errors that remain are likely to be small. DATA REVISIONS One important use of the data set is to characterize how data are revised. Many data series are revised on a regular basis because the government issues preliminary numbers before all the underlying information is available. For example, the Bureau of Economic Analysis (BEA), the government agency that issues the gross domestic product (GDP) data, releases its first report on the nation’s GDP near the end of the month following the end of a quarter; that release is called the advance report. But at the time of the advance report, the BEA doesn’t yet have complete information, so it makes projections about certain components of GDP from incom- 3 Prior to 1996, the components of real output added up to real output, but that’s not true under the chainweighting method used since 1996. 18 SEPTEMBER/OCTOBER 2000 plete source data. As time goes on, the source data become more complete. But it usually isn’t until the following year that better information, such as income-tax records and economic census data, is available. So the GDP data undergo a continual process of revision. The data for the first quarter of 2000 were first released on April 27, 2000; they were revised on May 25, 2000, again on June 29, 2000, and yet again on July 28, 2000. Some time will pass before the first quarter observation is revised again, generally in July of each of the following three years. Thus, the data for the first quarter of 2000 will change in July 2001, July 2002, and July 2003. Each revision will be based on more complete information, so the data should become more reliable over time. In addition to this regular schedule of revisions, the government periodically (about every five years) makes major changes, called benchmark revisions, to the data for the national income and product accounts. The most recent of these (as of this writing) occurred in October 1999. Benchmark revisions incorporate new source data and may also include changes in definitions of variables or changes in methodology. The changes are necessary, in part, because our economy is constantly changing: different types of products enter the market and different accounting methods need to be used. For example, in the benchmark revision of October 1999, the BEA changed the way it classified computer software purchased by businesses and government. Formerly treated as an office expense, such software is now treated as an investment, which is more logical because software lasts many years. The October 1999 revisions raised the average growth rate of real output over the previous two decades. Other benchmark revisions include changes in methodology that improve the quality of the data. In the benchmark revision of January 1996, for example, the method of calculating real output was changed from a fixed-weight to a chainweight method. Why? Because economic research had shown that the chain-weight method FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank was an improvement over the fixed-weight method, which tended to distort calculations of real output growth in the distant past. The chainweight method eliminates this problem.4 How Large Can Revisions Be? To get an idea of the size of revisions, let’s return to our example of the growth rate of real output for the first quarter of 1977. Earlier, we noted that in the May 1977 vintage, the growth rate was 5.2 percent, but in the May 2000 vintage, it was 5.0 percent. That difference of just 0.2 percentage point hides quite a wild ride (Figure 1). We began at 5.2 percent in May 1977, but in the August 1977 vintage, the growth rate for the first quarter of 1977 was revised to 7.5 percent, the result of the annual revision of the data that incorporated new information. In August 1978, the growth rate was revised down slightly to 7.3 percent as more new information, including data from tax 4 For more details on chain weighting and what it means, see the article by Steven Landefeld and Robert Parker. Dean Croushore and Tom Stark returns, was incorporated into the accounting process. Then in August 1979, the availability of even more new data caused the growth rate for the first quarter of 1977 to be revised up to 8.9 percent. Note that, even two and a half years after the fact, the raw data on real output were still being modified, as more and more records became available. But variation in the growth rate of real output for the first quarter of 1977—from 5.2 percent to 7.5 percent to 7.3 percent to 8.9 percent—is minor compared to what happened after that. A benchmark revision of the national income accounts in late 1980 caused the growth rate to rise to 9.6 percent. A minor change in August 1982 brought the growth rate back down to 8.9 percent. Yet another benchmark revision in late 1985 drove the growth rate, as recorded in our February 1986 vintage, all the way down to 5.6 percent. It remained there until late 1991, when another benchmark revision nudged it back to 6.0 percent. In February 1996, it changed to 5.3 percent. Then, in May 1997, 20 years after the fact, the growth rate was revised again, this time FIGURE 1: Real Output Growth for 1977Q1 (as viewed from the perspective of 93 different vintages) 19 BUSINESS REVIEW down to 4.9 percent, as the output data were changed to be consistent with newly available data on wealth. In early 2000, the growth rate was revised up slightly to 5.0 percent. These changes in the measure of the growth rate of real output in a particular quarter are fairly dramatic. It’s particularly interesting that the numbers changed so much from their initial values long after the fact, especially the decline in the growth rate from 8.9 percent to 5.6 percent in the February 1986 vintage. Another perspective on the size of revisions can be gained by examining a chart that shows the relative frequency of revisions of a given size to the growth rate of real output (Figure 2).5 The revision represents the difference between the annualized growth rate of real output as reported in the BEA’s advance report and the growth rate 5 This figure shows the revisions for all quarters from the third quarter of 1965 to the second quarter of 1999. The labels associated with the ranges shown on the horizontal axis are rounded to one decimal place. SEPTEMBER/OCTOBER 2000 for that quarter in the latest vintage of data at the time this article was written. Each bar in the chart shows the percentage of times (on the vertical axis) a revision of a particular size occurs (shown by the ranges on the horizontal axis). For example, the tallest bar on the chart shows that just over 25 percent of the time, the total revision to quarterly real output growth from its initial release to the latest available data ranged from a decline of 0.5 percent to an increase of 0.5 percent annually. You can see that many of the revisions aren’t too far from zero, but a few are quite large, either positive or negative. How Big Are Revisions Over Longer Periods? The example above showed that data revisions in a particular quarter can be fairly substantial. But we know there’s a lot of volatility in quarter-to-quarter growth rates of real output and not nearly as much over longer periods. The same may be true of revisions to the growth rates. Consequently, we examine the extent to which data revisions affect five-year growth rates. If we examine data on nominal output growth, real output growth, and inflation over FIGURE 2: Relative Frequency of Data Revisions Quarterly Growth Rate of Real Output Size of Revision from Advance Report to Latest Vintage 20 FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank five-year periods, we see that even long after the fact, the five-year growth rates can change (Table).6 For example, inflation averaged 7.7 percent from 1975 to 1979 according to the 1995 benchmark vintage of the data, but only 7.2 percent according to the 1999 benchmark vintage of the data. Real output growth (the inflationadjusted growth rate of output) from 1955 to 1959 was as low as 2.7 percent in the 1995 benchmark vintage of the data, but as high as 3.2 percent in the 1999 benchmark vintage. Thus, even five-year average growth rates may be substantially different across vintages of the data, though revisions are much smaller than for quarterly data. Even nominal output (the dollar value of output), which is easier to measure than real output and inflation, gets revised long after the fact, thanks to changes in how output is defined. Another way to see how large data revisions may be is to look at a time-series plot that compares the data as they appeared in the BEA’s advance report to how they stand today. Since we’ve already seen that revisions to quarterly growth rates are very volatile and revisions to five-year growth rates are smoother but still substantial, 6 The vintages chosen in this table are the last vintages of the data set prior to a benchmark revision: November 1975, November 1980, November 1985, November 1991, November 1995, and August 1999. Dean Croushore and Tom Stark Table: Averages Over Five Years For Benchmark Vintages Annualized percentage points Vintage Year: ’75 Period ’80 ’85 ’91 ’95 ’99 Nominal Output Growth Rate 1950 to 1954 7.9 7.9 7.9 8.1 8.0 8.0 1955 to 1959 5.6 5.6 5.7 5.7 5.7 5.7 1960 to 1964 5.6 5.5 5.6 5.6 5.7 5.6 1965 to 1969 8.0 8.1 8.2 8.3 8.2 8.2 1970 to 1974 8.6 8.8 8.9 9.1 9.0 9.1 1975 to 1979 NA 11.1 11.2 11.3 11.4 11.4 1980 to 1984 NA NA 8.5 8.2 8.5 8.6 1985 to 1989 NA NA NA 6.5 6.7 6.7 1990 to 1994 NA NA NA NA 5.2 5.1 1950 to 1954 5.2 5.1 5.1 5.5 5.5 5.3 1955 to 1959 2.9 3.0 3.0 2.7 2.7 3.2 1960 to 1964 4.1 4.0 4.0 3.9 4.0 4.2 1965 to 1969 4.3 4.0 4.1 4.0 4.0 4.4 1970 to 1974 2.1 2.2 2.5 2.1 2.3 2.6 Real Output Growth Rate 1975 to 1979 NA 3.7 3.9 3.5 3.4 3.9 1980 to 1984 NA NA 2.2 2.0 1.9 2.2 1985 to 1989 NA NA NA 3.2 3.0 3.2 1990 to 1994 NA NA NA NA 2.3 1.9 Inflation 1950 to 1954 2.6 2.7 2.7 2.5 2.4 2.6 1955 to 1959 2.6 2.6 2.6 2.9 2.9 2.4 1960 to 1964 1.4 1.5 1.5 1.6 1.6 1.3 1965 to 1969 3.6 3.9 3.9 4.1 4.1 3.7 1970 to 1974 6.3 6.5 6.2 6.8 6.5 6.3 1975 to 1979 NA 7.1 7.0 7.5 7.7 7.2 1980 to 1984 NA NA 6.1 6.1 6.4 6.2 1985 to 1989 NA NA NA 3.3 3.6 3.4 1990 to 1994 NA NA NA NA 2.9 3.1 21 BUSINESS REVIEW SEPTEMBER/OCTOBER 2000 FIGURE 3: Revisions to Real Output Growth we’ll take a look at real output growth over one year (Figure 3). The figure shows the differences between the growth rates of real output as they appear in one recent vintage (November 1999) and the growth rates of real output as each was first reported in the BEA’s advance report. As you can see, the one-year growth rates are often revised dramatically—by over 3.5 percentage points in one instance. Do Data Revisions Change Our Perception of Recessions? An important aspect of data revisions is how they affect our view of business cycles, in particular, the severity of recessions. Our sense of the severity of recessions, measured by the average rate of growth of real output, often changes when data are revised.7 For example, in our November 1991 vintage, the average growth rate of real output in the recession that lasted from the third quarter of 1990 through the first quarter of 1991 was -1.0 percent. But the recession appeared worse when real output data were revised downward; in the August 1992 vin- 7 Note that in a recession, many sectors of the economy turn down together, so the growth rate of real output is usually negative. 22 tage, the average growth rate of real output was -2.8 percent. However, later still, the recession appeared less severe, when the average growth rate of real output was revised to -1.8 percent (in the November 1999 vintage). IS RESEARCH IN MACROECONOMICS SENSITIVE TO DATA REVISIONS? The real-time data set can also be used to examine research in macroeconomics to see if results are sensitive to the vintage of data being used; that is, do the results change significantly if a researcher uses a different vintage? In a recent paper, we examined a number of different empirical studies and found that some hold up very well, but other results change when different vintages of the data are used. These tests for the sensitivity of results are helpful to macroeconomic researchers who need to know if they can draw general conclusions from their results.8 We examined the 1990 paper by Finn Kydland and Ed Prescott, which showed the relationship 8 See our 1999b research paper for more examples and more details. FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank of a number of economic variables to real output.9 Kydland and Prescott used some simple statistics to show the relationships between different macroeconomic variables. The article is important because its results are one standard by which macroeconomists decide whether their business-cycle models fit the facts well enough to be useful. The main statistic Kydland and Prescott looked at was the correlation statistic, which measures the degree to which variation in one variable is associated with variation in another variable. A negative correlation would mean that when one variable rises or falls, the other usually moves in the opposite direction. A positive correlation would mean that when one variable rises or falls, the other one usually moves in the same direction. The correlation can never be greater than 1 or less than -1, and the closer the correlation is to 1 (or to -1), the closer is the association between the two variables. Kydland and Prescott found that the price index had a negative correlation with real output of –0.55, which means that the price index and real output generally move in opposite directions. Using a more recent vintage of the data, we find that the correlation is now slightly more negative: –0.66. Kydland and Prescott found that the correlation between output and consumer spending was 0.82; in today’s vintage data it’s 0.88. They found that the correlation between the M2 measure of the money supply and real output was 0.46; in today’s vintage data it’s 0.48. Looking at many other variables yielded similar results, so we conclude that the results of Kydland and Prescott hold up quite well. We also examined a 1989 paper by Olivier Blanchard and Danny Quah, who used a small model of the economy to examine how a shock to the demand for goods and services (such as a 9 The data were adjusted by a statistical procedure to remove long-term trends, in order to focus on their movement over the business cycle. Dean Croushore and Tom Stark war, which increases government purchases sharply) or a shock to the supply of goods and services (such as a dramatic increase in oil prices) affected the economy.10 While most of Blanchard and Quah’s empirical results hold up fairly well when we look at different vintages of the data, in one case they don’t. When we examine how a demand shock affects the unemployment rate, we find that in more recent vintages of the data, there’s a much larger effect (Figure 4).11 Each line in the figure corresponds to a different vintage of the data and shows how the unemployment rate responds over time to a demand shock. When we use the February 1988 vintage, the unemployment rate drops immediately, then declines even more for several quarters until the end of the third quarter after the shock. Then the rate gradually returns to its starting point. But the impact of a shock to demand on the unemployment rate is bigger when we use the November 1993 vintage of the data and gets dramatically bigger when we use the February 1998 vintage. So although most of Blanchard and Quah’s results weren’t affected by the choice of vintage, the vintage strongly affected their estimate of the impact of a shock to demand on the unemployment rate. Evidently the statistical technique used in that study is sensitive to data revisions. From these and other studies we examined, we concluded that most empirical work in mac- 10 A shock is a sudden and surprising change to supply or demand. 11 The figure shows the response of the unemployment rate to a demand shock that increases demand enough to lower the unemployment rate by one percentage point if no other variable in the model responds to the shock in the period in which the shock occurs. (In technical terms, the demand shock shifts the equation in the model describing demand, by changing the intercept term for unemployment by one percentage point.) The opposite effect on the unemployment rate would occur if there was a decrease in demand. 23 BUSINESS REVIEW SEPTEMBER/OCTOBER 2000 FIGURE 4: How the Unemployment Rate Responds To a Shock That Increases Demand (as viewed from the perspective of the 3 vintages shown) roeconomics holds up fairly well when the vintage of the data is changed, but some empirical methods, like that used by Blanchard and Quah, are more sensitive to vintage than others. POLICY ANALYSIS The real-time data set also helps economists understand policy actions. An economist studying past economic policies is probably doing so in light of the data as they exist today. But today’s data have been revised extensively and may be quite different from the data that policymakers had available to them when they made their decisions. But if the economist has a real-time data set, she can see exactly what the economy looked like to policymakers when they made their decisions. Consider the situation in early October 1992. Today’s data tell us the economy was in pretty good shape in late 1992. Real output grew 4.3 percent in the first quarter, 4.0 percent in the sec24 ond quarter, and 3.1 percent in the third quarter. But if you read accounts from that time, policymakers were clearly worried about whether the economy was recovering from the recession, and they were contemplating actions to stimulate the economy. Why were policymakers so worried? According to the data available to them, the economy had grown just 2.9 percent in the first quarter (less than today’s revised number of 4.3 percent shows) and 1.5 percent in the second quarter (much lower than today’s 4.0 percent). Statistics for the third quarter had not yet been released, but forecasts suggested that economic growth had not picked up much from the second quarter’s anemic 1.5 percent. In addition, a number of monthly indicators pointed to a decline in the economy. (Later, many of these indicators were also revised up significantly.) Thus, it would be hard for an economist today to understand the policy concerns of the past without knowing the data FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank policymakers were looking at. Using the data that policymakers had before them would seem to be especially important if we were trying to model how policymakers act, a research area some economists have been interested in recently.12 USING REAL-TIME DATA FOR ANALYZING FORECASTS The real-time data set can be used in a variety of ways to evaluate forecasts. Its main use, however, is likely to be in constructing new forecasting models. Sometimes an economist creates a new forecasting model using today’s data, then claims that had this model been used in the past, it would have generated better forecasts than those generated by the models forecasters were using at the time. But such a claim isn’t valid because past forecasters didn’t have the same data to work with as today’s economists do. To properly compare forecasts, an economist needs to work with a real-time data set, feed the proper vintages of the data into the forecasting model, and then see if the forecast is better. A Simple Model with One Variable. To illustrate this idea, we’ve generated a simple forecasting model that uses only the history of real output to generate forecasts of future real output. We ran a simulation exercise comparing two procedures: (1) using today’s data vintage and pretending that such data were available earlier; and (2) feeding data from the real-time data set into the model to generate forecasts. The first method is the technique an economist is forced to use in the absence of a real-time data set. Doing so assumes that the data aren’t too different from what would have been available to a forecaster at the time. But as we’ve seen, that’s not true. The second method uses the data available to a forecaster at the time a particular forecast was made. Dean Croushore and Tom Stark The simulation exercise amounts to reconstructing what a forecaster would have done in real time. Consider a forecaster in February 1975 who wanted to forecast real output growth for the coming year. Data on real output through the fourth quarter of 1974 were available to her. For illustrative purposes, we assume that she used a very simple model to forecast future real output based on its history.13 Using our realtime data set, we know exactly what data were available to her (our February 1975 vintage data), and we generate a forecast for the growth rate of real output over the next four quarters. The forecast turns out to be 1.3 percent. Then, imagine that three months go by, and we repeat the exercise, this time using the May 1975 vintage data. Again, we forecast real output over the next four quarters, and we find that the forecast is –3.0 percent (that’s a recession forecast, with the economy’s real output declining 3 percent from one year to the next). We continue this way, taking subsequent vintages of our data set one at a time, until we include very recent data, generating a new forecast with each new vintage of data. We call these forecasts real-time forecasts, since they’re based on real-time data. We want to see how different these forecasts are from forecasts generated using today’s data (the latest available data at the time we did our study) instead of real-time data. So we repeat the same exercise, but we use just the data available today in the same type of procedure. To compare these two sets of forecasts, we can plot them against each other to see how different they are (Figure 5). The plot shows the forecasts based on the real-time data on the horizontal axis and the forecasts based on today’s data on the vertical axis. If the forecasts were unaffected by whether we had real-time data, they’d all be 13 12 See the paper by Dean Croushore and Charles Evans for an example of recent research in this area. We’re using a time-series model called an autoregressive model with a four-quarter lag structure. For more details on these methods and the results, see our 1999a paper. 25 BUSINESS REVIEW SEPTEMBER/OCTOBER 2000 FIGURE 5: Two Real Output Growth Forecasts From a Simple Model FIGURE 6: Two Real Output Growth Forecasts From a Complex Model 26 on the diagonal (45-degree) line that’s drawn through the figure. Points on that line are those for which the forecast based on realtime data is identical to the forecast based on today’s data. Though many points are on or near the diagonal, some points are far away from it. Notice, for example, the point that’s far to the left. That point came from the forecast made using real-time data available through May 1975, mentioned above, which forecasts a decline in real output of 3.0 percent. But revisions to the data over time caused the forecast using today’s data to be much different—a 1.3 percent rise. Similarly, the point that’s far to the right was from the forecast for the fourth quarter of 1976; the real-time forecast is for 6.2 percent growth, but the forecast using today’s data is 4.1 percent. Thus, in this simple model, revisions to the one variable being forecast cause the forecasts to diverge, in some cases by significant amounts. A Complex Model with Many Variables. We can confirm the importance of using the real-time data set by performing a similar exercise using a complex forecasting model we’ve developed to forecast seven major macroeconomic variables, including real output, inflation, and interest rates.14 Our tests have shown that this model provides dramatically better forecasts than the simple model used in the previous exercise. Repeating the same type of analysis used FEDERAL RESERVE BANK OF PHILADELPHIA A Funny Thing Happened on the Way to the Data Bank in the simpler model generates forecasts that aren’t affected nearly as much by the choice of data vintage (Figure 6). The forecasts are generally quite close to the diagonal line, so that the real-time forecasts and the forecasts based on today’s data are generally close to each other. Still, the forecasts diverge considerably from each other at certain dates. For example, the point furthest to the right is the forecast for the third quarter of 1976. The real-time forecast is 9.9 percent, but the forecast using today’s data is 7.9 percent. In this model, the divergence between forecasts can arise because of revisions to any or all of the seven variables in the model, so figuring 14 The model is a quarterly Bayesian vector errorcorrections model. For more details, see the paper by Tom Stark. Dean Croushore and Tom Stark out the cause of the differences isn’t easy. Nonetheless, the fact that differences arise indicates that data vintage matters for complex forecasting models as well as simple ones. In both models, forecasts may be sensitive to the vintage of the data being used. For analyzing a new forecasting model, the best data set to use is the real-time data set. SUMMARY The real-time data set has a variety of uses, such as helping us understand how data are revised, testing the robustness of macroeconomic studies, analyzing policy actions and concerns, and developing forecasting models. It’s our intention to keep adding variables to the data set over time and to maintain the data on the Internet for interested researchers. Though developing this data set was not easy, we hope it will prove valuable to economists and policymakers, regardless of their vintage. REFERENCES Blanchard, Olivier Jean, and Danny Quah. “The Dynamic Effects of Aggregate Demand and Supply Disturbances,” American Economic Review 79 (September 1989), pp. 655-73. Croushore, Dean, and Charles L. Evans. “Data Revisions and the Identification of Monetary Policy Shocks,” manuscript, December 1999. Croushore, Dean, and Tom Stark. “Does Data Vintage Matter for Forecasting?” Federal Reserve Bank of Philadelphia Working Paper 99-15, October 1999a. Croushore, Dean, and Tom Stark. “A Real-Time Data Set for Macroeconomists: Does the Data Vintage Matter?” Federal Reserve Bank of Philadelphia Working Paper 99-21, December 1999b. Kydland, Finn E., and Edward C. Prescott. “Business Cycles: Real Facts and a Monetary Myth,” Federal Reserve Bank of Minneapolis Quarterly Review (Spring 1990), pp. 3-18. Landefeld, J. Steven, and Robert P. Parker. “BEA’s Chain Indexes, Time Series, and Measures of Long-Term Economic Growth,” Survey of Current Business (May 1997), pp. 58-68. Stark, Tom. “A Bayesian Vector Error Corrections Model of the U.S. Economy,” Federal Reserve Bank of Philadelphia Working Paper 98-12, June 1998. 27
@FedFRASER