The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.
Working Paper Series
Learning About Consumer Uncertainty
from Qualitative Surveys:
As Uncertain As Ever
WP 15-09
Santiago Pinto
Federal Reserve Bank of Richmond
Pierre-Daniel Sarte
Federal Reserve Bank of Richmond
Robert Sharp
Federal Reserve Bank of Richmond
This paper can be downloaded without charge from:
http://www.richmondfed.org/publications/
Learning About Consumer Uncertainty from Qualitative Surveys:
As Uncertain As Ever∗
Santiago Pinto†
Pierre-Daniel Sarte‡
Robert Sharp§
Federal Reserve Bank of Richmond
August 2015
Working Paper No. 15-09
Abstract
We study diffusion indices constructed from qualitative surveys to provide real-time assessments
of various aspects of economic activity. In particular, we highlight the role of diffusion indices
as estimates of change in a quasi extensive margin, and characterize their distribution, focusing
on the uncertainty implied by both sampling and the polarization of participants’ responses.
Because qualitative tendency surveys generally cover multiple questions around a topic, a key
aspect of this uncertainty concerns the coincidence of responses, or the degree to which polarization comoves, across individual questions. We illustrate these results using micro data on
individual responses underlying different composite indices published by the Michigan Survey
of Consumers. We find a secular rise in consumer uncertainty starting around 2000, following
a decade-long decline, and higher agreement among respondents in prior periods. Six years after the Great Recession, uncertainty arising from the polarization of responses in the Michigan
Survey stands today at its highest level since 1978, coinciding with the weakest recovery in U.S.
post-war history. The formulas we derive allow for simple computations of approximate confidence intervals, thus affording a more complete real-time assessment of economic conditions
using qualitative surveys.
JEL classification: C18; C46; C83; D80; E32; E66
Keywords: Economic Uncertainty; Qualitative Data; Diffusion Index
∗
We are grateful to Mark Watson for helpful comments and suggestions. The views expressed in this paper are
those of the authors and do not reflect those of the Federal Reserve Bank of Richmond or the Federal Reserve System.
All errors are our own.
†
santiago.pinto@rich.frb.org
‡
pierre.sarte@rich.frb.org
§
robert.sharp@rich.frb.org
1
1
Introduction
Quantitative information regarding the state of U.S. economic activity is mostly compiled and
published by various statistical agencies such as the Bureau of Labor Statistics (BLS), the Bureau
of Economic Analysis (BEA), or the Federal Reserve Board. An important aspect of such data
is that releases often involve a one-month lag, and are further subject to later revisions, typically
at the three-month and one-year mark.1 In part as an attempt to provide information somewhat
closer to real time, or to collect information simply not compiled by official statistical agencies,
a growing number of institutions and government agencies produce diffusion indices constructed
from qualitative survey data. For example, the Michigan Survey of Consumers (MSC) provides
several diffusion indices of consumer sentiment based on monthly nationwide surveys; the Institute
for Supply Management publishes different monthly diffusion indices regarding various aspects of
manufacturing, such as production, shipments or new orders; several regional Federal Reserve Banks
including Atlanta, Dallas, Kansas City, New York, Philadelphia, and Richmond produce diffusion
indices meant to capture the direction of change, in real time, of different facets of economic activity,
such as inventories, capital expenditures, or wages, at a more regional level. In the latter cases,
the information is not otherwise compiled or readily available. Moreover, by sharing a common
methodology, diffusion indices allow for comparable benchmarks not just across U.S. regions but
across countries, as in the case of the manufacturing Purchasing Managers Index (PMI) for France
or Germany for instance.
Diffusion indices, throughout the paper, are defined in terms of the proportions of a set of
disaggregated series or survey responses moving in different directions, thus defining a notion of
optimism and pessimism, and provide a measure of the breadth of change in the corresponding
aggregate series. This traditional interpretation is discussed in early work by Moore (1983), or the
Federal Reserve Bulletin (1991) among others, and is distinct from the interpretation of diffusion
indices constructed by principal components analysis in Stock and Watson (2002).
To this point, a large part of the literature on survey-based diffusion indices has focused on their
ability to forecast economic activity using various approaches, as surveyed for example in Nardo
(2003).2 Diffusion indices have also been used to explore different properties of expectations, given
assumptions on the underlying data, as surveyed in Pesaran and Weale (2006).3 More recently,
this second strand of research has been able to take advantage of the micro data on individual
responses underlying diffusion indices, when directly available, as in Bachmann and Elstner (2013)
in the case of the IFO Business Climate Survey.4 Our paper departs somewhat from this literature
1
See Croushore (2011) for a comprehensive survey of real-time data analysis, and the caveats that arise from the
distinction between real-time and revised data. See also Runkle (1998), Croushore and Stark (2001), and Fernald
and Wang (2005).
2
See, for example, Kennedy (1994), Smith and McAleer (1995), Bram and Ludvigson (1998), and Bachmann and
Sims (2012).
3
See also Ivaldi (1992), Jeong and Maddala (1996), and Claveria, Pons, and Suri˜ach (2006).
n
4
A challenge remains, however, in that for many published diffusion indices, individual survey responses are either
2
in that it studies the properties of diffusion indices as estimates of the breadth of change in an
aggregate series of interest. In particular, we highlight the role of the diffusion index as an estimate
of change in a quasi extensive margin, in a way closely related to the work of Gourio and Kashyap
(2007), and characterize the distribution of a general diffusion index, potentially averaging different
individual indices, focusing on the uncertainty implied by both sampling and the polarization of
participants’ responses.
While the use of survey data has traditionally emphasized quantitative surveys, for example
with respect to properties of inflation expectations in the presence of nominal rigidities (Coibion
and Gorodnichenko, 2012; Coibion, Gorodnichenko, and Kumar, 2015), or to measure disagreement
among forecasters as an indication of uncertainty (Boero, Smith, and Wallis, 2008; Bomberger, 1996;
D’Amico and Orphanides, 2008; Rich and Tracy, 2010; Sill, 2012), the use of qualitative surveys
is relatively more recent.5 Specifically, we build on Bachmann, Elstner, and Sims (2013) who use
qualitative survey data capturing notions of optimism and pessimism, through the proportion of
positive and negative responses to particular questions, to construct an empirical proxy for timevarying business level uncertainty. We show, by way of a central limit theorem argument that all
diffusion indices, including composite indices, are asymptotically normal in large samples, and that
the uncertainty proxy in Bachmann, Elstner, and Sims (2013) is the variance of a particular albeit
widely used individual diffusion index (scaled by the square root of the sample size). As such, this
proxy can also be used to provide confidence intervals, taking into account sampling uncertainty,
for the corresponding diffusion index.
In consumer and business tendency surveys, diffusion indices are almost always published
as composite indices, combining multiple individual indices corresponding to different questions
around a given topic. Thus, we emphasize that a general notion of uncertainty based on multiple
questions reflects not only the polarization of answers, or degree of disagreement, with respect to
an individual survey question but also the extent to which agreement or disagreement coincides
across individual questions. In particular, given individual response level data, we describe how the
pairwise coincidence of answers across survey questions, or lack thereof, maps into the covariance
between individual diffusion indices making up the composite index. In deriving the distribution of composite indices, and their associated uncertainty, we show that given data on individual
responses, one need only keep track of pairwise proportions of answer types across questions, for
example the proportion of responses indicating optimism to one question and pessimism to another.
We illustrate these results using micro data on individual responses used to construct different composite diffusion indices around sentiment published by the Michigan Survey of Consumers
(MSC). We find a steadily rising trend, starting around 2000, in consumer uncertainty that contrasts with gradually declining uncertainty over the previous decade, and higher agreement in prior
not systematically recorded or not publicly available.
5
See Barsky and Sims (2012), and Bachmann and Sims (2012), for applications using consumer sentiment indices
constructed by the Michigan Survey of Consumers.
3
periods. Furthermore, following this steady increase, uncertainty arising from the polarization of
responses in the Michigan Survey currently stands, six years removed from the Great Recession,
at its highest level since 1978. The secular rise in uncertainty that we estimate among consumers
can also be seen to a degree with respect to policy in the index of Economic Policy Uncertainty
(EPU) calculated by Baker, Bloom, and Davis (2013). However, while uncertainty around consumer sentiment tends to be countercyclical as with the EPU, the period starting around 2000 is
somewhat unique in that unlike the EPU, which declined sharply from its post-war period peak two
years after the Great Recession, uncertainty among consumers never substantively declined after
the 2007-09 recession. This continued increase in uncertainty among consumers coincides with the
weakest post-war expansion on record.
2
Diffusion Indices: Measuring Changes in a Quasi-Extensive Margin of Economic Activity
Diffusion indices, such as the Index of Consumer Sentiment produced by the MSC, receive widespread
coverage in part because they have been empirically found, over time, to correlate well with economic activity (Bram and Ludvigson, 1998). This section uses the diffusion index of employment, produced by the Bureau of Labor Statistics (BLS), to underscore two key points regarding
diffusion indices. First, scaled appropriately, diffusion indices can be thought of as capturing a
quasi-extensive margin of change in economic activity or, in this case, employment. Second, this
quasi-extensive margin accounts for a large portion of the variation in aggregate employment, although interestingly to different degrees in expansions and recessions.
Beginning in 1991, the BLS began publishing a diffusion index of employment in its monthly
statistical release, covering roughly 264 sectors corresponding to the 4-digit level of the North American Industrial Classification System (NAICS), separate from its release of measured employment.
Overall employment growth, which reflects a weighted average of sectoral employment growth, gives
us one measure of employment performance. Overall employment growth, however, does not provide a sense of how the change is shared across sectors; for instance a given aggregate growth rate
may be consistent with all sectors doing equally well or, instead, a relatively few sectors growing
rapidly with most others simply muddling through. In contrast, the BLS employment diffusion
index summarizes the direction of change in a set of disaggregated sectors over a given time period,
thus providing a measure of the breadth of the change in employment. We now explore and describe
the relationship between these two measures of economic performance.
The diffusion index is most conventionally reported as
µD + κ
(1)
where D is the proportion of a set of disaggregated series that increased over a given time period
less the proportion that decreased over the same period, D = nu /n − nd /n, where n is the total
4
number of series or categories, such as sectors, and nu and nd are the number of categories that
experienced an increase and decline in activity respectively; µ and κ are constants. In the case of
the BLS employment diffusion index, µ = 1 and κ = 0. Thus, if an individual series increases over
the span of the diffusion index, it receives a value of 1; if it declines, it receives a value of −1; and if
it is unchanged, it receives a value of zero. The diffusion index is then calculated by summing these
values for each of the components and dividing the result by the number of series included in the
diffusion index (in this case multiplied by 100). A value of the index above zero is then interpreted
as an expansion in employment and vice versa for values less than zero; a value of 100 would be
indicative of an expansion in all sectors.6
Consider employment in a given sector i and denote its monthly annualized growth rate by
∆xi,t = 1200×ln(xi,t /xi,t−1 ). Because we are mainly concerned with issues related to the assessment
of various economic conditions in real time, the key objective for many of the surveys considered
below, our focus in this paper will be on monthly data. Let n represent the number of sectors
covered by the employment index and denote overall employment growth at t by ∆xt . Then, we
have that ∆xt ≈
1
n
n
i=1 ∆xi,t
or
1
∆xt =
n
nu
t
∆xu
i,t
i=1
1
−
n
∆xu
t
nd
t
∆xd ,
i,t
(2)
i=1
∆xd
t
where ∆xu = ∆xi,t if ∆xi,t ≥ 0, and ∆xd = −∆xi,t if ∆xi,t < 0.7 Simply put, equation (2)
i,t
i,t
distinguishes between those sectors that contribute positively to overall employment growth (up
sectors), which sum to nu and contribute ∆xu , and those that take away from overall growth (down
t
t
sectors), which sum to nd and contribute negatively ∆xd .
t
t
Let µu represent the cross sectional mean growth rate, at a point in time, of the sectors that
t
add to overall growth in employment, µu =
t
1
nu
t
nu
u
t
i=1 ∆xi,t ,
and similarly let µd =
t
1
nd
t
nd
d
t
i=1 ∆xi,t
for
those sectors where employment declined. Then, equation (2) can alternatively be written as
∆xt =
Define µu = T −1
T
u
t=1 µt
nu u nd d
t
µ − t µt .
n t
n
T
d
t=1 µt
and µd = T −1
(3)
as the time averages, or long-run cross-
sectional means, of sectors contributing and subtracting from overall employment growth respectively; we assume that the data is stationary with corresponding population conditional means
µu = Ei,t (∆xi |∆xi ≥ 0) and µd = Ei,t (∆xi |∆xi < 0). We may then express positive contributions
t
t
t
t
6
Note that the information conveyed by the index is invariant to an affine transformation. For example, the Federal
Reserve Board defines the diffusion index of Industrial Production as the proportion of sectors where production
increased plus half the sectors where production was unchanged, in which case µ = κ = 1/2 in equation (1).
7
We define overall growth in employment using uniform weights in this case. Foerster, Sarte, and Watson (2011)
show, in the context of Industrial Production, that the choice of weights, either uniform, constant mean shares, or
time-varying shares, is somewhat unimportant in aggregating growth rates across sectors.
5
to ∆xt as
nu
t
− ϕu µu + ϕu (µu − µu ) +
t
n
∆xu =
t
T
u
t=1 nt /n
where ϕu = T −1
growth.8
∆xu
t
nu
t
− ϕu (µu − µu ) + ϕu µu ,
t
n
(4)
is the long-run average of proportion of sectors that raise overall
In other words, at a point in time, a large increase in overall employment growth via
may arise because the proportion of expanding sectors is higher than usual given their average
nu
t
n
contribution,
− ϕu µu > 0 akin to a rising extensive margin; the cross sectional average of
those positive contributions is higher than usual given the typical proportion of expanding sectors,
ϕu (µu − µu ) > 0 akin to a rising intensive margin; or both to the extent that both are true,
t
nu
t
n
− ϕu (µu − µu ) > 0. Similarly for the negative contributions, we have that
t
∆xd =
t
where ϕd = T −1
nd
t
− ϕd
n
T
d
t=1 nt /n.
µd − µd + ϕd (µd − µd ) +
t
t
nd
t
− ϕd µd + ϕd µd ,
n
(5)
Then, it follows that overall employment growth may be approximated
as
ϕu (µu − µu ) − ϕd (µd − µd )
t
t
∆xt ≈
+
Change in “how much” or intensive margin
where Dt =
nu
t
n
−
nd
t
n
µu D t ,
(6)
Change in “how many” or extensive margin
is the difference in the proportions of increasing and decreasing series
defined earlier.
Put another way, by re-interpreting the scalar µ in (1), we may decompose employment growth
as arising primarily from the change in an intensive margin, the difference between how fast “up”
sectors grew and how badly “down” sectors declined, and the change in a quasi-extensive margin,
the difference between the proportion of sectors that expanded versus those that declined or the
breadth of the expansion, µu Dt . The approximation in (6) comes about in part because the positive
and negative cross-sectional growth rates µu and µd are not necessarily the same, and the fact that
the difference in the way the extensive and intensive margins interact in (4) and (5) also matters in
principle.9 However, as we now illustrate, this difference in interaction terms is largely immaterial
for the behavior of overall employment growth, except in the last recession.
The top panel of Figure 1 shows the decomposition of ∆xu (demeaned) in equation (4) for U.S.
t
employment. Examination of the figure indicates that variations in the proportion of expanding
sectors, conditional on their average contribution, closely tracks the positive contributions to overall
employment growth, ∆xu , while the interaction term between extensive and intensive margins is
t
8
Details of all derivations in the paper are given in online-only supplementary notes, Pinto, Sharp, and Sarte
(2015).
9
The derivation in (6) makes use of the fact that, without loss of generality, µu = µd + ε for some ε
u
0. In the
d
case of employment, µ and µ are close, 3.84 percent versus 3.22 percent respectively over the period 1991-2014, so
that either one may be used to scale the diffusion index to arrive at an approximate measure of extensive margin.
6
mostly unimportant. Figure 1 also illustrates, in the bottom panel, the decomposition of ∆xd in
t
(5). As with the top panel, the variation in the proportion of declining sectors explains a large
fraction of the negative contribution to overall employment growth, although the intensive margin
now also plays an unambiguous role, especially at times when ∆xd spikes during recessions.
t
Figure 2 combines these two decompositions and shows that, to a large extent, variations in overall employment growth arise from changes in the quasi-extensive margin in (6). Put another way,
movements in employment tend to reflect the breadth of change (scaled appropriately) rather than
large variations in cross-sectional growth rates driven by particular sectors. Table 1 summarizes
these results.
Table 1. Decomposition of BLS Employment Growth Rates
Series
∆xt =
1990-2014
1
n
∆xi,t
2.8
Components
D t µu
1.8
ϕu (µu − µu ) − ϕd (µd − µd )
t
t
u − ϕu )(µu − µu ) − (ϕd −
(ϕt
t
t
0.9
ϕd )(µd − µd ) + εϕd
t
t
0.7
Note: Entries are the sample standard deviation of ∆xt and its components.
Percentage points at annual rates.
This observation is reminiscent of Gourio and Kashyap (2007) who show, using plant-level data
from Chile and the U.S., that “the number of establishments undergoing investment spikes (the
“extensive margin”) account for the bulk of variation in aggregate investment.” In that sense, it is
conceivable that variations in the breath of change, Dt , captured in various surveys, say regarding
the business outlook of a sample of purchasing managers, would have proven a useful indicator of
economic activity, even without more “concrete” data at hand. In the case of Figure 2, notable
exceptions concern pronounced downturns in employment that take place during recessions. In
those periods, changes in the intensive margin defined in (6) contribute as much to the downturn
as the extensive margin captured by the diffusion index, and thus so does the interaction of the
two margins, and reveal a potentially interesting asymmetry across expansions and downturns.
It should be noted that the notion of a “quasi extensive margin,” in this context, abstracts from
entry and exit since the number of sectors is held fixed in the BLS calculation of the employment
diffusion index. In the many surveys that produce diffusion indices, including those of the Michigan
Survey of Consumers that we study below, the sample size is also often targeted to a fixed level.
In that sense, diffusion indices are then specifically concerned with the distribution of change in an
existing set of disaggregated series.
7
3
A Formal Description of Survey-Based Diffusion Indices
In the previous section, we saw that the extensive margin captured by BLS’s employment diffusion
index is an important component of overall employment performance. In an attempt to assess
various economic conditions in real time, a number of institutions and government agencies carry
out qualitative surveys that are then similarly translated into a diffusion index. For instance,
the Michigan Survey of Consumers (MSC) constructs widely used composite diffusion indices that
reflect consumer sentiment regarding various current and expected economic conditions, including
household financial conditions, overall business conditions, and spending on durable goods, the
latter being akin to a measure of investment. Similarly, several Federal Reserve Banks construct
and publicly report diffusion indices that are meant to track the breadth of economic performance
in their respective regions in real time. Other diffusion indices that receive widespread attention
by markets and policymakers include the monthly index of manufacturing and service conditions
released by the Institute for Supply Management. Similar to the employment diffusion index
described in the previous section, these surveys most often rely on trichotomous classifications
whereby participants are asked whether conditions are better, worse, or unchanged relative to the
previous month.
3.1
The Individual Diffusion Index and its Cross-Sectional Distribution
Consider a sample of n survey participants drawn randomly from a population at a point in time.
Given the focus on timeliness underlying diffusion indices, the vast majority of indices provide
monthly information. Each participant answers a survey question relating to an economic series of
interest, say household financial conditions or overall business conditions, according to a predefined
set of qualitative responses. We denote the set of possible answer types by A and consider r possible
qualitative responses, A ={1, 2, ..., r}. Answer types from participants are indexed by a ∈ A. In the
most common example, there are 3 types of responses, “up” (u), “down” (d), and “same” (s), or
alternatively “better off”, “worse off,”, or “same.” For example, a question in the Michigan Index
of Consumer Sentiment (ICS) asks: “do you think that a year from now, you (and your family
living there) will be better off, worse off, or just about the same as now?” In that case, we might
write that a ∈ A ={u, d, s}. A typical sample of responses might be summarized by the vector
(u, s, u, d, d, u, ..., ).
(7)
r
a
a=1 n
ω a and ω a
Let na denote the number of respondents associated with answer a ∈ A,
Answers of type a are assigned a value of ω a ∈ R in the diffusion index, where
10
= n.10
denote
In principle, the number of responses of a particular type a (relative to the total number of responses) may
be serially correlated over time. As will be made clear, however, our focus is on characterizing the cross-sectional
distribution, and related uncertainty, that arises from the different survey responses at a point in time, so that we
drop the time subscript. To the extent that the number of responses of each type are serially correlated, to reflect
persistence in participants’ views, so will be the measured uncertainty.
8
lower and upper bounds respectively associated with possible values of ω a . In the conventional
example with 3 categories, ω u = 1, ω s = 0, and ω d = −1. An alternative formulation might
have ω u = 1, ω s = 1/2, and ω d = 0, as with the Board of Governors diffusion index of industrial
production.
The general diffusion index statistic is then given by11
r
ωa
D=
a=1
na
.
n
(8)
By construction, the resulting index will range from ω a to ω a , with numbers above (1/r)
r
a
a=1 ω
generally being interpreted as an expansion in the condition of interest, say household spending
on durable goods. Observe that different combinations of answers can result in the same index.
For example, in the case comprising 3 categories, ω u = 1, ω s = 0, and ω d = −1, and 100 firms
being asked about overall business conditions, a value of D = 0 might emerge from having 50 firms
responding “up” and 50 firms responding “down,” or all firms responding “no change.” While both
cases may be interpreted as a reading of unchanged business conditions overall, we describe below
the sense in which these two cases imply a different degree of confidence in the overall reading of
“no change.”
Let pa denote the probability, in a given month, that a participant’s answer is a ∈ A =
{1, 2, ..., r}, with
r
a
a=1 p
= 1. The survey process, according to which n participants are drawn
randomly from a population, has a natural interpretation in terms of n independent trials, each of
which leads to a success for exactly one of r types of responses, with each answer type having a
given fixed success probability, pa . The multinomial distribution then gives the probability, given
n, of observing any particular combination of numbers of responses, {n1 , n2 , ..., nr }, for the various
categories {1, ..., r}. In particular, the probability mass function of this distribution is
f (n1 , ..., nr ; n, p1 , ..., pr ) =
a
n!
Πr (pa )n
r
a ! a=1
Πa=1 n
,
(9)
where the expected number of answers of type a is E(na ) = npa , with associated variance V ar(na ) =
npa (1 − pa ). The covariance between the numbers of answers of types a ∈ A and a ∈ A is given
by Cov(na , na ) = −npa pa . Observe that E(na ) and V ar(na ) in this case are respectively the mean
and variance of a Binomial distribution defined by the marginal distribution, for answers of type
a, of the multinomial described in (9).
Let pa = na /n denote the proportion of answers of type a ∈ A observed in a given survey. We
can alternatively write pa as
pa =
11
1
n
n
xa ,
i
(10)
i=1
A constant is sometimes added to D, as in equation (1), to correct for sample design changes but this modification
is immaterial for questions related to uncertainty.
9
where xa is an indicator variable that takes on the value 1 when survey participant i answers a and
i
is zero otherwise. Since pa then has the interpretation of a sample Bernoulli mean, the Multivariate Central Limit Theorem, combined with the fact that sums of Bernoulli random variables are
binomial, immediately gives that,
p1 − p1
√ p2 − p2
→ N
n
D
...
pr−1 − pr−1
0
...
−p1 pr−1
p2 (1 − p2 ) ...
p1 (1 − p1 )
−p2 pr−1
. (11)
−p1 p2
0 −p2 p1
,
...
...
r−1 p1
−p
0
...
...
−pr−1 p2
...
...
pr−1 (1
− pr−1 )
Moreover, because the generic individual diffusion index, D, is a linear combination of sample
r
a a
a=1 ω p
Bernoulli means,
, that are asymptotically jointly normally distributed according to
(11), the distribution of D will also be asymptotically normal.12 Thus, in large finite samples, the
distribution of the diffusion index statistic is approximately normal and given by
√
r
n D − D ∼a N
(ω a )2 pa
0,
− D2
,
(12)
a=1
r
a a 13
a=1 ω p .
where D = E(D) =
Given equation (12), it immediately follows that confidence intervals for D are symmetric and
r
a 2 a
a=1 (ω ) p
can be approximately constructed as D ± z
− D2 /n, where z is the standard
score corresponding to a critical value of interest. Confidence intervals, therefore, reflect in part
sampling uncertainty, which decreases with the square root of the sample size, n, and the degree
of polarization among survey respondents’ answers,
r
a 2 a
a=1 (ω ) p
− D2 . To the degree that
the weights, ω a , or the number of responses, r, determine the scale of D, the standard deviation
r
a 2 a
a=1 (ω ) p
− D2
then preserves the same units. The following examples provide some
intuition.
3.1.1
Practical Applications and Special Cases
In many common applications, such those regarding any question making up the MSC’s Index of
Consumer Sentiment (ICS), we have that A ={u, d, s}, and ω u = 1, ω s = 0, ω d = −1. Then,
D = (pu − pd ) and
√
n D − D → N 0, (1 − ps ) − D2 .
D
(13)
In this example, the variance of the diffusion index is also, in large samples, the population equivalent of the uncertainty proxy estimated in Bachmann, Elstner, and Sims (2013). We make two
12
13
Note that pr is simply defined as a residual, pr = 1 − r−1 pa , and thus need not be included in equation (11).
a=1
A derivation of this result is given in Appendix A1. Observe that, in principle, different functions of these
probabilities might be of interest, including, for instance, measures of concentration such as that captured by the
Herfindahl index,
2
i
pi , whose distribution might be characterized following the same steps.
10
observations to provide intuition regarding the variance of D. First, it decreases as responses become less polarized, or alternatively split between extremes. In particular, the variance in (13)
decreases with the magnitude of the diffusion index itself, D2 = (pu − pd )2 ; simply put, the only
way for D2 to be large, or more specifically to approach 1, is for all respondents to answer either
“up” or “down,” in which case answers coincide and there remains little room for uncertainty,
ps = 0 and (1 − ps ) − D2 = 0. Alternatively, consider the case where where “up” and “down”
responses are evenly split, pu = pd so that D = 0 irrespective of ps . Then uncertainty decreases
as ps rises (and both pu and pd fall while D remains unchanged); in other words, we are relatively
more confident of a reading of no change, D = 0, when everyone reports “no change,” ps = 1 and
all answers coincide, than when half the respondents report “up” and half the respondents report
“down,” ps = 0 (the largest even split). Second, since D derives from a weighted sum of means, its
variance simply decreases at rate n.
As another example, consider the Federal Reserve Board’s diffusion index of Industrial Production, where A = {u, d, s}, ω u = 1, ω s = 1/2, ω d = 0, and D = pu + (1/2)ps . In that case,
√
n D−D →N
D
1
D, pu + ps − D2 .
4
As before, the standard deviation of the index decreases as responses become less polarized; when
either pu or pd is 1, ps = 0 and pu + 1 ps − D2 = 0.
4
At a more granular level, diffusion indices may, in some cases, take into account different
categories or groups of respondents. For example, participants may be reporting from different
sectors, j = 1, ..., J,
J
r
ωa
D=
j=1 a=1
na
j
,
n
where na denotes the number of responses of type a from group j with
j
(14)
J
a
j=1 nj
= na . This last
expression may be equivalently written as
J
D=
j=1
nj
n
r
ωa
a=1
na
j
,
nj
Dj
where
J
j=1 nj
= n, so that the overall diffusion index may be interpreted as a weighted sum of
group-specific diffusion indices, Dj , with weights
nj
n,
where nj is the total number of responses
obtained from group j. In principle, one may choose to scale those weights either up or down to
emphasize particular target groups relative to the number of responses obtained in those groups,
say by γj ,14
14
For instance, the weights might ultimately reflect sectoral shares in value added or gross output. In practice,
membership in the ISM Business Survey Committee is in fact initially based on each industry’s contribution to GDP.
11
J
D=
j=1
nj
γj
n
r
ωa
a=1
na
j
.
nj
How then does this alternative weighting scheme affect the computation of confidence intervals?
The answer depends not only on the alternative scaling, γj , but also the more granular information
captured in different groups by way of pa , the probability of observing answer type “a” in group
j
j in the overall survey, where
r
a=1
J
a
j=1 pj
= 1. In particular, letting pa = na /n denote the
j
j
proportion of answers of type a ∈ A observed in sector j in the overall sample, we have that
r
J
√
n D − D ∼a N 0,
(ω a γj )2 pa − D2 ,
(15)
j
a=1 j=1
where D =
4
r
a
a=1 ω
J
a 15
j=1 γj pj .
Composite Indices and their Distribution: Measuring Uncertainty Using Multiple Diffusion Indices
In practice, very few, if any, diffusion indices are reported as individual indices but rather are
reported as composite indices, constructed as weighted averages of individual component indices,
described in the previous section, across different categories. For example, the headline Michigan
Index of Consumer Sentiment combines together information from individual indices with respect to
five categories: current and expected household financial conditions, current and expected overall
business conditions, and spending on big ticket items or durable goods. To the extent that each
series is associated with some degree of uncertainty, arising from both sampling and fundamental
polarization of survey answers, the uncertainty associated with the overall composite index will in
turn depend on the extent to which this polarization comoves across individual component indices.
In characterizing the approximate distribution of diffusion indices, one objective throughout
the paper is to highlight the nature of the micro data needed to measure uncertainty in the survey
responses, and thus how to organize and maintain incoming individual survey answers. As we will
see, this involves keeping track of pairwise proportions of answer types across different questions.
Intuitively, since composite indices are weighted averages of individual indices, each providing
feedback on a particular question, the uncertainty surrounding the composite index will reflect
not only the uncertainty associated with individual component indices but also the covariances
between them. For example, survey participants responding “up” to a particular category, such
as their household financial condition, may be more or less likely to also respond “up” to another
category, such as spending on durable goods, and this degree of comovement may change as the
state of the economy changes. Confidence intervals, therefore, must be adjusted accordingly. The
next section formalizes these ideas.
15
See appendix A2.
12
4.1
Composite Diffusion Indices
To describe composite diffusion indices, in a way that builds directly on the intuition presented
thus far, we now consider a sample of n survey participants responding to questions concerning k
economic conditions of interest. In practice, these might include financial conditions, employment
conditions, etc. Answers from each participant corresponding to conditions of a particular component, k, are indexed by ak , confined to a set Ak , each comprising r possible types of responses,
{1, 2, ..., r}. For example, A1 might describe whether conditions are “better off,” “worse off,’ or
“unchanged” in a given month, with the subscript “1 ” identifying the category “household financial
conditions,” and a1 ∈ {u, d, s}, and similarly the subscript “2” in A2 might denote “overall business
conditions.”
A survey participant’s answers across all components, k = 1, ..., k, are collected in a k−tuple
a = (a1 , ..., ak ) that lives in the set A = Πk Ak . In our simple example with 3 components, each
k=1
comprising 3 possible responses, an example of a might be (u, u, u), indicating that conditions are
“up,” meaning improving, in household financial conditions, overall business conditions, and, say,
spending on durable goods. In this case, A = A1 × A2 × A3 = {u, d, s} × {u, d, s} × {u, d, s} =
{(u, u, s), (u, u, d), (u, u, s), (d, u, u), (d, d, u), (d, s, u), (s, u, u), (s, d, u), (s, s, u), ...} has 27 elements.
In general, A will have dimension rk .
It will be convenient below to distinguish between answers for a given component, k, and
those for all other components, which we denote by (ak , a−k ), where a−k ∈ A \ Ak . We let na
denote the number of respondents associated with answers a ∈ A, where
that given our notation,
na
may also be expressed as n
(ak ,a−k )
. We let
na
k
a∈A n
a
= n. Observe
denote the number of
responses associated with answer ak ∈ Ak for component k summed across all other components,
na =
k
a−k ∈A\Ak
n(ak =a,a−k ) . For example, in the Michigan ICS, nu might be the number of
2
respondents reporting improving (u) overall business conditions (indexed by the subscript “2”),
where nu =
2
a−2 ∈A\A2
n(u,a−2 ) includes all those reporting “up” on business conditions irrespective
of their answers to other questions.
As before, answers of type ak ∈ Ak for component k are assigned a value of ω a ∈ R independently of k. Then, an individual component diffusion index, Dk , associated with category or
question k is given by
r
Dk =
ω
a=1
a
a−k ∈A\Ak
n(ak =a,a−k )
=
n
r
ωa
a=1
na
k
,
n
(16)
and the composite index is averaged over all k components,
k
D=
δk Dk ,
(17)
k=1
where 0 < δk < 1. Note that the weights, δk , do not necessarily sum to 1, as in the Michigan ICS
which gives each of five questions an equal weight, and uses this weight to normalize the overall
index to a base year.
13
4.2
The Distribution of Composite Diffusion Indices
Let the probability of drawing answers a = (a1 , ...ak ) ∈ A be denoted by pa , with
a∈A p
a
= 1,
where pa can also be expressed as p(ak ,a−k ) . Thus, we denote the marginal probability of drawing a
given response a ∈ Ak for component k as pa =
k
a−k ∈A\Ak
p(ak =a,a−k ) . Because the composite index
(17) averages across different component indices, say Dk and D , we will need to take into account
the pairwise covariance between components. We now describe how this process in turn relies
on keeping track of all pairwise joint probabilities between any two components of the composite
diffusion index.
Denote the joint probability of observing ak = a and a = a for the components k and
paa
k
by
. This probability is given by
paa =
k
p(ak =a,a
=a ,a−{k, } )
, k= ,
(18)
a−{k, } ∈A\Ak ×A
where the notation (ak = a, a = a , a−{k, } ) distinguishes between answers for component k,
component , and all other components, −{k, }. The marginal with respect to component k
satisfies pa =
k
a ∈A
paa . We let pk denote the vector comprising all pairwise joint probabilities,
k
paa for given components k and , where the dimension of pk is r2 . Thus, for the example with 3
k
components and 3 possible responses, the element pud in p13 gives the joint probability of observing
13
“up” along dimension “1,”or improving household financial conditions, and “down” along dimension
“3,”or deteriorating spending on durable goods.
Denote the number of survey participants answering ak = a and a = a for the components k
and
by naa , where similarly to equation (18),
k
nk =
aa
n(ak =a,a
=a ,a−{k, } )
, k= ,
a−{k, } ∈A\Ak ×A
As before, the number of participants answering a given response a for component k satisfies
na =
k
Let
a ∈A
pa =
k
naa .
k
na /n in the component index (16) and paa = naa /n. Observe that
k
k
k
r
r
ω a pa =
k
Dk =
a=1
ωa
a=1
for any component , where the components other than
paa ,
k
(19)
a ∈A
have already been integrated out in (18).
Equation (19) effectively allows us to write each component index, Dk , in the overall index, D, in
terms of joint pairwise probabilities with any other component index, D , and, therefore, capture
the pairwise comovement across these indices.
As in section 2, each element paa , which we collect in the vector pk , may be interpreted as the
k
sample mean of Bernoulli random variables so that
√
n(pk − pk ) → N (0, Σpk ).
D
14
(20)
As shown in Appendix A.3, the information contained in (20), constructed for all pairs of questions
k and
in the survey, is all that is needed to construct sampling uncertainty around a composite
index that combines different individual indices.16 A typical element of pk , say pab , is such that
k
E(pab ) = pab , V ar(pab ) = n−1 (1 − pab )pab , and its covariance with any other element pb a is
k
k
k
k
k
k
Cov pk , pk a
ab b
√
b
= −n−1 pab pk a . In particular, we have that
k
k
n D − D ∼a N 0,
2
δk V ar Dk + 2
k=1
δk δ Cov Dk , D ,
(21)
1≤k< ≤k
where
k
r
D=
ω a pa
k
δk
a=1
k=1
V ar Dk =
and
Cov Dk , D
1
n
r
(ω a )2 pa
k
,
(22)
a=1
1
=
n
ω a ω a paa −
k
b,b ∈Ak ×A
(a,a )∈Ak ×A
5
− (Dk )2
pab pb a .
k k
(23)
Diffusion Indices in Practice
We now illustrate what the results derived in sections 3 and 4 imply in practice. First, we revisit the
BLS employment diffusion index highlighted earlier as an example of an individual diffusion index.
We then consider the historical behavior of consumer uncertainty, by way of composite diffusion
indices, using micro data on individual survey responses published by the Michigan Survey of
Consumers (MSC). In either case, measures of uncertainty associated with the index variance tend
to fall the more a given direction of change is shared across sectors, or the more answers to given
survey questions agree, and rise as sectoral performance or survey responses become more polarized.
The top panel of Figure 3 reproduces the quasi-extensive margin contribution to employment
growth depicted in Figure 2, along with 95 percent confidence intervals. Since the distribution in
(12) is Normal, it is straightforward to construct confidence intervals for this measure, shown as
the solid blue lines around µD in the top panel. Moreover, since the BLS considers 264 sectors,
the sampling error associated with this quasi-extensive margin component is relatively small. The
√
bottom panel of Figure 3 shows the standard deviation of the BLS (scaled by n) extensive margin
component of employment growth in (12), along with its Hodrick-Prescott (HP) trend and with
the recessions shaded in gray. Observe that from 1990 to 2000, this measure of uncertainty, or
16
Observe that the elements of pk sum to 1, in that pk can be used to define a marginal multinomial distribution
obtained by integrating out all categories other than k and
from the underlying primitive mutlinomial distribu-
tion over all categories (i.e. marginal distributions constructed from integrating out dimensions of a multinomial
distribution remain multinomial).
15
polarization, tends to move opposite the index itself, as is typical of other conventional measures
of uncertainty. As employment performance improves in the mid-1990s, the improvement also
becomes more widespread across sectors and uncertainty falls; similarly, as overall employment
starts declining towards the 2001 recession, it also becomes more polarized with some sectors
holding up while others lose employment.
In sharp contrast, however, the Great Recession stands out in that not only does BLS employment growth reach an all time low during this period, the breadth of the decline is also unprecedented, with the degree of polarization thus also reaching an all time low. In that sense, the
Great Recession was not only particularly severe along the intensive margin but also particularly
widespread. One key aspect of the notion of uncertainty emphasized here, as measured by the
variance of diffusion indices, is precisely that it is tied to the breadth of change; we become more
or less confident in a given change as it becomes more or less widespread. Survey data, when they
are purely qualitative, lack the intensive margin - for example, how strongly respondents might
feel about overall business conditions. However, as we now show, we may nevertheless estimate a
notion of uncertainty around sentiment based on how widely it is shared.
5.1
The Michigan Survey of Consumer Sentiment
The consumer survey carried out by the Survey Research Center at the University of Michigan
publishes 3 composite diffusion indices, the Index of Current Conditions (ICC), the Index of Consumer Expectations (ICE), and the headline Index of Consumer Sentiment (ICS), from a sample
of between 400 and 600 monthly interviews on average regarding household and overall conditions.
Specifically, the composite indices are derived from the following 5 questions:
Q1) “We are interested in how people are getting along financially these days. Would you
say that you (and your family living there) are better off or worse off financially than you were a
year ago, or just about the same as now?”
Q2) “Now looking ahead – do you think that a year from now you (and your family living
there) will be better off financially, or worse off, or just about the same as now?”
Q3) “Now turning to business conditions in the country as a whole – do you think that
during the next twelve months we’ll have good times financially, or bad times, or what?”
Q4) “Looking ahead, which would you say is more likely – that in the country as a whole
we’ll have continuous good times during the next five years or so, or that we will have periods of
widespread unemployment or depression, or what?”
Q5) “About the big things people buy for their homes – such as furniture, a refrigerator,
stove, television, and things like that, generally speaking, do you think now is a good or bad time
for people to buy major household items?”
16
A somewhat unique feature of the MSC composite indices, relative to other published diffusion
indices, concerns the individual response level data underlying the index calculations, which are
made readily available through the Survey Center’s online archives starting in 1978. In particular,
from the archives, it is possible to obtain response-level data that allow keeping track, in each
month, of a given respondent’s answers to each question making up the various MSC composite
indices. Given this level of detail, it is then possible to construct, in any given month, all pairwise
responses, naa , and proportions, paa , where k denotes any pair of questions from the set of
k
k
questions Q1 through Q5 listed above, and aa any pair of answers, for example “better off,” and
“good times.”
5.1.1
The Michigan Index of Current Conditions
To calculate the Index of Current Conditions (ICC), the MSC first computes the diffusion indices,
or the proportion giving favorable replies less that giving unfavorable replies (plus 100) for questions
Q1 and Q5 listed above, denoted D1 and D5 respectively. Each individual index is rounded to the
nearest whole number. The ICC is then calculated as:
ICC =
D1 + D5
+ 2,
2.6424
(24)
where the denominator in the above expression establishes a base period (1966), and the constant
corrects for sample design changes in the 1950s.17 Observe that D5 summarizes a direction of change
in households’ current attitudes towards the purchase of large ticket items or durable goods. As
such, it is closely linked to households’ attitudes towards investment. The individual index D1
summarizes instead the direction of change in the state of households’ finances, which likely has
a direct bearing on their “ability and willingness to buy,” which is used in part by the MSC as a
working definition of consumer confidence.
The top panel of Figure 4 shows the historical behavior of the standard deviation of the ICC
√
(times n), as given by equation (21), along with its HP trend and the recessions shaded in gray.
Beginning in the early 1990s, the uncertainty reflecting the degree of polarization in responses
underlying the ICC begins a decade-long decline to its lowest point in the sample, around 1999.
This decade-long decline in uncertainty corresponds to one of the strongest expansions in postwar U.S. economic history, as measured in part by consumption growth, ending with the dot-com
bust and the start of the 2001 recession. Prior to the 2001 recession, the degree of polarization
in the ICC begins to rise as consumers increasingly disagree on the questions related to current
conditions. As in Bachmann, Elstner, and Sims (2013), and Baker, Bloom, and Davis (2013), spikes
in disagreement prior to the recessions of 1991, 2001, and 2007 are clearly visible. In principle,
however, disagreement about the state of current conditions would not necessarily have to rise
during recessions since it may be generally recognized, and agreed upon, by consumers that the
17
To the best of our knowledge, the micro or response level data is publicly available only since 1978.
17
state of the economy is poor, or more specifically in this case that it is not a good time to purchase
durable goods.
The recovery from the 2001 recession was relatively subdued relative to other post-war U.S.
recoveries, especially where employment is concerned, and the recovery from the 2007 recession
is widely known to be the weakest on record from a number of standpoints, most notably per
capita GDP growth. Interestingly, the top panel of Figure 4 indicates that throughout the 2001
and 2007 recessions, and the recovery in between, uncertainty in consumers’ views regarding the
state of current conditions, at least as captured by the MSC, continued to rise steadily. Moreover,
the degree of polarization in answers to questions regarding current conditions remained noticeably
flat in the nearly six years that followed the Great Recession, although disagreement has recently
started to fall somewhat.
Remarkably, the level of uncertainty in the ICC today, almost six years removed from the most
recent recession, is comparable to that which emerged during the twin recessions of the 1980s
and the period immediately surrounding the 1991 recession. The standard deviation of the ICC
(normalized by
n) currently stands around 20 percent higher than in 1999, at which time it began
to generally rise to the level we see today. In fact, the period since 2000 is notable relative to other
post-war recoveries in the sample in that uncertainty unambiguously declined during the periods of
pronounced economic expansion of the late 1980s and 1990s. These observations are suggestive, as
argued by Bloom (2014) and others, that the state of economic activity is intrinsically linked to the
degree of uncertainty households perceive, not only at business cycle frequencies but as shown here
at longer frequencies as well. As with other measures of uncertainty in the literature, uncertainty
in the ICC, in spite of currently being at a near all time high 6 years into an expansion, tends to
be countercyclical, falling in expansions and rising in and around recessions.
The bottom panel of Figure 4 illustrates the decomposition of uncertainty in the ICC in terms
of the variances of the individual diffusion indices associated with questions Q1 and Q5 in the MSC,
and the covariance between them (adding the series in the bottom panel, and dividing by 2.64242 ,
gives the square of the series in the top panel). The main driver of uncertainty or variance in the
ICC is the variance of the diffusion index associated with question Q5 in the MSC; the question
related to consumers’ perception of whether it is a good or bad time to purchase major household
items or durable goods. The degree of polarization in answers to that question has tended to rise
between 1999 and 2009, and has come down somewhat since then. However, observe that since
the end of the most recent recession, the covariance between the diffusion indices D1 and D5 has
continued to rise slightly, a trend that began after the 2001 recession, indicating rising coincidence
in answers to questions Q1 and Q5 over this period. In other words, the extent to which consumers’
answers to questions Q1 and Q5 have tended to locate in the extremes, good or bad rather than
middle, has gradually become more consistent across questions. As described in section 4, in the
case of composite diffusion indices, coincident polarization across questions increases uncertainty
as captured by the variance of the composite index.
18
5.1.2
The Michigan Index of Consumer Expectations
The Michigan Index of Consumer Expectations (ICE) is based on diffusion indices that summarize
households’ perceptions of times ahead, both for themselves and the country as a whole, over
different time horizons, one and five years out. In particular, the ICE is given by
ICE =
D2 + D3 + D4
+ 2,
4.1134
where D2 , D3 , and D4 are the diffusion indices associated with questions Q2, Q3, and Q4 listed
above respectively. As with any survey, the questions are potentially subject to varying interpretations by consumers, and the level of polarization in answers might then also vary considerably over
time. As we now illustrate, however, uncertainty in the ICE tends to be remarkably stable throughout the 1980s and into the 1990s, before starting a steady rise that is even more pronounced with
respect to expectations than that shown with respect to consumers’ uncertainty around current
conditions.
The top panel of Figure 5 shows the level of the ICE published by the MSC, and the bottom
panel uncertainty in the ICE implied by the standard deviation in equation (21). Although more
involved than in the case of the individual diffusion index, the distribution of composite indices
in (21) remains Normal, so that constructing confidence intervals for the level of the ICE remains
conceptually straightforward. These are shown as the solid blue lines around the MSC index
in the top panel. However, as we address in more detail below, the calculations now entail the
consideration of multiple pairwise objects, pab , across answers and survey questions.
k
Observed first that the top and bottom panels of Figure 5 paint distinctly different pictures;
one provides a measure of the direction of change in households’ perception of times ahead, while
the other is indicative of the level of polarization in those perceptions. Starting in the early 1990s,
the ICE begins to climb steadily, in the top panel, as the U.S. economy enters one of its longest
expansion periods and households contemporaneously grow more optimistic about the future. At
the same time, as with the ICC, consensus that the next one to five years are likely to be “good
times” also grows among households and the level of polarization in answers falls in the bottom
panel. With the arrival of the 2001 recession, the ICE experiences a significant dip and levels off,
while uncertainty among consumers about what to expect begins to increase dramatically. As the
2007 recession starts, the ICE falls further and remains at relatively subdued levels throughout
the subsequent recovery period. At the same time, uncertainty about the next one to five years
generally continues to increase and polarization among consumers stands today, six years into the
weakest recovery of the post-war period, more than 20 percent above its 1999 level. Observe that,
in contrast to the ICC, the 1991 and 2007 recessions are associated with pronounced downward
spikes and a general pessimistic consensus about the future. In terms of trend, however, uncertainty
about what lies ahead conveyed by consumers remains relatively stable throughout the 1980s and
up to the mid-1990s. The period since 2000, in contrast, is one of striking rising uncertainty
in consumers’ perception of the future, indicating today a level of polarization in households’
19
expectations unprecedented since 1978.
Figure 6 shows the decomposition of uncertainty in the Michigan ICE. The top panel of Figure
6 illustrates the behavior over time of the variances (scaled by n) associated with the individual
diffusion indices, D2 , D3 , and D5 , while the bottom panel shows the behavior of the pairwise
covariances. In the top panel, we see that the level of polarization in answers to question Q2,
regarding personal finances one-year ahead, is remarkably constant throughout the entire sample.
Uncertainty regarding the national outlook, however, at both the one and five year horizon, sees
more variations over time. In the bottom panel, all covariances experience a generally increasing
tendency starting in 2000, but this tendency is especially pronounced where the comovement between answers to questions Q3 and Q4 are concerned. Recall that questions Q3 and Q4 in the
MSC differ mainly with respect to the time horizon over which households are asked about their
expectations of the national outlook. Put another way, households’ answers regarding the country
as a whole grow more coincident between the one and five year horizon. This finding suggests that
with the onset of the 2001 recession, households begin to perceive the state as more persistent,
regardless of whether it is good or bad. This more coincident polarization of expectations across
the one and five year horizon in turn is a key driver of the rising uncertainty in the ICE in the
bottom panel of Figure 6. In this case, the covariation in the degree of disagreement across different
questions of a qualitative survey plays a substantive role in the determination of overall uncertainty
conveyed by the survey.
Figure 7 illustrates the more coincident polarization of responses with respect to the one and
five year expectations of overall business conditions by plotting the various elements that make up
the covariance between the diffusion indices associated with questions Q3 and Q4, Cov D3 , D4 .
In particular, in this case, the expression in (23) reduces to
Cov (D3 , D4 ) = (puu + pdd ) − (pud + pdu ) − D3 × D4 .
34
34
34
34
(25)
Intuitively, the more responses to questions Q3 and Q4 in the MSC coincide, or the larger puu and
34
pdd , the more D3 and D4 tend to comove, while the reverse is true the more answers disagree across
34
questions, or the larger pdu and pud . Furthermore, analogously to the squared term in the equation
34
34
describing the variance of individual diffusion indices (22), the more one sided responses become
in the same direction, or alternatively as both D3 and D4 approach either 1 or −1, the less room
there is for the indices to comove.
In the case of questions Q3 and Q4 in the MSC, the top panel of Figure 7 shows the behavior
of the various proportions making up equation (25). First note that the terms capturing the
coincidence in responses, puu and pdd , tend to dominate relative to the terms reflecting disagreement,
34
34
pud and pdu , and are noticeably procyclical and countercyclical respectively. Second, in the bottom
34
34
panel of Figure 7, we see that the rise in comovement between D3 and D4 , depicted in the bottom
panel of Figure 6, indeed arises from increasing coincidence, and decreasing disagreement, among
responses concerning overall business conditions at the one and five year horizons. In that sense,
20
survey responses indicate that participants increasingly see the state one year ahead as persisting
into a five year horizon.
5.1.3
The Michigan Index of Consumer Sentiment
The Michigan headline Index of Consumer Sentiment (ICS) summarizes the direction of change
in consumer feedback by combining both their assessment of current conditions, which drives the
ICC, and their expectations, which drives the ICE,
ICS =
D1 + D2 + D3 + D4 + D5
+ 2.
6.7558
Given the historical behavior of uncertainty in the ICC and ICS, it is not surprising to see, in the
bottom panel of Figure 8, uncertainty in the ICS begin to rise dramatically around 2000, and again
following the 2007 recession, to reach an unprecedented level over our sample period. Uncertainty
√
in the ICS today, as measured by the standard deviation of the ICS (times n), is approximately
25 percent higher than throughout the 1980s and 1990s. Observe, in particular, that as consumer
sentiment, depicted in the top panel of Figure 8, has steadily risen since the end of the last recession,
so has the degree of polarization of responses reflected in the standard deviation of the ICS in the
bottom panel. Consumers, therefore, while more confident are also more polarized in their view.
Other indices frequently used in the literature to measure uncertainty include the Chicago
Board Options Exchange Market Volatility Index (VIX) and, more recently, the Economic Policy
Uncertainty (EPU) index developed by Baker, Bloom, and Davis (2013).18 Both indices are shown
in Figure 9. Note that the secular behavior of ICS uncertainty, in the bottom panel of Figure 8,
and that of the EPU is remarkably similar, with the EPU also reaching its peak well after the end
of the Great Recession in 2012. However, the key difference between ICS uncertainty and the EPU,
and indeed the VIX, concerns the current period. Both the EPU and VIX are now declining, with
the decline in the VIX starting immediately after the Great Recession, whereas ICS uncertainty
in Figure 8 has steadily risen throughout the post 2007 recession recovery, the weakest post-war
recovery on record.
We end this section by highlighting the fact that, in characterizing the full distribution of
general composite diffusion indices, the formulas derived in (21) and (23) may be used to produce
confidence intervals around any diffusion index estimate. In the case of the Michigan Survey of
Consumers, Table 2 summarizes the margins of error associated with 95 percent confidence intervals
for its different indices averaged over different sample periods.
18
This VIX captures the stock market’s expectation of volatility over the next 30 days, and is constructed using
a series of options included in the S&P 500 index. The EPU intends to capture uncertainty in economic policy
and consists of three components. The first component is based on media coverage of economic policy uncertainty.
The second tracks federal tax provisions that are set to expire in the next few years. And the third one captures
disagreement between economic forecasters.
21
Table 2. Consumer Surveys: 95 Percent Confidence Intervals
Series
1978-1999
2000-2014
1978-2014
Index of Current Conditions
±4.19
±4.58
±4.34
Index of Consumer Expectations
±4.02
±4.49
±4.20
Index of Consumer Sentiment
±3.29
±3.71
±3.46
Note: Entries are time averages of ±1.96
k
2
k=1 δk V
ar Dk + 2
1≤k< ≤k δk δ
Cov Dk , D
/n.
As shown in the top panels of Figures 5 and 8, with around 500 respondents, these margins
are relatively tight compared to the span of the indices, although variations within those margins
are not infrequently taken up or discussed as meaningful changes in direction.19 The Institute
for Supply Management, and the Philadelphia Business Outlook Survey, do not publicly report
margins of errors, and indeed few if any of the most widely published diffusion indices do.
6
Concluding Remarks
In this paper, we thoroughly examine the property of diffusion indices defined as estimates of the
breadth of change in an aggregate series of interest. The analysis highlights, in the first place, the
relevance of diffusion indices in capturing changes in the extensive margin. We show, for instance,
that the BLS’s employment diffusion index explains the most part of overall employment growth.
Next, we characterize the distribution of general composite diffusion indices, defined as the
weighted sum of individual indices based on responses to different individual survey questions. We
show that diffusion indices are asymptotically normal, and that the uncertainty proxy in Bachmann,
Elstner, and Sims (2013) is the variance of a particular albeit widely used individual diffusion index
(scaled by the square root of the sample size). This proxy, which takes into account the uncertainty
implied by both sampling and the polarization or disagreement of participants’ responses, can
be used to construct confidence intervals for the diffusion index. Our approach reveals that a
general notion of uncertainty based on composite indices reflects both the degree of disagreement
with respect to an individual survey question, and the degree of disagreement across individual
questions. In particular, we show that only pairwise proportions of answer types across questions
are relevant to derive the variance of composite indices.
Finally, we use micro data published by the Michigan Survey of Consumers to illustrate our
results. We find that starting in 2000, consumer uncertainty, calculated using our approach, steadily
19
Specialized media follows closely changes in consumer sentiment. In the wake of the government shutdown of
2013, for example, the Wall Street Journal reported that “U.S. consumers turned less optimistic about the economy in
early October, according to data released Friday. The Thomson-Reuters/University of Michigan preliminary October
sentiment index slipped to 75.2 from an end-September level of 77.5, according to an economist who has seen the
numbers.” In this case, the index decreased only 2.3 points in anticipation of the government shutdown, which
statistically did not indicate a change in consumer sentiment.
22
increases contrasting with the gradual decline in uncertainty observed in the previous decade,
and higher agreement in prior periods. Furthermore, uncertainty arising from the polarization of
responses in the Michigan Survey currently stands, six years removed from the Great Recession,
at its highest level since 1978. Relative to the index of Economic Policy Uncertainty (EPU), and
although consumer uncertainty generally shows similar secular fluctuations throughout the period
under consideration, the series tend to diverge in recent years. While the EPU declined sharply
from its post-war period peak two years after the Great Recession, consumer uncertainty has tended
to increase since 2009 recession, reaching the highest level on record.
References
Bachmann, R¨diger and Steffen Elstner (2013). “Firms’ Optimism and Pessimism”. Working Paper
u
18989. NBER.
Bachmann, R¨diger, Steffen Elstner, and Eric R. Sims (2013). “Uncertainty and Economic Activity:
u
Evidence from Business Survey Data”. American Economic Journal: Macroeconomics, 5 (2),
pp. 217–49.
Bachmann, R¨diger and Eric R. Sims (2012). “Confidence and the transmission of government
u
spending shocks”. Journal of Monetary Economics, 59 (3), pp. 235–249.
Baker, Scott R., Nicholas Bloom, and Steven J. Davis (2013). “Measuring Economic Policy Uncertainty”. Working Paper 13-02. Chicago Booth Research Paper.
Barsky, Robert B. and Eric R. Sims (2012). “Information, Animal Spirits, and the Meaning of
Innovations in Consumer Confidence”. American Economic Review, 102 (4), pp. 1343–77.
Bloom, Nicholas (2014). “Fluctuations in Uncertainty”. Journal of Economic Perspectives, 28 (2),
pp. 153–76.
Boero, Gianna, Jeremy Smith, and Kenneth F. Wallis (2008). “Uncertainty and Disagreement in
Economic Prediction: The Bank of England Survey of External Forecasters”. The Economic
Journal, 118 (530), pp. 1107–1127.
Bomberger, William A. (1996). “Disagreement as a Measure of Uncertainty”. Journal of Money,
Credit and Banking, pp. 381–392.
Bram, Jason and Sydney C. Ludvigson (1998). “Does consumer confidence forecast household
expenditure? A sentiment index horse race”. Economic Policy Review, 4 (2).
Claveria, Oscar, Ernest Pons, and Jordi Suri˜ach (2006). “Quantification of expectations. Are they
n
useful for forecasting inflation?’” Economic Issues, 11 (2), pp. 19–38.
Coibion, Olivier and Yuriy Gorodnichenko (2012). “What Can Survey Forecasts Tell Us about
Information Rigidities?” Journal of Political Economy, 120 (1), pp. 116–159.
Coibion, Olivier, Yuriy Gorodnichenko, and Saten Kumar (2015). “How Do Firms Form Their
Expectations? New Survey Evidence”. Working Paper 21092. NBER.
23
Croushore, Dean (2011). “Frontiers of real-time data analysis”. Journal of Economic Literature,
49, pp. 72–100.
Croushore, Dean and Tom Stark (2001). “A real-time data set for macroeconomists”. Journal of
Econometrics, 105 (1), pp. 111–130.
D’Amico, Stefania and Athanasios Orphanides. “Uncertainty and disagreement in economic forecasting”. Working Paper 56. Divisions of Research & Statistics and Monetary Affairs, Federal
Reserve Board.
Federal Reserve Bulletin (1991). “Diffusion indexes of industrial production”. 7, p. 534.
Fernald, John and Stephanie Wang (2005). “Shifting data: a challenge for monetary policymakers”.
FRBSF Economic Letter, (35).
Foerster, Andrew T, Pierre-Daniel G Sarte, and Mark W Watson (2011). “Sectoral versus Aggregate
Shocks: A Structural Factor Analysis of Industrial Production”. Journal of Political Economy,
119 (1).
Gourio, Francois and Anil K. Kashyap (2007). “Investment spikes: New facts and a general equilibrium exploration”. Journal of Monetary Economics, 54, pp. 1–22.
Ivaldi, Marc (1992). “Survey evidence on the rationality of expectations”. Journal of Applied Econometrics, 7 (3), pp. 225–241.
Jeong, Jinook and G. S. Maddala (1996). “Testing the rationality of survey data using the weighted
double-bootstrapped method of moments”. The Review of Economics and Statistics, pp. 296–
302.
Kennedy, James E. (1994). “The information in diffusion indexes for forecasting related economic
aggregates”. Economics Letters, 44 (1), pp. 113–117.
Moore, Geoffrey H. (1983). “Why the leading indicators really do lead”. In: Business Cycles, Inflation, and Forecasting, 2nd ed. Ballinger, pp. 339–352.
Nardo, Michela (2003). “The quantification of qualitative survey data: a critical assessment”. Journal of Economic Surveys, 17 (5), pp. 645–668.
Pesaran, M. Hashem and Martin Weale (2006). “Survey expectations”. Handbook of Economic
Forecasting, 1, pp. 715–776.
Rich, Robert and Joseph Tracy (2010). “The relationships among expected inflation, disagreement,
and uncertainty: evidence from matched point and density forecasts”. The Review of Economics
and Statistics, 92 (1), pp. 200–207.
Runkle, David E. (1998). “Revisionist history: how data revisions distort economic policy research”.
Federal Reserve Bank of Minneapolis Quarterly Review, 22 (4), pp. 3–12.
Sill, Keith (2012). “Measuring economic uncertainty using the survey of professional forecasters”.
Federal Reserve Bank of Philadelphia Business Review, 92, pp. 16–27.
Smith, Jeremy and Michael McAleer (1995). “Alternative procedures for converting qualitative response data to quantitative expectations: an application to Australian manufacturing”. Journal
of Applied Econometrics, 10 (2), pp. 165–185.
24
Stock, James H. and Mark W. Watson (2002). “Macroeconomic forecasting using diffusion indexes”.
Journal of Business & Economic Statistics, 20 (2), pp. 147–162.
7
Appendices
r
a a
a=1 ω p .
A1. The distribution of individual diffusion indices, D =
Since the pa ’s are approximately normally distributed in large samples, as described by equation
(11), so is their linear combination. Moreover, we have that
D = E(D)
r
r
ω a E(pa ) =
=
a=1
ω a pa .
a=1
Finally,
r
V ar D
ω a pa
= V ar
a=1
r
(ω a )2 V ar (pa ) +
=
a=1
r
a=a
(ω a )2
=
a=1
=
ω a ω a Cov(pa , pa )
1
n
pa (1
−
n
pa )
ωaωa
−2
1≤a<a ≤r
r
pa pa
n
r
(ω a )2 pa −
a=1
(ω a )2 (pa )2 − 2
a=1
or
V ar D =
1
n
ω a ω a pa pa
1≤a<a ≤r
,
r
(ω a )2 pa − D2
,
(26)
a=1
where the last line follows from the Multinomial Theorem.
A2. The distribution of weighted individual diffusion indices, D =
J
j=1 γj
r
a a
a=1 ω pj .
Let
pa
j
where
xa (j)
i
1
=
n
n
xa (j),
i
i=1
is an indicator variable that takes on the value 1 when survey participant i answers a
and belongs to group or sector j, and is zero otherwise. The Multivariate Central Limit Theorem
25
gives
p1 − p1
1
1
√
n
p2 − p2
1
1
→N
D
0 −p2 p1
1 1
... ,
...
0
−pr−1 p1
1
J
...
pr−1
J
−
pr−1
J
0
...
−p1 pr−1
1 J
p2 (1 − p2 ) ...
1
1
−p2 pr−1
1 J
.
p1 (1 − p1 )
1
1
−p1 p2
1 1
...
...
−pr−1 p2
1
J
...
...
pr−1 (1
J
J
j=1 γj
r
a a
a=1 ω pj
Thus, in large finite samples, the distribution of the diffusion index statistic, D =
J
j=1 γj
is approximately normal with mean D =
r
a
a=1 ω E
J
j=1 γj
pa =
j
− pr−1 )
J
r
a a
a=1 ω pj ,
and vari-
ance
V ar(D) = V ar
(ω a γj ) pa
j
(a,j)
(ω a γj )2 V ar pa +
j
=
(a,j)
a
2 pj (1
(a,j)
=
=
cov pa , pa
j j
(a,j)=(a j )
(ω a γj )
=
ω a ω a γj γj
−
n
pa )
j
ω a ω a γj γj
−
(a,j)=(a j )
1
(ω a γj )2 pa −
(ω a γj )2 pa
j
j
n
(a,j)
(a,j)
1
(ω a γj )2 pa − D2 .
j
n
2
pa pa
j j
n
ω a ω a γj γj
+
(a,j)=(a j )
pa pa
j j
(a,j)
A3. The distribution of composite individual indices, D =
r
a=1 ω
a
a nk
n
k
k=1 δk Dk ,
where Dk =
.
D = E(D)
k
r
=
ω E
δk
k=1
k
a
(pa )
k
a=1
k
V ar(D) = V ar
=
r
ω a pa .
k
δk
k=1
a=1
δ k Dk
k=1
k
2
δk V ar Dk +
=
k=1
δk δ Cov Dk , D
k=
k
2
δk V ar Dk + 2
=
k=1
δk δ Cov Dk , D
1≤k< ≤k
26
.
(27)
In equation (27), analogously to the expression in(26), we have that for each individual diffusion
index Dk ,
V ar Dk =
where pa =
k
a ∈A
paa =
k
r
1
n
(ω a )2 pa
k
− (Dk )2
,
a=1
p(ak =a,a
a−{k, } ∈A\Ak ×A
=a ,a−{k, } )
, k = , using the notation intro-
duced in the text.
In addition,
r
Cov Dk , D
r
ω a pa ,
k
= Cov
a=1
ωa p a
a =1
pab ,
k
ωa
= Cov
r
r
a=1
b∈A
pb a
k
ωa
a =1
b ∈Ak
ω a ω a Cov
=
(a,a )∈Ak ×A
b∈A
(a,a )∈Ak ×A
b ∈Ak
Cov pab , pb a
k
k
ωaωa
=
pb a
k
pab ,
k
(b ,b)∈Ak ×A
As explained in the text, the elements, pab , of pk sum to 1 and define a multinomial distribution
k
so that pk is approximately normally distributed in large samples, with E(pab ) = pab , V ar(pab ) =
k
k
k
n−1 (1 − pab )pab , and the covariance between any two pairs, pab and pb a , given by Cov pk , pk a
k
k
k
k
ab b
−n−1 pab pb a . In particular,
k k
Cov Dk , D
1
=
n
ω a ω a paa −
k
b,b ∈Ak ×A
(a,a )∈Ak ×A
27
pab pb a .
k k
=
Figure 1: Decomposition of Positive and Negative Contributions to Employment
Growth
28
Figure 2: Decomposition of Employment Growth
29
Figure 3: Uncertainty and the Breadth of Change in Employment
30
Figure 4: Uncertainty in Consumers’ Perception of Current Conditions
31
Figure 5: Direction of Change and Uncertainty in Consumers’ Expectations
32
Figure 6: Decomposition of Uncertainty in Consumers’ Expectations
33
Figure 7: Decomposition of Comovement in Answers Across the 1 and 5-Year Horizon
34
Figure 8: Direction of Change and Uncertainty in Consumer Sentiment
35
Figure 9: Uncertainty Indicators
36
@FedFRASER