The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.

The 2016 and 2017 Surveys of Consumer Payment Choice: Technical Appendix 2018 ● No. 18-04 Marco Angrisani, Kevin Foster, and Marcin Hitczenko Abstract: This document serves as the technical appendix to the 2016 and 2017 Surveys of Consumer Payment Choice administered by the Dornsife Center for Economic and Social Research (CESR). The Survey of Consumer Payment Choice (SCPC) is an annual study designed primarily to collect data on attitudes toward and use of various payment instruments by consumers over the age of 18 in the United States. The main report, which introduces the survey and discusses the principal economic results, can be found at frbatlanta.org/banking-and-payments/consumer-payments/survey-of-consumer-paymentchoice. In this data report, we detail the technical aspects of the survey design, implementation, and analysis. JEL classification: D12, D14, E4 Key words: survey design, sample selection, raking, survey cleaning, poststratification estimates https://doi.org/10.29339/rdr2018-04 This version: August 2018 Marco Angrisani is an associate economist at the University of Southern California’s Dornsife Center for Economic and Social Research. Kevin Foster is a survey methodologist and Marcin Hitczenko is a statistician, both in the Research Department of the Federal Reserve Bank of Atlanta. The coauthors’ email addresses are marco.angrisani@usc.edu, kevin.foster@atl.frb.org, and marcin.hitczenko@atl.frb.org. This paper, which may be revised, is available on the website of the Federal Reserve Bank of Atlanta at frbatlanta.org/banking-andpayments/consumer-payments. To receive email notifications about new data reports, go to frbatlanta.org/forms/subscribe. The views expressed in this paper are those of the authors and do not necessarily represent the views of the Federal Reserve Bank of Atlanta or the Federal Reserve System. The authors thank their colleagues and management in the Consumer Payments Research Center and the Boston and Atlanta Fed research departments. In addition, we thank the management and staff at the CESR. We’d like to thank these people from the Boston Fed: Scott Schuh, Joanna Stavins, and Bob Triest. From the Atlanta Fed, we’d like to thank Claire Greene. And we’d like to thank these people from the CESR: Tania Gursche, Arie Kapteyn, Bart Orriens, and Bas Weerman. Special thanks go to Erik Meijer from the CESR, who contributed to earlier SCPC appendices, which formed the basis for this paper. Finally, the authors acknowledge John Sabelhaus and the staff of the Survey of Consumer Finances at the Federal Reserve Board of Governors for their advice and mentorship. Geoff Gerdes and May Liu from the Board also shared advice and knowledge. 1 Introduction This document serves as the technical appendix for the 2016 and 2017 Surveys of Consumer Payment Choice (SCPC), an annual survey sponsored by the Consumer Payment Research Center (CPRC) at the Federal Reserve Bank of Boston (Boston Fed) since 2008. The CPRC is responsible for writing the questionnaire, analyzing the collected data, and publishing results. The programming of the survey instrument for online use, sample selection, and data collection is outsourced to an external survey vendor. From the initial version of the SCPC in 2008 until 2013, the CPRC worked exclusively with the RAND Corporation (RAND) and their American Life Panel (ALP). In 2014, in addition to RAND, the CPRC contracted with the Center for Economic and Social Research (CESR) at the University of Southern California to use their Understanding America Study (UAS) panel for additional observations. Since 2015, the CPRC has worked exclusively with the CESR and the UAS. This document is designed to be the primary reference for the 2016 and 2017 SCPCs. The organization of this work is identical to that of previous years, based on the natural, chronological progression of considerations involved in conducting and analyzing a survey. As a result, comparisons of strategies, methodologies, and results across years can be easily done by referencing corresponding sections in earlier versions of the technical appendix. We begin by establishing the context and goals of the survey in Section 2 and follow that by highlighting changes in the survey questionnaire from the 2015 version to the 2016 version and from the 2016 version to the 2017 version in Section 3. In Section 4, we detail the sample selection strategy in the context of that used in previous years and present statistics relating to survey response and completion. Section 5 delineates the methodology used to generate the sample weights, which are used to make inferences about the entire population of U.S. consumers. Section 6 discusses our general philosophy toward data preprocessing of categorical and quantitative variables and provides detailed algorithms for key data-editing procedures. Finally, in Section 7, we present the statistical methodology used for estimating and comparing population estimates. 2 Survey Objective, Goals, and Approach In this section we describe the SCPC survey program’s overall objectives, goals, and approach, and explain the choices made in selecting the observation unit and the interview mode of the SCPC. In both cases, the choice was influenced by best survey practices along 1 with constraints imposed by the SCPC budget. 2.1 Survey Objective and Goals The main objective of the CPRC is to measure U.S. consumer payment behavior, as outlined in greater detail in Foster, Schuh, and Zhang (2013). To this end, the primary purpose of the SCPC data program is to provide an annual consumer-level dataset to support research on consumer payments and to provide aggregate data on trends in U.S. consumer payments. Key attributes of the data are the longitudinality, in which responses are collected from many of the same individuals in subsequent years, and an increasingly strong link with the Diary of Consumer Payment Choice, a second source of payments information. The change in primary survey vendor necessarily implies a discontinuation of the longitudinality built with RAND from 2008 to 2014. Differences between the RAND panel and the UAS panel are discussed further in Section 4, and the challenges of such a change in panel is discussed in Section 7.1. 2.2 Unit of Observation The SCPC uses the individual consumer as both the sampling unit and the observation unit. This choice stands in contrast to those of many other finance surveys, most notably the Survey of Consumer Finances, which is organized by primary economic units in the household, and the Consumer Expenditure Survey, which uses the household as the sampling unit and observation unit. One practical reason that the SCPC focuses on the consumer is that it is less expensive to collect data about an individual than an entire household. Household surveys require either thorough interviews with all adult household members, which is logistically difficult, or having one selected household member enter data for the entire household. The latter strategy imposes a considerable burden on the household representative. Since SCPC incentives are based on the average length of time it takes respondents to complete the survey, the cost of each survey would increase if the household were the unit of observation. This, in turn, would limit the total number of responses within a fixed budget. In addition, there is a methodological reason for the choice of observational unit. Namely, for many economic concepts on which the SCPC focuses, it seems that asking each respondent about his or her behavior rather than the entire household’s is likely to yield more accurate 2 data. Prime examples include information about getting, carrying, and using cash and the number of non-bill payments made in a month. It may be difficult for one household member to accurately report the behavior of other household members, and, even if asked, household members may not feel comfortable sharing such information with one another at such a level of detail. Therefore, it is most appropriate to ask the individual consumer about his or her own behavior and not about the habits of other household members. Admittedly, the use of the consumer as the unit of observation may not be ideal for other variables, most notably the payment of bills or other expenses more closely associated with the household than an individual. For consistency within the survey instrument, the SCPC asks respondents to estimate only the number of bills that they physically pay themselves, either by mail, by phone, online, in person, or by having set up an automatic payment. In theory, this should produce accurate measurements on bill payment behavior when averaged across the entire sample. The difficulty comes from the fact that these kinds of payments may be paid out of joint accounts or pooled resources, or may have been set on an automatic schedule so long ago that no household member can recall who set up the automatic payments. As a result, it can be difficult to attribute responsibility for such payments, often leading to under-counting, if they are not reported at all, or double-counting, if several household members each claim responsibility for the same payment. In addition, research on SCPC data suggests that survey respondents are more likely to have a higher share of financial responsibility within the household than would be expected if household members were selected at random, and thus tend to be more likely to make certain types of payments than an average sample of the population (Hitczenko 2015). Treating such a sample as representative of all consumers may lead to overestimation of the number of bills paid. Therefore, to accurately measure bills, it might be preferable to ask about the entire household’s bill payment behavior. 2.3 Interview Mode The SCPC is a computer-assisted web interview (CAWI). This mode of interview fits best with our sampling frame, which is an internet-based panel. To maximize coverage, all ALP members are given internet access upon recruitment into the panel. The survey instrument is the NubiS survey system, developed by team members at the CESR.1 The CAWI mode is beneficial to the SCPC because of the length and complexity of the survey. The projected median length in minutes for the SCPC survey in each year is around 1 More information on NubiS is available at https://cesr.usc.edu/nubis/node/2. 3 30 minutes. In 2016, the median time spent on survey screens was 37.6 minutes and the middle 50 percent spent between 28.5 and 49.1 minutes on the survey screens. Changes to the survey from 2016 to 2017, described in detail in Section 3, resulted in a significantly shorter survey. Specifically, in 2017, the median SCPC took 30.6 minutes, while the middle 50 percent took between 23.1 and 39.6 minutes. Using a CAWI allows the respondent to log off and come back to the survey later if interrupted. This is cheaper than using face-to-face interviews or telephone because there are no interviewers who need to be trained and paid. Finally, respondents may be more willing to answer some sensitive questions, like the amount of cash stored in their home, if the survey is conducted via the web (De Leeuw 2005). 2.4 Public Use Datasets Individuals who are interested in downloading the original, unprocessed datasets can obtain these from the UAS website.2 The SCPC website also contains a link to the UAS data download site.3 Interested users must create a username and password to download data from the UAS website. These data contain only the survey variables found directly in the survey instrument itself. These survey variables have not been edited or processed. For example, survey items that allow the respondent to choose a frequency (week, month, or year) have not been converted to a common frequency. For those interested in using these data, we recommend identifying survey variables by directly finding them in the SCPC questionnaire, which can be downloaded from the SCPC website. An extension of the data, which includes edited variables and new variables created by the CPRC, can be downloaded at the SCPC website as well. The data are available in Stata, SAS, and CSV formats. Information about the definitions and naming schemes for all new variables not found in the original dataset are described in the companion document, “SCPC Data User’s Guide: 2016-17” (Foster 2018), which is also available at the SCPC website. Before using the data, it is useful to read the warning against using consumer-level estimates to aggregate up to U.S. total population estimates, in Section 7.2.1 of this paper. The variable prim key is the unique identifier for each respondent. This variable is used as the primary key for both SCPC and the Diary of Consumer Payment Choice (DCPC) datasets, and is based on the terminology established by RAND and used in SCPCs and DCPCs prior to 2015. We choose to continue with this nomenclature for internal consistency, 2 2016 SCPC https://uasdata.usc.edu/UAS-62, 2017 SCPC https://uasdata.usc.edu/UAS-105 Because of a change in ownership of the SCPC and DCPC from the Federal Reserve Bank of Boston to Atlanta, all relevant information can now be found at:https://www.frbatlanta.org/ banking-and-payments/consumer-payments.aspx 3 4 even though the UAS dubs the unique individual identifier as uasid. As a result, to merge any UAS dataset, including the the raw, uncleaned dataset, with any processed SCPC and DCPC dataset, the user must merge the variables prim key and uasid. In some statistical software programs, this requires renaming one of the variables to match the other’s name. 3 Questionnaire Changes The SCPC questionnaire is written by the CPRC and is available to download at the SCPC website. For the most part, the survey questions for the 2016 and 2017 SCPC are the same as or similar to those in prior versions, although changes are introduced every year either to collect new information or to improve the data collection process for the same information. This section describes the changes in two parts, first, the changes to the questionnaire from 2015 to 2016, then the changes from 2016 to 2017. 5 3.1 Changes from 2015 SCPC to 2016 SCPC Table 1: New questions in the 2016 SCPC, Assessment of Characteristics section Variable ID as004 Question description How do you rate the security of the following means of making a payment? as005 g How would you rate the security of each type of debit card transaction? Online without providing a security code (CVV) as005 i How would you rate the security of each type of debit card transaction? During a voice telephone call, without security code (CVV) as012 Please tell us which payment characteristic is most important when you decide which payment method to use. Table 2: New questions in the 2016 SCPC, Virtual Currency section Variable ID pa133 Question description In the past 12 months, have you exchanged virtual currency for US dollars or exchanged US dollars for virtual currency? pa135 In the past 12 months, how many times did you exchange virtual currency for US dollars or vice versa? pa128 In the past 12 months, have you used virtual currency to make a payment for goods or services to another person? 6 Table 3: New questions in the 2016 SCPC, other sections Variable ID pa045 c Question description In the past 12 months, have you authorized a text message payment using one of the following methods? Authorize your mobile phone company to pay for you. ph025 g Do you use any of the following online personal financial management services or apps to budget and monitor your spending, saving, or account balances? MoneyWiz ph025 h Do you use any of the following online personal financial management services or apps to budget and monitor your spending, saving, or account balances? GoodBudget de012 Please tell us the total combined income of all members of your family living here during the past 12 months. (only asked if item de010 = 18, or “$500,000 or more”) 7 Table 4: Questions that were edited from 2015 to 2016, all sections Variable ID as005 Question description Description of change How would you rate the security of each type of Changed some of the debit card transaction text for existing questions, added two new questions. pa001 Screen for reporting number of checking and Edited instruction text savings accounts to be shorter and more concise. pa001 Screen for reporting number of checking and Added definition of savsavings accounts ings accounts pa055 Table of questions to determine underbanked Edited to present items status as two tables on the same screen instead of one pa120 List of virtual currencies Dropped Stellar added Ethereum pa129 Who did you pay using virtual currency? Edited list of response options pu011 How would you compare your unpaid balance Added new response opon your credit card to your unpaid balance 12 tion – I did not have a months ago? balance 12 months ago pa189 c In the past 12 months, have you used a mobile Edited the response for phone to make any of these kinds of payments? using a mobile app pa001 e List of mobile apps Added Samsung Pay and Square Cash pa001 f List of mobile apps Added Samsung Pay and Square Cash ph004 In the past 12 months, have you, or anyone you Added definition of idenknow well, been a victim of identity theft? tity theft 8 and Table 5: Dropped questions from 2015 to 2016, all sections Variable ID pa007 b Question description Is your savings account linked to your primary checking account? pa015 c About how much cash stored elsewhere are you holding for cash payments? pa015 d About how much cash stored elsewhere do you have set aside for long-term savings? pa198 n Prepaid card type: Other types of passes or membership cards (museum, gym, parking, recreation) pa202 n Prepaid card logos: Other types of passes or membership cards (museum, gym, parking, recreation) pa203 n Prepaid cards that can pay anywhere: Other types of passes or membership cards (museum, gym, parking, recreation) 9 3.2 Changes from 2016 SCPC to 2017 SCPC Table 6: Dropped questions from 2016 to 2017, adoption section (Part One) Variable ID pa007 Question description At what type of financial institution is your primary savings account? pa006 a, b At what type of financial institution is your [primary, secondary] checking account? pa073 a, b About how much money do you have in your [primary, secondary] checking account? pa085 a, b Average balance of [primary, secondary] checking account pa086 a, b Drop-down list with dollar ranges for balance of [primary, secondary] checking account pa080 a, b With whom do you share your jointly owned [primary, secondary] checking account? pa022 Please choose the most important reason why you don’t have an ATM card. pa021 Please choose the most important reason why you don’t have a debit card. pa034 If you are given a choice while completing a debit card purchase, do you prefer to enter your PIN or give your signature? 10 Table 7: Dropped questions from 2016 to 2017, adoption section (Part Two) Variable ID pa055 Question description In the past 12 months, did you use any of the following financial services? (Underbanked screen) pa027 Please choose the most important reason why you don’t have a credit card. pa051 a-e How many of each kind of these kinds of credit cards that are also branded with a company logo? pa195 Please choose the most important reason why you don’t have a general purpose prepaid card. pa192 Do you use any phone apps that are funded by buying a prepaid card and entering the number on the card into your app? pa109 Please choose the most important reason why you don’t have any automatic bill payments set up. pa001 d2, d3 Do you have an account with any of the following payment services? [Google Wallet, Amazon Payments] pa044 b, c In the past 12 months, have you used [Google Wallet, Amazon Payments] to make a purchase or pay another person? pa047 a, b, c About how much money do you have in your [PayPal, Google Wallet, Amazon Payments] account? pa048 a2-e2, a3-e3 In the past 12 months, have you used any of the following methods to make payments with your [Google Wallet, Amazon Payments] account? 11 Table 8: Dropped questions from 2016 to 2017, cash section Variable ID Cash Q’s Question description The entire cash section has been moved to Day 2 of the Diary pa015 a About how much cash do you have in your wallet, purse, and/or pocket? pa015 b About how much cash do you have stored elsewhere in your home, car, office, etc. pa016 When you get cash, where do you get it most often? pa017 a When you get cash from [FILL WITH ANSWER FROM PA016], what amount do you get most often? pa018 1 In a typical period (week, month, or year), how often do you get cash from [FILL WITH ANSWER FROM PA016]? pa017 b When you get cash from all other sources besides [fill from answer PA016], what amount do you get most often? pa018 2 In a typical period (week, month, or year), how often do you get cash from all other sources besides [fill from answer PA016]? Table 9: Dropped questions from 2016 to 2017, all other sections Variable ID as004 Question description How do you rate the security of the following means of making a payment? as005 How would you rate the security of each type of debit card transaction? as012 Please tell us which payment characteristic is most important when you decide which payment method to use. ph025 a-h Do you use any of the following online personal financial management (PFM) service or app to budget and monitor your spending, saving, or account balances? 12 Table 10: New questions in the 2017 SCPC, adoption section Variable ID pa909 Question description Have you ever had a store-branded card linked to your bank account? Yes/No pa031 b Have you ever had blank paper checks for any of your checking accounts? Yes/No pa008 c Number of store-branded cards linked to your bank account pa042 a Did you purchase any of the money orders you used in the past 12 months from a non-bank source? pa042 e Did you send any of the remittances you used in the past 12 months from a non-bank source? pa041 e Have you ever sent a remittance, even once? pa120 b7 Have you heard of Bitcoin Cash (BCH)? pa052 Do you own any kinds of credit cards that also are branded with a company logo? pa001g1, 2, 3 In the past 12 months, have you used any of the following features of your bank’s mobile banking app? Table 11: New questions in the 2017 SCPC, all other sections Variable ID pu020 Question description On your last bill(s), about how much were the new charges made to all of your credit cards and/or charge cards? pu004 f1-3 Number of bill payments by mail, in person, or by phone paid using your bank account and routing numbers pu021 g Person-to-person payments made by using account-to-account payments using a nonbank service such as PayPal or Venmo 13 Table 12: Questions that were edited from 2016 to 2017, all sections Variable ID text screens Question description Description of change Any screen with all text—instructions, defini- Reduced the number of tions, etc. words on each of these screens pa075 a, b Is your primary [secondary] checking account jointly owned with someone else? pa050 In the past 12 months, have you used any of the Moved to following payment methods, even once? Cash with other instruments a table payment pa040 e In the past 12 months, have you used any of the Moved to following payment methods, even once? Remit- with other tance instruments a table payment pa055 a2, b1-b5 Underbanked questions Pa055 a2 is now on its own screen, Pa055 b1-b5 are on a separate screen in a table VC section All questions Added ticker symbols for currencies VC section List of currencies Removed Dash and Dogecoin, added IOTA and NEM pu010, pu013 Credit card balance (pu010) and limit (pu013) Moved these questions to be inside a skip condition pa201 Do you have any of the following types of pre- Changed from askpaid cards? ing Yes/No to asking Number of Cards pa001 e, f Do you have any of the following mobile apps Added Amazon Payor online accounts? ments, Zelle, and “Your bank’s mobile banking app” pu005 b, pu006a c, pu006c c Payment use section questions about debit cards pu021 c, d Person-to-person payments using debit cards Added text about Venmo and credit cards Person-to-person payments using account-to- Added text “using a seraccount payments vice provided by your bank” 14 pu021 e Changed from Yes/No to four different response options from pa080 Added text about store branded linked debit cards 4 Data Collection This section describes various aspects of the data collection process for the 2016 and 2017 SCPC. Once the survey instrument is finalized, the goal is to recruit a sample of respondents that can be effectively used to make inferences about the U.S. population of consumers and, then, effectively administer the survey to those individuals. The methodologies and their underlying philosophies adopted by the CPRC in this process are outlined below. In addition, outcome statistics related to the fielding of the survey are detailed. Similar expositions focusing on the previous editions of the SCPC can be found in the official releases by the CPRC (Angrisani, Foster, and Hitczenko 2013; 2014; 2015; 2016; 2017). As in 2015, the SCPC in 2016 and 2017 relied exclusively on the Understanding America Study (UAS), a panel of respondents created and managed by the Center for Economic and Social Research’s (CESR) at the University of Southern California, after a change in 2014 from RAND’s American Life Panel. In the long term, the CPRC believes that the best practices used to construct and maintain the UAS panel, most notably address-based sampling for recruiting panelists, will provide for better population estimates in the future. The motivations for the change in survey vendor are discussed in greater detail in the 2015 SCPC Technical Appendix (Angrisani, Foster, and Hitczenko 2017). However, in the short term, panel effects, biases related to the particularities of respondent selection and survey administration associated with each panel vendor, introduce possible challenges to time series analysis (Kennedy et al. 2016). In Section 7 of this document, we suggest a statistical methodology for inference based on data collected from multiple panels. 4.1 Understanding America Study The UAS originated in 2014 as a collection of individuals recruited to participate in a wide variety of surveys. The vast majority of panelists are recruited as part of the “Nationally Representative” subsample, intended to well-represent unincarcerated individuals aged 18 years and older who live in the United States, the target population for the CPRC. In addition, the UAS features two smaller subsamples constructed for specific research projects hosted by CESR: individuals of Native American origin and families with young children who live in Los Angeles county. The Native American cohort was used in 2015, when the small size of the UAS panel required it to ensure a large enough sample. Since then, no individuals from this cohort were recruited for the SCPC. Individuals from the Los Angeles county cohort have never been included in the SCPC sample, because effectively incorporating data from 15 such a specialized subpopulation into general population estimates is difficult.4 Therefore, the description below relates exclusively to the Nationally Representative subset of the UAS panel. The UAS panel uses address-based sampling, in which zip codes and then addresses within those zip codes are drawn at random, to generate a probability panel. A detailed description of the recruitment process can be found at the UAS website https://uasdata.usc.edu/ recruitmentoverview/1, but a brief summary is given below. After an initial introductory postcard, an invitation featuring a $5 prepaid card and a 10-minute paper survey is mailed to the selected households. An additional incentive of $15 is provided for the return of the survey. The use of mail in the initial outreach ensures that all households, even those without internet or an online presence, are included. Those who join the panel but do not have internet access are eventually provided with internet access and a tablet with which to take surveys. Multiple stages of correspondence, featuring reminders, follow-up surveys, and additional financial incentives, are conducted through mail, phone, and the internet to build a relationship with the individual and encourage enrollment in the panel. A key feature of the recruitment process is that individuals are encouraged and incentivized to enroll fellow household members. As a result, about 18 percent of enrolled households featured multiple members in the panel. Recruitment into the UAS panel has been conducted in waves by the CESR. The initial recruitment wave, commenced in February 2014, was also the largest with 9,284 households selected for contact. An additional wave of 1,799 households were invited in September 2015, followed by seven waves of recruitment from January 2016 to June 2016 that invited from around 1,800 to around 3,500 households each. The waves conducted in 2016 were used to improve demographic representation of the panel by favoring ZIP codes more heavily populated with demographic groups lacking in the panel. In addition, the CESR conducted experiments on the details of the recruitment process itself, such as the incentives and types and order of correspondence. In general, the recruitment waves had about 15 percent success rate, defined as the percent of individuals contacted who eventually became panel members. The recruiting effort in 2016 resulted in a sizeable increase in panel size, up to 4,776 at the time of sampling in 2016 from 2,140 at the same time in 2015. The panel size decreased by 18 people to 4,759 at the time of sampling in 2017, a net result of attrition, which is less than 10 percent annually, and a lack of additional recruitment efforts other than allowing household members to join. Thus, between August 2016 and August 2017, the only new 4 To see how the Native American cohort was incorporated into the sample via poststratification weights, see the 2015 Technical Appendix. 16 panelists were additions from households with a member already featured in the UAS. Once in the panel, UAS members are eligible to be recruited for surveys, unless they formally ask to be removed or stop participating in surveys over a prolonged period of time. At the beginning of each year, the CESR contacts all members who did not take any survey for at least a year and removes them from the panel, unless they explicitly declare continued interest in participating. Since inactive members are removed only once a year, the pool of those invited in August to answer the survey at a given point in time may include inactive members. 4.2 SCPC Sample Selection Sample selection for the SCPC is done jointly with the Diary of Consumer Payment Choice (DCPC), a second data collection instrument developed by the CPRC and co-sponsored by the Federal Reserve Banks of San Francisco and Richmond. The DCPC is a diary that asks individuals to track details of all payments and financial transactions over the course of three consecutive days in October. As discussed in Section 2, the DCPC has increasingly grown to be a complement to the SCPC. In fact, the 2016 and 2017 version of the DCPC pulls in information collected in the SCPC. Because of this strengthening connection, since 2015, the CPRC has introduced a strict requirement that respondents must take the SCPC prior to their assigned diary period. Selection of respondents is primarily based on trying to maximize the number of respondents that participate in both surveys. Traditionally, the fielding of a survey such as the SCPC by the CESR simply involves making it accessible to each potential respondent online on some specified release date, with notification of its availability coming in the form of an email and link. However, the unique nature of the DCPC, which distributes responses throughout an entire month and requires prior commitment from individuals in order to mail diary-related materials, requires an additional consent survey. This consent survey is treated like a typical survey fielded by the CESR, released at the end of August with a $5 incentive for individuals. The incentive was introduced in 2016 in an effort to increase the percentage of individuals who answer the consent survey. Although fielding this consent survey with an incentive reduces the budget available for the SCPC and DCPC, experience has shown that without the incentive a much lower percentage of panelists respond, thus making planning for the DCPC much more difficult. Based on the improved results from 2015, a year in which no incentive was used for the consent survey, to 2016, the incentive was continued in 2017. The consent survey describes both surveys, including the $20 incentive for the SCPC and the $70 incentive for the DCPC, and 17 asks respondents if they are willing to take both surveys. Those who decline are asked for a reason why and are presented with a second request that aims to assuage their concerns. In 2016, individuals who did not agree to participate in both surveys were asked if they would be willing to take the SCPC only. In 2017, because of the high consent rate and the smaller target sample size, this option was removed, so that only those willing to take both surveys were able to take the SCPC. The selection of the individuals to whom the consent survey is sent is governed by two basic principles. First, it is our desire to have the demographic composition of the sample match that of the U.S. population of adults as closely as possible. To ascertain this, the population is divided into 30 disjoint strata defined by household income (3 groups), age (3 groups), gender (2 groups), and race (2 groups), and sample proportions in each stratum are compared to those implied by the March supplement of the Current Population Study of the relevant year. The second principle is a desire to maintain a longitudinal panel across years. The benefits of such a longitudinal panel, in the form of added power associated with tracking trends at the individual level, are well known (Baltagi 2008; Duncan and Kalton 1987; Frees 2004; Lynn 2009). For many research agendas, it is advantageous to base results on a longitudinal panel, rather than on a sequence of cross-sectional studies. The CPRC does not construct a formal longitudinal panel, governed by rules for replacing members who drop out in order to maintain similar panel composition. Instead, we simply put a heavy emphasis on inviting respondents who participated in previous years. Because the participation rates in the UAS panel are quite high, this somewhat informal strategy effectively yields a longitudinal panel. The number of desired respondents for the joint SCPC and DCPC is determined by the available budget and the cost of management, programming, and data collection in a given year. In 2016, the desired number of respondents was 3,300, while in 2017 it was 3,150. These numbers along with the predicted participation rates, estimated to be between 60 and 70 percent, inform the number of individuals necessary to invite to achieve the target number of respondents. In 2016, getting an adequate sample size meant inviting the entire set of nationally representative individuals, so no selection according to demographics was done. In 2017, initial calculations revealed that perhaps not all panelists would need to be invited. So, stratified quota sampling was used to determine the number of respondents in each of the 30 demographic strata that would need to be invited to yield a representative sample of the desired size. To maximize longitudinality, respondents were classified into two tiers according to whether they had any prior experience with the SCPC or DCPC or not. The invitation survey was first sent out to the first tier, which consisted of 3,677 individuals, so that spots that could be filled with individuals with more experience were not filled by new 18 respondents simply because they responded faster. Based on observed participation rates among the first tier, it was determined worthwhile to send the second wave of invitations to all of the remaining 1,082 panelists who had no prior experience with the survey. This second wave was sent two weeks after the first wave. Although the strategy was effectively the same in 2017 as in 2016, with all panelists sent an invitation, the tiering used in 2017 shows how longitudinality can be maintained in future years as the UAS panel increases in size. Table 13 shows the number of individuals invited and the number who consented to participate in 2016 and 2017. In both years, the pattern of consent is similar. The major barrier to consent was getting panelists to take the invitation survey, as only 74 percent of individuals did so in 2016 and 68 percent did so in 2017. However, willingness to participate among those who took the consent survey was very high. In both years, about 94 percent of respondents agreed to participate in the SCPC and DCPC, yielding an overall consent rate of close to 65 percent. Additionally, over 98 percent of those who consented to both surveys did so with the initial invitation, without requiring assurances about their concerns. In 2016, of the 211 individuals who did not agree to the joint survey, 105 or nearly 50 percent agreed to take the SCPC only. Table 13: Number of survey invitations, consents, and participations for the 2016 and 2017 SCPC. Panel Size Sent Invitation Took Invitation Survey Consent to Both Consented to Initial Invite Consented to Secondary Invite Consented to SCPC Only Total Started the SCPC Completed the SCPC Year 2016 2017 4,776 4,759 4,776 4,759 3,572 3,293 3,291 70 105 3,466 3,404 3,386 3,110 48 NA 3,158 3,099 3,075 Source: Authors’ calculations Figure 1 shows the sizes of various panels within the UAS framework dating back to the 2014 SCPC survey. The most notable feature of the figure is the significant jump in sample size from 2015 to 2016, an increase of around 150 percent. As a result, while the four-year panel is composed of around 600 individuals, the two-year panel starting in 2016 is much larger, with over 2,600 respondents. The expectation is that the longitudinal panel beginning in 19 2016 and stretching into the future year will continue to be over 2,000 individuals. 2015 2016 2017 SCPC Longitudinal Panel Structure 2014 2014−2017 Panel (598) 2015−2017 Panel (297) 2016−2017 Panel (1737) All Other 0 1000 2000 3000 4000 Figure 1: The longitudinal panel structure from 2014 to 2017. Source: Authors’ calculations 4.3 Survey Participation The CPRC has generally limited the survey release to a period in the fall, ranging from the end of September through October. From an economic point of view, this time of year is a reasonably representative period of time with respect to certain economic variables such as employment or sale volumes; it includes no major holidays and falls between summer and winter. For the same reason, all previous versions of the DCPC, with the exception of the 2015 version, were also administered in October, with the added incentive that responses from both surveys could be linked more easily if they correspond to the same period of economic activity.5 The increasing link between the SCPC and the DCPC, in which the former is a prerequisite to the latter, effectively forces the release of the SCPC into the middle of September. 5 The 2015 DCPC was delayed in administration due to complications of coordinating with multiple vendors. More information is provided in the 2015 Technical Appendices. 20 The official release of the SCPC involves emailing a notification and a survey link to all respondents who have consented to participate. Each respondent can begin the survey at any point after receiving the notification of the SCPC release. In both 2016 and 2017, the SCPC was released on September 19th in order to give the earliest diarists 10 days to complete the SCPC before the first module of the DCPC, which can occur as early as September 29th. Participants do not have to finish the survey in one sitting and have the option to log off and continue later. Respondents were given reminders every so often at the discretion of the CESR if they had not logged on to the SCPC. In particular, reminders were sent out a few days before the assigned DCPC dates to each respondent to ensure that individuals have completed the SCPC by the time their DCPC commences. Starting the survey, which is defined as logging on to the survey, is an important threshold because everyone who does so is considered a participating respondent and is assigned a survey weight. The lower half of Table 13 shows that, in both 2016 and 2017, 98 percent of respondents who consented to participate in the SCPC started the survey. A second metric used to gauge participation is that of completion, which we define as logging off after the final survey screen. It is important to note that logging off may not accurately reflect total completion of the survey, as it is possible to finish the survey without logging out. Other standards to define survey completion can be used. For example, one such standard would be answering all of the SCPC questions and reaching the last screen, which asks individuals for feedback on the survey questionnaire itself, but not logging out. Indeed, reaching the last question is the minimum requirement for the respondent to receive the financial incentive. As can be seen in Table 13, fewer than 1 percent of those who started did not complete the survey, which marks an improvement from previous years when 2 to 3 percent did not complete the survey. As expected due to the joint recruitment effort, the sample overlap between the SCPC and DCPC is high in both years. In 2016, 90 percent (all but 356) of individuals who participated in the SCPC also took the DCPC. In 2017, this number was 93 percent (all but 228). However, if one excludes the 105 individuals who only committed to the SCPC in the first place in 2016, the overlap rate between both samples is very similar in both years. More information about how DCPC diary periods relate to the SCPC will be provided in the technical appendices for the DCPCs for each year. Figure 2 shows the proportion of surveys completed as a function of the number of days since the survey was distributed for the 2012 – 2017 versions.6 Throughout the life of the 6 Similar plots that include data from 2008 to 2011 can be found in earlier versions of the SCPC Technical Appendix. 21 survey, the pace of completion for the 2016 and 2017 SCPC was generally much faster than in previous years and very similar within the two years. Whereas about 50 percent of respondents had started the SCPC within the first three days in 2016 and 2017, only 30 to 40 percent had done so in previous years. Within a month of release, the completion rate is around 95 percent in the two most recent years, as compared to below 90 percent in previous years. A higher completion rate is also reached, as noted above. A major part of this trend might be the necessity of taking the SCPC before the DCPC, which results in a greater number of reminders and a greater incentive to do so. SCPC Completion Since Release 1.0 Proportion Completed 0.8 2017 2016 2015 2014 2013 2012 0.6 0.4 0.2 0.0 0 10 20 30 40 50 60 Days Since Survey Release Figure 2: The proportion of respondents who completed the survey as a function of the number of days since the survey was received. Source: Authors’ calculations Figure 3 shows the proportion of surveys completed by each calendar day within each of the years from 2012 to 2017. This depiction reveals the relatively wide range of time frames in which the bulk of annual responses are collected. In earlier versions, a majority of data responses were collected at the end of September, while in 2014 and 2015, most data were gathered near the middle of October. In 2016 and 2017, responses were collected even earlier, with most coming in the middle part of September. An ideal survey design would specify that responses should be collected in a way that stan22 dardizes the response period across years. From an analytical point of view, trends from year to year are more easily identified if differences in behavior are not attributable to seasonal behavioral variation. Although the SCPC asks respondents about behavior in a “typical” month to reduce seasonal effects, it is possible that recent activity may influence responses. For example, to the extent that taking the survey near the beginning or end of a month influences responses, one should be aware of this when comparing results across years. In addition, if typical behavior changes in November due to the ensuing holiday season, payment use responses for a larger fragment of the 2015 SCPC sample than in other years will reflect this. Optimal survey fielding may involve accounting not only for the time of year and the time within a month, but also the time within a week. This is difficult to do, as respondents must be given a reasonable window of time in which to take the survey. The observed temporal gaps are even more extreme at the individual level, where a particular respondent might respond in October of one year and as late as January in a different year. Again, this raises potential issues of comparability. SCPC Completion By Time of Year 1.0 Proportion Completed 0.8 2017 2016 2015 2014 2013 2012 0.6 0.4 0.2 0.0 September October November December January Day in Year Figure 3: The proportion of respondents who completed the survey as a function of the date within the year. Source: Authors’ calculations Figure 4 compares the distributions of the number of minutes it took respondents to complete the survey for the past five years of the SCPC, defined here as the difference in minutes 23 between the time of first log-in to the survey and the last log-out. Individuals who take breaks while taking the survey will thus have long completion times, yielding the skewed-right nature of the observed timing curves. Changes in median time of completion generally correspond to changes in the length or complexity of the questionnaire. Although the 2013 survey has a median completion time of 32 minutes, almost every respondent provided additional information in a follow-up survey, dubbed Module B, which had a median time of 15 minutes (see Angrisani, Foster, and Hitczenko (2015) for details). The 2014 survey was designed to be considerably shorter than all previous versions (and was not paired with a followup survey) and has a median completion time of 29.5 minutes. A detailed description of the changes, mostly the removal of questions, from 2013 to 2014 made can be found in the 2014 Technical Appendix (Angrisani, Foster, and Hitczenko 2016). The 2015 and 2016 versions reincorporated a lot of information from Module B, and the 2016 SCPC began asking more detailed questions that were fed to the DCPC, such as checking account balances. The 2017 version was again made shorter, as documented in more detail in the following section, with a result of shortening the median time of completion. Of course, other factors, such as greater familiarity with the survey questions by the repeat respondents, may also contribute to changes in completion time distributions. Finally, in 2015, there was evidence that completion times depended on the speed of survey loading and the CESR servers, an effect that could influence observed distributions in any given year. 4.4 Item Response For a survey to provide a valid picture of the overall population, it is very important that the item response rates for each question be high. High nonresponse rates not only mean there is less information on which to base estimates but also raise concerns about potential bias in the estimates. If the fact that an observation is missing is independent of the value of the observation, a condition referred to as “missing at random” (Little and Rubin 2002), imputation procedures can be used to generate estimates of sample statistics. However, if there is a confounding variable that relates to both the value of a variable and the likelihood of nonresponse, it is impossible to adjust for the effects on sample statistics. Certain economic variables, such as net worth or personal cash holdings, are potentially sensitive topics, and it is possible that there is a correlation between the true values and the willingness of respondents to provide these values. Naturally, variables with low nonresponse rates are less susceptible to this type of bias. The SCPC has over 200 survey variables, although the survey itself is administered with a 24 SCPC Completion Times Density 2017: Median = 37.8 2016: Median = 48.3 2015: Median = 48.6 2014: Median = 29.5 2013: Median = 32.4 2012: Median = 37.7 0 15 30 45 60 75 90 105 Time (in minutes) Figure 4: The proportion of respondents who completed the survey as a function of time. The vertical line at 30 minutes represents the intended average length of completion. Source: Authors’ calculations relatively complicated skip logic so not everyone answers the same set of questions. However, taking a set of eight questions asked of everyone, dispersed throughout the survey, we found item nonresponse rates were low in both years, as shown in Table 14. Seven out of the eight nonresponse rates were under 1 percent. The exception is a question about the number of checking accounts (pa001 a), with around 3 percent of respondents not answering in both years. One possible explanation is that individuals who are nonadopters leave the answer box blank rather than writing in zero. Overall, the response rate is very high within the SCPC, which may be partly attributable to the fact that respondents have volunteered to take surveys and are being paid to do so. In 2016 and 2017, slightly less than 97 and 96 percent of respondents, respectively, answered all eight of the selected questions. 25 Table 14: Nonresponse rates (%) for eight questions in the 2016 and 2017 SCPC. The exact text of the corresponding questions can be found in the 2015 SCPC Questionnaire. Question Section in SCPC 2016 SCPC 2017 SCPC fr001 a II 0.23 0.23 as003a4 III 0.32 0.45 pa001 a IV 2.82 3.10 pa050 IV 0.47 0.94 pa053 IV 0.47 0.35 pa024 IV 0.53 0.48 ph006 VI 0.56 0.71 de011 VII 0.73 0.87 Source: Authors’ calculations. 5 5.1 Sampling Weights Post-Stratification An important goal of the SCPC is to provide accurate estimates of payment statistics for the entire population of U.S. consumers over the age of 18. Although the UAS panel uses addressbased sampling, a form of probability sampling, relatively low response rates may mean that the final set of respondents taking the SCPC may not be a good representation of the target population. As a rough estimate, consider that only around 15 percent of individuals invited joined the UAS panel, and, of those, around 65 percent took the SCPC, yielding an overall response rate of 9.8 percent. If there are systematic differences in the likelihoods that a randomly selected individual ends up taking the SCPC across demographics, this could manifest itself in a final sample that looks different from the original population of invitees. Even relatively minor shifts in demographic composition can lead to bias, if not accounted for, for economic variables that show a lot of heterogeneity across demographic strata, as is the case with many payments variables (Stavins 2016). Table 15 shows the unweighted and weighted marginal proportions of various demographic groups in the 2016 and 2017 SCPCs. Overall, the demographic composition of the samples in the two years is very similar, as should be expected from the fact that most respondents feature in both samples. Based on the weighted results, the raw sample overrepresents females, older individuals, people who identify as white, and well-educated individuals. This reflects imbalances in the UAS panel itself as there is no evidence that there are strong demographic effects relating to propensity of consenting once invited. Slight deviations in the weighted demographics are partly due to changes in the true population values and partly due to the differences in the unweighted sample compositions in the two years. Optimal allocation of respondents across strata depends on the degree of variation of responses across demographics, with strata that have greater variability in responses requiring 26 more observations. If the variance of responses is fixed across demographic strata, the most efficient estimate will be based on a sample in which the number of responses for each stratum is proportional to its overall frequency in the population. For this reason, without a priori knowledge about demographic patterns, proportional representation of strata in the sample is most often the goal of sample selection. Nevertheless, work by Wang et al. (2009) suggests that nonrepresentative polling can provide relatively accurate estimates with appropriate statistical adjustments. Table 15: Unweighted percentages for various marginal demographics in the 2014 and 2015 SCPC sample, as well as weighted percentages for the 2015 SCPC. The weighted values are based on CPS data. Demographics Gender Male Female Age 18−24 25−34 35−44 45−54 55−64 65 and older Race White Black Asian Other Ethnicity Hispanic Education No HS diploma High School Some College College Post-graduate Income < $25K $25K − $49K $50K − $74K $75K − $99K $100K − $124K $125K − $199K ≥ $200K Unweighted 2016 SCPC 43.2 56.8 4.1 15.8 19.6 20.6 22.9 17.0 84.2 8.5 1.8 5.5 6.8 4.6 19.5 38.9 21.5 15.5 21.3 23.4 20.3 13.1 8.9 9.7 3.4 Unweighted 2017 SCPC 44.0 57.0 3.2 14.9 19.8 18.2 23.5 20.4 83.5 9.1 1.8 5.6 6.6 4.8 19.8 38.0 21.4 15.9 20.0 23.7 20.1 13.8 9.0 9.9 3.4 Source: Authors’ calculations 27 Weighted 2016 SCPC 48.3 51.7 6.8 23.2 16.3 17.4 16.8 19.5 74.3 13.3 3.1 2.2 12.1 7.3 33.4 28.5 17.3 13.4 21.1 23.8 17.8 12.0 10.5 11.1 3.7 Weighted 2017 SCPC 48.2 51.8 5.1 24.6 16.2 17.1 17.0 20.1 74.2 13.7 3.6 1.4 11.9 7.0 33.0 28.4 18.0 13.5 18.3 23.4 19.4 13.2 10.2 11.7 3.8 To enable better inference of the entire population of U.S. consumers, SCPC respondents are assigned post-stratified survey weights designed to align as much as possible the composition of the SCPC sample with that of a reference population. Specifically, each year the benchmark distributions against which SCPC surveys are weighted are derived from the CPS. This follows common practice in other social science surveys, such as the Consumer Expenditure Survey (CES). 5.2 Raking Algorithm Sampling weights are generated by the CESR, using a raking algorithm (Deming and Stephan 1940; Gelman and Lu 2003) that is very similar to that used by RAND in previous years. This iterative process assigns a weight to each respondent so that the weighted distributions of specific socio-demographic variables in the SCPC sample match their population counterparts (benchmark or target distributions). The weighting procedure consists of two main steps. In the first part, demographic variables from the CPS are chosen and mapped onto those available in the SCPC. Table 16 shows the 31 variables used in weighting as well as the levels within each variable. The number of levels for each variable should be small enough to capture homogeneity within each level, but large enough to prevent strata containing a very small fraction of the sample, which could cause weights to exhibit considerable variability. The socio-economic variables chosen for the raking procedure result from recent research conducted by RAND regarding the sampling properties of weights based on different demographic factors. Sample weights produced by different combinations of variables were evaluated on the basis of how well they matched the distributions of demographic variables not used as raking factors (test variables). To assess the robustness and accuracy of different combinations of weighting variables, Monte Carlo samples were drawn and demographic distributions of the test variables were generated based on the weights for that particular sample. Mean deviation from the CPS-defined levels for test variables was estimated by averaging over the samples. The combination of variables in Table 16 consistently matched the target distributions of the CPS for a variety of different sample sizes. The pairing of gender with other socio-demographic variables allows one to correct better for discrepancies between distributions within each gender, while avoiding the problem of small cell counts. In other words, implementing the raking algorithm on the set of pairs shown in Table 16 ensures that the distributions of age, ethnicity, and education in the SCPC are matched separately for men and women to their population counterparts in the 28 CPS. Moreover, since bivariate distributions imply marginal distributions for each of the two variables, this approach also guarantees that the distributions of gender, age, ethnicity, and education for the entire SCPC sample are aligned with the corresponding benchmarks in the CPS. The same is true for household size and household income. Table 16: The set of weighting variables. “M” stands for male, and “F” stands for female. The highest income brackets for single households were combined to avoid small cell sizes. M, 18 − 32 F, 18 − 32 Gender × Age M, 33 − 43 M, 44 − 54 M, 55 − 64 F, 33 − 43 F, 44 − 54 F, 55 − 64 M, 65+ F, 65+ Gender × Ethnicity M, White M, Other F, White F, Other M, High School or Less F, High School or Less Single, < $30K Couple, < $30K ≥ 3 , < $30K Gender × Education M, Some College M, Bachelor’s Degree or More F, Some College F, Bachelor’s Degree or More Household Size × Household Income Single, $30K − $59K Single, ≥ 60K Couple, $30K − $59K Couple,$60K − $99K Couple, ≥ $100K ≥ 3 , $30K − $59K ≥ 3 ,$60K − $99K ≥ 3 , ≥ $100K In the second step, the raking algorithm is implemented and sample weights are generated by matching the proportions of predefined demographic groups in the SCPC to those in the CPS. Missing information about education, household size, and income are imputed, if necessary, using ordered logistic regression with gender and age as predictors. Race is imputed using logistic regression. The procedure is sequential, so that variables with the least number of missing values are imputed first and, in turn, used as inputs to impute the variables with the most missing values. Imputations are performed by ordered logistic regression for ordered categorical variables, and by multinomial logistic regression for categorical variables. The order of imputation moves across variables with the least number of missing entries to those with the most, and each level of imputation uses all previous demographic variables. In the case of SCPC, income is the most commonly missing variable. In 2016, 54 individuals had some demographic characteristic imputed, while 36 respondents did so in 2017. We describe the raking algorithm in greater detail below. Let d = 1, . . . , 4 represent the four bivariate demographic categories presented in Table 16. For each demographic, we index the possible groups with the variable g, so that dg represents group g for demographic category d. For example, d = 1 corresponds to the intersection of gender and age and group g = 1 might 29 correspond to males between 18 and 32 years of age. We use the shorthand notation i ∈ dg (k) to indicate that individual i belongs to group g in demographic d. Then, we define wi to be the weight assigned to individual i after iteration k of the raking algorithm. Within each (k,d) such iteration, the raking algorithm iterates across demographic groups d, so we let wi define the assigned weights after iterating over d. In 2016, as in previous years, the base weights serve only to distinguish the nationally representative subset of the UAS panel from those recruited through targeted efforts. In practice, because of the sampling design, this means that every base weight in 2016 has a value of 1.0. In 2017, however, CESR began using more advanced processes for base weights. Sampling of households is based on an initial sampling of U.S. ZIP codes, which is an iterative process that depends on demographic compositions of the U.S. ZIP codes (provided by American Community Survey and the 2010 Urban Area to ZIP Code Tabulation Area). The result is that ZIP codes are not equally likely to be selected, thus even within strata, there is a lack of uniformity. The idea behind the base weights is to adjust for the fact that certain individuals are more likely to be sampled due to residence in more highly sought ZIP codes. Thus, the base weight for each individual corresponds to the inverse of the estimated likelihood of that individual’s household being selected. This is captured by the product of the likelihood that an individual’s ZIP code was selected and the likelihood of that individual’s household being conditionally selected from the ZIP code. The former is modeled as a function of characteristics such as census region, population density, population size, sex, race, age, marital status, and education composition of the ZIP code. The latter is simply a ratio of the number of households selected to the number of households in the ZIP code, because household selection within selected ZIP codes is done at random. Within each iteration, k = 1, 2, 3, . . ., we do the following, mirroring the algorithm found in Valliant, Dever, and Kreuter (2013): (k,0) 1. wi (k−1,4) = wi (k,d) 2. Otherwise, for d = 1, . . . , 4, we let wi P mk,s = (k,d−1) = wi mk,sd [i] , where (k,d−1) wi 1 [i in stratum s] × fd,g , P (k,d−1) wi where fd,g represents the proportion of the U.S. population that belong in group g of demographic d and where 1 [i in stratum s] is 1 if individual i belongs in stratum s and is 0 otherwise. This ensures that, after iteration d, the weighted marginal frequencies in the sample for demographic d will match perfectly those in the population. 30 3. Trim weights by letting w̄(k) represent the average weight within the sample and then assign weight values according to: (k,4) wi (k) (k,4) , if wi < 0.25w̄k 0.25w̄ (k,4) = 4w̄(k,4) , if wi > 4w̄k (k,4) wi else. (1) Therefore, within each iteration, weights that are less than a quarter of the average or more than four times the average are trimmed. Trimming is performed to decrease large weights and increase small weights, thereby decreasing the variation in the weights. While this may sacrifice unbiasedness of estimators it does so by reducing the mean-squared error, which is adversely affected by high variations in weights. The CESR runs 50 iterations of this algorithm or until all marginal weight matching and trimming specifications are achieved. The algorithm for the SCPC data converges in both years. Upon convergence, we let wi represent the weight given to individual i. Weights are standardized to have a mean of 1.0, so the maximum weight is 4.0 and the minimum weight is 0.25. Overall, there are 508 unique weights, meaning 508 out of a possible 660 strata implied by the demographics in Table 16 are represented in the 2016 sample. The change in base weights in 2017 leads to 1,786 unique weights, though around the same number of demographic strata are represented. The standard deviation of weights is 0.8 in 2016 and 1.05 in 2017. The increase represents one greater than what one would expect due to the decrease in sample size alone, which would see an increase of around 5 percent rather than the observed 31 percent. The greater than expected variation is likely due to the use of base weights, which additionally account for a different type of variation relating to the demographics of each individual’s ZIP codes. Because the UAS sample itself is not representative of the U.S. population, post-stratification is an important step in inference for the population. The fact that not all strata of interest are represented in the sample makes raking the natural method for assigning weights. However, doing so introduces a few complications related to the statistical framework and analysis of the data. The first relates to the increased difficulty in calculating standard errors of population estimates, which are weighted averages of the sample values. In all tables and publications, the standard errors have been calculated by taking the weights as fixed values, thereby reducing the standard errors. The sampling weights, which are a function of the strata representation in the sample, are random variables, and their variation should be factored into the calculation of standard errors (Gelman and Lu 2003). However, high 31 response rates and targeted sampling (as described in Section 3.2) mean that the variability in the observed sample composition is small, which in turn implies that the variability in the raked weights is small. Therefore, conditional on a chosen weighting scheme, the variance of our estimators can be attributed largely to the variation in the observed responses themselves and not in the sample composition. The second area of concern regards the effects of the sampling scheme on the weights and on the estimates they produce. In order for the raking algorithm to be appropriate in the sense that the expected weights for each stratum equal those of the population, the sampling procedure must be such that, in expectation, each stratum is proportionally represented in the sample. To be precise, the expected proportion of the sample belonging to a specific stratum is directly proportional to the relative proportion of that stratum within the population. A sampling procedure that does not have this property is likely to consistently produce weights for certain strata that do not reflect the true representation in the entire population. If strata properties correlate with payment behavior, this could lead to biased population-wide estimates. In the case of a sampling procedure in which some strata tend to be overrepresented and others underrepresented, the raking algorithm, which strives to match marginal proportions rather than those of the cross-sections of all the variables, may generate sample weights that fail to align the sample composition with the reference population. Although the sample from the UAS does not perfectly reflect the U.S. population (for example, it tends to have more females than males), the differences between the panel and the broader population are relatively small for the demographics used in weighting. In addition, for many SCPC variables, there is little evidence of strong correlations with these variables used in weighting, so any bias is likely to be small. Overall, comparisons of changes in the estimates based on the SCPC data from year to year are likely to be meaningful. While the estimate levels themselves naturally vary with different weighting schemes, estimates of trends are likely to be more robust. A discussion of using the post-stratification weights to generate per-consumer as well as aggregate U.S. population estimates appears in Section 7.2.1. 6 Data Preprocessing Prior to further statistical analysis, it is important to examine the data carefully and develop a consistent methodology for dealing with potentially invalid and influential data points. As a survey that gathers a large range of information from each respondent, much of it about a 32 rather technical aspect of life that people may not be used to thinking about in such detail or many know little about, the SCPC, like any consumer survey, is susceptible to erroneous input or missing values. This section describes the general types of data preprocessing issues encountered in the SCPC and outlines the general philosophy used in studying the reliability of data at the respondent level. Section 6.1 describes the methodology of imputing missing data, while Section 6.2 describes procedures used to identify and edit data entries that are likely to be erroneous (commonly referred to as “cleaning the data”). Overall, there were no changes in the statistical methodologies used to edit the data from those used in 2015. Nevertheless, the methodologies are described in detail for all edited variables featured in the 2016 and 2017 SCPC. The CPRC uses the edited variables in its analysis, most notably to generate population estimates provided in the SCPC tables, as the edited distributions hopefully better represent the truth. However, in 2015, even after editing, the data corresponding to two variables measuring the frequency of cash withdrawals were so puzzling that the CPRC decided not to publish any estimates of these variables. We continued to collect this data in 2016 and 2017, but we also continue the approach of not publishing estimates based on these variables. A detailed discussion of this decision is provided in Section 6.2.2. Nevertheless, as for all edited variables, both edited and unedited data are released to the public. A guide on which variables were edited and how to access the pre- and post-processed versions of the variables is given in Section 6.3. 6.1 Data Imputation As the post-stratification weights depend on certain demographic variables, the CESR imputes the necessary variables for respondents for whom the information is missing. In the case of many demographic variables, such as age group, gender, or race, missing information can be verified from other surveys taken within the context of the UAS. For household income and household size, both attributes that could easily change within a year, values are imputed by the CESR through logistic regression models for the purpose of creating poststratification weights. The imputations are only used to generate post-stratification weights and are left as missing in the dataset. The CPRC also relies on imputation to edit certain created categorical variables. The types of categorical variables in the SCPC are diverse, ranging from demographic variables, to binary variables (answers to Yes/No questions), to polytomous response variables (multiple 33 choice questions with more than two possible answers). Currently, the data imputation performed on SCPC data relates to identifying missing values as negations of statements within the question or as implying an answer of 0 for numerical responses. This often relates to questions in which respondents are asked binary questions, such as “Do you have an ATM card?” or questions that ask respondents to enter numerical values for a set of related items, such as the number of credit cards owned for several credit card categories or the dollar value stored on different types of prepaid cards. In either of these cases, if at least one of the items features a non-missing response, we impute the values of all missing responses in the same sequence. Specifically, in the case of binary questions, missing variables are coded as “No,” while in the case of numerical values, they are coded as 0. At the moment, no other types of imputations are done, although multiple imputation procedures are being considered for future editions of the survey results. It is very difficult, without making strong assumptions, to identify irregular or erroneous data inputs, especially for multiple choice questions. Research conducted by the CPRC suggests that response bias in sequences of Likert scale questions introduced by a form of anchoring effects is present, but not of economic significance (Hitczenko 2013). See Daamen and de Bie (1992); Friedman, Herskovitz, and Pollack (1994) for general discussion on anchoring effects. Because the item response rates are high, the effect of missing values is not a major concern for the SCPC. Nevertheless, the CPRC is considering developing multiple imputation techniques for missing numerical data entries. 6.2 Data Editing The greatest challenge in data preprocessing for the SCPC comes in the form of quantitative variables, especially those that represent the number of monthly payments or dollar value of cash holdings or withdrawals. Measurement errors in such a context, defined as any incongruity between the data entry and the true response, can be attributed to a variety of sources ranging from recall error to rounding errors to data entry errors or even to misinterpretation of the question. A data entry subject to measurement error can take many forms, but practically the only identifiable forms are those that lie outside the realm of possible values and those that fall in the realm of possibility, but take extreme values. Data entries that defy logic are easily identified by range checks and logical reasoning. The first line of data inspection consists of a basic range and consistency check of the demographic variables to ensure that reported values are logical and that they correspond to established categorical codes. Any response item that fails this check is edited to a missing value. One 34 example is the entry of a negative monthly payment count. A second example of a question in which data entries are potentially changed to missing values is one that first asks respondents whether or not they own various types of credit cards and then asks for the number owned for only the categories that were declared as owned. In such a case, it is technically possible for someone to claim that he or she is an adopter of a card, but, when prompted, say that he or she owns zero of such cards, a clear inconsistency. The CPRC treats responses to questions in any potential sequence as correct until an inconsistency occurs. Then, at all subsequent levels, all responses inconsistent with those to earlier questions are marked as missing. Thus, in the case of credit card adoption, the hypothetical respondent would be recorded as an adopter, but with the number of credit cards owned missing. Identifying data that are possible, but very unlikely, is much more difficult, as it requires assessing the heterogeneity of behavior within the population. This is especially true for economic variables such as cash holdings and value of assets, which are characterized by highly right-skewed distributions. In other words, it is possible that data entries that by some numerical evaluations are statistical outliers are actually accurate and valid. This issue is not unique to the SCPC. Many consumer surveys, such as the Survey of Consumer Finances (SCF) and the Consumer Expenditure Survey (CES) must also tackle the cleaning of such fat-tailed variables. While the details of the preprocessing of outliers are not provided in either survey, the general strategy is similar to that adopted in the SCPC (Bricker et al. 2012; Bureau of Labor Statistics 2013). First, all relevant information in the data particular to each variable is used to identify statistical outliers and inconsistent responses. Then, values that cannot be confirmed or reconciled are imputed. It should be noted that the SCPC does not benefit from in-person interviews (as does the SCF) or multiple phases and modes of interview for each respondent (as does the CES), making it more difficult to identify inconsistent responses. It is important to distinguish conceptually between influential and potentially invalid data points. An influential point is one whose inclusion or exclusion in any inferential analysis causes a significant difference in estimates (Bollen and Jackman 1990; Cook and Weisberg 1982), and thus the influence of a point depends on the statistical procedure being performed. An invalid data entry is, technically, any entry that does not represent the truth. As mentioned above, data cleaning procedures focus predominantly on identifying invalid entries in the tails of the distribution (Chambers and Ren 2004). An invalid data point need not be influential and an influential point is not necessarily invalid. To the degree possible, the procedures adopted by the CPRC rely on economic intuition to identify potentially invalid data entries. Thus, the cleaning procedures for variables for which we have a higher degree of economic understanding seek to identify invalid entries and edit their value. For variables for which there is less economic intuition available, we rely 35 more on raw statistical procedures such as matching known parametric distributions to the data or Cook’s distance to identify influential points in the context of estimating weighted sample means (Cook 1977; Cook and Weisberg 1982). Below, we outline the considerations and economic motivations in cleaning several different variables and provide adopted algorithms for each. The variables relate to the typical number of monthly uses of payment instruments, reported dollar amounts in various contexts, and the number of payment instruments or accounts owned. In the case of cash withdrawals, we argue that new data patterns observed for the reported frequency of cash withdrawal would require much more aggressive editing methodologies in order to be in line with even vague economic priors. As a result, we conclude that this set of questions is yielding implausible responses from a sufficiently large percentage of respondents to justify discarding estimates based on these variables. 6.2.1 Preprocessing: Typical Monthly Payment Use The number of typical payments in a month is an aggregate from data entries for 41 different combinations of payment method and transaction type. The SCPC delineates 10 payment methods, nine payment instruments plus income deduction, and seven transaction types. For example, the use of cash is reported in a series of questions about cash use in the context of paying for a service, for a bill, for a product, or as a payment to a specific person. All combinations of payment method and transaction type are listed in the SCPC User’s Guide (Foster 2014). In addition, for each of the 41 variables, the SCPC allows the respondent to answer on either a weekly, monthly, or annual frequency, so that recall periods better match frequencies of use that are natural to the respondents. Since only “adopters,” defined as those people who claim to possess the payment method, are asked to provide information on use, missing entries for this question are assumed to be zero (for example, a person who has a credit card need not make use of it). Before preprocessing, all 41 payment number if reported by week and variables are standardized to a monthly frequency (multiplied by 52 12 divided by 12 if reported by year). The 10 payment methods are indexed by j = 1, 2, . . . , 10. For each payment method, there is a variety of potential transaction types, k = 1, . . . , Kj . In addition, each data entry is associated with an individual, labeled i = 1, . . . , N , and a year, labeled t = 2014, . . . , 2017. Therefore, Yijkt is the recorded number of typical monthly payments by individual i via payment method j of the k th transaction type for that particular method in year t. Then, PKj Yijt = k=1 Yijkt is the number of reported monthly payments by payment method j in year 36 t and Yit = P10 j=1 Yijt is the number of total number of monthly payments reported in year t. More economic intuition exists about the total number of monthly payments than about which instruments and in what contexts those payments are made. In addition, economic theories dictate that the number of payments made with a particular payment method depends on the payment methods adopted by the individual. The collection of adopted payment methods is called a “bundle.” The general cleaning procedure first identifies a hard threshold for the total number of monthly payments and then, in turn, a bundle-dependent threshold for each payment method. For each payment method, if the reported value exceeds this threshold, the lower-level components are imputed. If an individual component stands out as an outlier, it is winsorized. Otherwise, all components are scaled down to bring the resulting number of payments with the method in question to the threshold, while preserving the relative shares within the payment method. The economic idea behind this latter adjustment is that the individual is likely consistently overestimating use of the payment method. Although the fundamental idea behind the adopted procedure is based on the common approach of using known distributions to identify potential invalid data points, the unique characteristics of payment choice require some additional assumptions. As a result, many aspects of the procedure are based on original ideas developed at the CPRC. This process is described in more detail below and is fully delineated in Algorithm 1. An initial threshold for the total number of monthly payments was assumed to be 300, representing 10 payments per day for 30 days. The bottom panel in Figure 5 shows that this roughly corresponds to the 98th percentile of the raw SCPC data for each year, and is also where the yearly distributions seem to start diverging from one another somewhat more. From a statistical point of view, the ability to pool data to estimate empirical distributions is a great advantage, as pooling enables one to base estimates on more information. In the future, other sources, such as the Diary of Consumer Payment Choice (DCPC), could also be used to inform this threshold. Given a number of monthly payments, the distribution of the number of payments reported for each payment method quite naturally depends on which payment methods are adopted by the individual. A simple model assumes that the number of payments made with each instrument follows a multinomial distribution, conditional on the total number of payment instruments adopted. Thus, the model assumes that with each incoming payment there is some set of probabilities {pj } that correspond to the probability of using payment j. The decision is assumed to be independent for each individual and for each of the necessary 37 Total Number of Payments, Cleaned Number of Payments (Log Scale) 1 4 1 14 42 2 33 234 3 3 3 2 33 111 44 2 33 3 111 44 23 3 444 111 34 34 32 3 3 1 2 2 4 2 24 111 33 444 43 3 43 2 2 43 43 34 34 34 111 3 2 4 4 111111 3 43 32 43 43 43 34 4 3 4 34 4 2 34 11111 3 4 43 43 43 4 4 4 3 3 2 2 245 4 4 4 43 11111 4 4 4 3 4 2 4 4 4 3 3 3 3 4 2 3 4 1 3 4 3 4 2 4433 333 232 2 332 2 3 233 4 2332 11111111 3 4 4 2 34 2 34 43 2 43 34 42 34 43 3 23 24 3 2 2 3 2 3 34 111111 234 4 4 43 43 43 2 243 42 4 34 3 34 3 3 242 2442 244 4 33 3 244 4 43 1111 34 34 3 43 244 34 2 243 244 244 333 2 34 1 1 1 1 12 2 2 2 2 4 4 4 34 334 43 2442 42 33 33 3333 244 4 34 4 34 4 3 3 43 43 42 4 3 3 3 3 1 1 1 1 1 1 1 1 1 1212 242 3 3 4 2 3 3 4 4 2 2 4 4 3 3 4 4 4 4 4 4 4 4 4 4 3 3 3 433333333333333333333 4 4 43 43 2 43 43 4 2 244 244 244 442 33 34 34 2 442 334 3 24 33 34 34 34 444 442 442 34 34 333333333333333333333 148 403 1 2 3 4 2014 2015 2016 2017 0.95 1 0.96 0.97 0.98 0.99 1.00 Percentile Total Number of Payments: Original Number of Payments (Log Scale) 162755 59874 1 2 3 4 2014 2015 2016 2017 2 1 3 334 444 4 3 3 3 3 1 41 1 3 2 1 3 3 3 3 4 3 1 4 34444 2 1 34 32 2981 34 3 3 4 4 222 3 4 43 2 2 1 4 2 43 33 4 3 1 34 34 34 3 4 24 4 43 43 43 2 43 443 3 4 34 4 43 34 443 2332 1 2121 1 3 2 442 442 4 1097 4 4 4 2 4 3 1 42 2 2333 3 1 444 442 44 442 333 44 44 111 233 333 2 1 3 44444 33 21 3 333 442 31 32 44442 1331 1 1 44444 442 2 441 1333 3 444 332 441 444 41 403 333 233 444 23 1 442 1331 444 41 13 1 1 1 33 44 42 3 41 42 4 332 442 2 44 333 332 14 233 1 231 44 4 1 41 14 333 32 2 2 4 4 43 42 43 4 3 4 1 1 4 33 32 4 1 4 4 4 4 4 4 3 4 2 4 4 2 4 4 3 4 3 4 3 2 3 2 4 3 4 3 3 1 3 1 1 3 3 3 4 1 4 4 3 4 2 3 4 3 2 4 3 4 3 1 1 4 3 3 2 4 4 2 3 4 3 4 4 4 1 3 2 3 4 4 3 4 1 4 2 3 1 3 4 1 2 3 2 3 1 3 1 1 1 4 3 4 2 2 2 3 3 1 3 4 3 4 3 2 4 4 3 3 1 4 1 4 3 4 3 4 1 4 2 4 1 3 4 4 4 3 3 4 1 2 2 2 1 2333333333333333333333 13 2 132 233 1 2332 2 233 2 2 333 332 333 333 332 33 3 33 3 34 148 22026 8103 0.95 0.96 0.97 0.98 0.99 1.00 Percentile Figure 5: The log values of the largest 5 percent of the total monthly payments data before and after processing for the past four years. Source: Authors’ calculations payments and to depend only on the individual’s adoption choices. While this assumption may not hold completely (for example, the choice of payment method might depend on the dollar value of the transaction), it is a suitable approximation for the purposes of identifying likely invalid data points. To make this more concrete, for individual i in year t, let Bit be the bundle adopted by individual i; for example, Bit = {1, 2} for an individual who adopts only cash and checks. In order to account for the fact that certain payment methods are used much more often than others yet to keep the calculations simple, the probabilities, {pj }, are assumed to be proportional to the relative prevalence of the adopted payment methods to one another. Thus, for j = 1, . . . , 10, rj is defined as the weighted mean of the bottom 95 percent of the number of monthly payments made by method j in the raw data. The 95th percentile is used to prevent undue influence of outliers, and changing this percentile does very little to change the relative prevalence. The intuition then is that rj represents a prior sense of the typical monthly rate of use of payment method j among the population. Based on the chosen rj , the approximated proportion of payments made by individual i with 38 payment method j in year t, defined as pijt will be pijt = P rj j 0 ∈Bit rj 0 1{j∈Bit } . The value pijt is a probability and the distribution of these values will be the same for every individual with the same bundle of payment methods. It should be noted that calculations of pijt are dependent not only on the prior assumptions but also on the assumption that using one payment method does not influence the relative use rates of the other methods. As an example, this means that the relative use ratio of cash to check does not depend on whether or not the individual uses credit cards. While this might be a strong assumption, it is one that avoids the need to make many assumptions about joint use rates for various bundles of payment methods. The cutoffs for each payment method are then defined as the 98th percentile of the number of monthly payments, with 300 total payments and probability of use pijt . Therefore, if Yijt ∼ Binomial(300, pijt ), the cutoff cijt is defined to be such that Prob(Yijt ≤ cijt ) = 0.98. Based on this, yijt is flagged whenever yijt > cijt . This flag indicates that the reported value is unusually high when taking into account the payment methods adopted. It is only at this point that the lowest level of data entry, yijkt , is studied. Because little intuition exists about the distributions of the yijkt , comparisons of flagged values are made to the 98th percentile of the empirical distribution estimated by pooling data from the past three years. Specifically, let qjk be the 98th percentile of the pooled set of data comprised of the yijkt for t = 2008, . . . , 2014 among people for all (i, t) for which j ∈ Bit . Then, for each flagged payment method, the flagged entry is imputed with the minimum of the calculated quantile ∗ and the entered value: yijkt = min(yijkt , qjk ). This form of winsorizing means that extremely high reported numbers are brought down to still high, but reasonable levels. If none of the data entries at the lowest level are changed, all yijkt for the payment method j are scaled down proportionally in order to bring the total for the payment method down to the cutoff value cijt . Once data at the lowest level of input are cleaned, aggregated values can naturally be reconstructed. Figure 6 shows the implied number of total monthly payments before and after preprocessing (on the log scale), and Figure 5 also shows the top 5 percent of edited payment totals. The fatter right tail observed in 2014 may reflect an underlying truth, but is also 39 consistent with natural sampling variation. A feature of this algorithm is that, although it uses 300 as a threshold to flag the total number of reported payments, it does allow individuals to have more payments if reported numbers at the lowest level are consistent with others’ responses. In each year, there are individuals with as many as 400 monthly payments. Figure 6 also indicates that the smallest number of payments to be edited is around 55, although the changes to the number of payments made are relatively small. Changes on this scale are due to a majority of the reported number of payments being reported for a payment instrument that has very low typical use, such as money orders. Total Number of Payments: 2014 Total Number of Payments: 2015 403 ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 7 ● 55 ● Cleaned Values (Log Scale) 7 55 Cleaned Values (Log Scale) 403 ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 7 55 403 2981 22026 7 Original Values (Log Scale) 403 2981 22026 Total Number of Payments: 2017 ● ● ● ● ●● ● ● ●● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 403 ● ● ● ● ● ● ● ● ● ● ● ● 55 ● 7 7 55 ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Cleaned Values (Log Scale) 403 Total Number of Payments: 2016 Cleaned Values (Log Scale) 55 Original Values (Log Scale) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 7 55 403 2981 ● 22026 7 55 403 2981 22026 ● ● Original Values (Log Scale) Original Values (Log Scale) Figure 6: The log values of the cleaned total monthly payments data plotted against the log values of the original values. Source: Authors’ calculations 6.2.2 Preprocessing: Cash Withdrawal A second concept that requires a fair amount of attention in terms of preprocessing is that of cash withdrawal. We begin by describing the editing algorithm. We then argue that the distribution of data related to the frequency of cash withdrawal is so different compared with other years that we conclude the survey question is not doing an adequate job of capturing the truth for a substantial subset of the population. For this reason, we suppress population 40 Algorithm 1 Preprocessing: Number of Monthly Payments for i = 1 : N do Determine Bit for j ∈ Bit do Calculate pijt and then cijt if yijt > cijt then Set change.subtotal = 0 {used to keep track if yijkt are changed} for k = 1 : Kj do if yijkt > qjk then Set yijkt = qjk Set change.subtotal = 1 end if end for if change.subtotal = 0 then for k = 1 : Kj do c Set yijkt = yijkt × yijt ijt end for end if end if end for end for estimates for all economic concepts based on cash withdrawal frequency from 2015 through 2017. Preprocessing Algorithm Cash withdrawal since the 2009 SCPC is reported as a combination of four separate variables: frequency of withdrawal at primary and all other locations and typical dollar amount per withdrawal at primary and all other locations. Because reported dollar amounts correspond to typical values, which could represent the mean, the median, or the mode, the value determined by multiplying the reported frequency and the dollar amount does not necessarily correspond to the average total cash withdrawal either for the primary or for all other locations. In preprocessing, data for the primary and for all other locations are treated separately. The editing process is described below. Assuming that N independent individuals report positive cash withdrawal in a typical month, let Cit = Ait Fit be the typical monthly value of all cash withdrawals, where Ait is the reported amount per visit in year t and Fit is the reported frequency of monthly visits in year t. In the case of cash withdrawals, information about the tails comes from distributional assumptions, so empirical estimates that rely on pooling data across years for more statistical power are not necessary. As a result, the subscript corresponding to year t is dropped for simplicity. 41 If Ci ∼ Log-Normal(µW , σW ) with independence across individuals, then it follows that log(Ci ) = log(Ai ) + log(Fi ) has a normal distribution, which in turn means that log(Ai ) and log(Fi ) are also normally distributed. The fact that individuals who withdraw a larger value of cash will likely need to do so fewer times than those who take out smaller values suggests a negative correlation between the two variables. Thus, the joint distribution will take the form " log(Ai ) log(Fi ) # " ∼N µA µF # " , σA2 ρAF ρAF σF2 #! , with ρAF likely to be negative. For simplicity of notation, let Wi = [log(Ai ) log(Fi )]T , where the superscript T refers to a matrix transpose, and let µ and Σ represent the respective mean and covariance of Wi . In order to determine distributional outliers, consider that if Λ is such that ΛT ΛΣ = I2 , the 2 × 2 identity matrix, (in other words, Λ is the cholesky decomposition of Σ−1 ), then the set of Zi = ΛT (Wi − µ) will be independent draws from a two-dimensional standard normal distribution. For the bivariate standard normal, Di = kZi k is the Euclidean distance of the ith draw, Zi , to the point (0, 0). Also, if f (· | 0, I) is the density function of the bivariate standard normal distribution, then Di2 > Di20 implies f (Zi | 0, I) < f (Zi0 | 0, I). This implies that if Di2 = Di20 , then the density at Zi is equal to that at Zi0 , which is why the bivariate standard normal curve has circular contour lines. The contour lines of a bivariate normal distribution with mean µ and variance Σ will be an ellipse centered at µ with points Wi and Wi0 having the same densities if and only if (Wi − µ)T Σ−1 (Wi − µ) = (Wi0 − µ)T Σ−1 (Wi0 − µ). Transforming the N independent draws from the true distribution to N independent draws of the bivariate distribution makes it easier to work with the data. This transformation preserves the sense of distance from the mean with respect to the assumed density (which is lower for less likely points and decreases as one moves away from the mean). Therefore, if Wi and Wi0 are such that Di2 > Di20 , then f (Wi | µ, Σ) < f (Wi0 | µ, Σ). So, the extremity of each of the N points can be measured by comparing the distances Di2 . It is known that Di2 are independent and identically distributed random variables from the Exp(0.5) or equivalently a Chi-Square(2) distribution. Therefore, we can easily determine 42 the 98th percentile for Di2 , which we call q.98 . Algorithm 2 Preprocessing: Monthly Cash Withdrawal Let wi = (log(ai ), log(fi )) for all i = 1, . . . , N Estimate µ̂ = mean(wi ) and Σ̂ = var(wi ) from sample statistics of the wi Calculate Λ̂ such that Λ̂T Λ̂ = Σ̂−1 Calculate q.98 based on µ̂ and Σ̂ for i = 1, . . . , N do Calculate zi = Λ̂T (wi − µ̂) Calculate d2i = kzi k2 if d2i ≤ q.98 then Calculate zknew Calculate wknew = µ̂ + Λ̂−T zknew Replace wk with wknew end if end for Keep changes to wi only if log(ai ) < µ̂A and log(fi ) < µ̂F . For all observation pairs for which Di2 > q.98 , the procedure reassigns the data entry to a point more consistent with the fitted distribution but a minimum distance from the original √ value. Specifically, the data point is reassigned so that its new distance is exactly q.98 . The imputation procedure is exactly the same as in previous years. First, Zi is reassigned to Zinew , which corresponds to a well-known constrained optimization problem. Namely, Zinew is such that kZinew − Zi k (the distance between the old and new points) is minimized, subject to the condition kZinew k2 = q.98 . Optimization programs for this paradigm are available for most computational packages (Press et al. 2007). The new value, Zinew , is then converted from the standard normal distribution to a corresponding value on the bivariate normal distribution defined by µ and Σ by letting Winew = µ + Λ−T Zinew . In practice, µ and Σ are not known and must be estimated from the data. We use lower-case notation, such as wi = (log(ai ), log(fi )), to represent the actual values observed in any given survey year, and estimate the bivariate mean and covariance with µ̂, the sample mean, and Σ̂, the sample covariance. The entire procedure is outlined in Algorithm 2. This procedure results in the editing of observations that are extreme with respect to the general mass of the sample data, even if the total monthly dollar value is reasonable. For example, if a person reports an amount of $1 per withdrawal and a frequency of 0.25 withdrawals per month, the corresponding pair on the log-scale will be (0, −1.38), which could 43 be determined to be extreme given the much higher average values of frequency and amount. Thus, additional rules to exclude points from the editing procedure above may be desired. One option is not to edit any pairs for which the implied monthly dollar total is below some threshold. A second option is to consider outliers by the quadrant they lie in. For the SCPC data, a rule is imposed so that no changes are made to data for which log(ai ) < µ̂A and log(fi ) < µ̂F . Discussion of Cash Withdrawal Frequencies Even after the editing has been done, the cleaned variable for cash withdrawal frequency has shown vastly different distributions especially in the tail over the past few years. In particular, it is characterized by a much fatter right tail in 2014 and 2015, meaning there are many more instances of a very high number of monthly cash withdrawals. Figure 7 shows the 98th percentile of the bivariate normal distribution estimated to fit the log of the dollar amount per withdrawal and the log of the number of monthly withdrawals for the past four years of data. This contour is significant because it is used as the cutoff for trimming in the algorithm described above. Clearly, while the distributions of the log dollar amounts are fairly similar across years, the distribution of frequencies is significantly different in 2015 from those in other years. The jump from 2014 to 2015 is of a much greater magnitude. The results of a further investigation are shown in the technical appendix for the 2015 SCPC, but no explanation was found for the change. Rather than impose a very aggressive editing algorithm and thus affect a large portion of the data, the CPRC has concluded that this set of questions simply did not collect reliable data. Although the tails of the distribution of withdrawal frequency are much more reasonable in 2016 and 2017, as can be seen in Figure 7, the history of this variable suggests it has flaws as a way of collecting data. Until more research is done and a better understanding of the nature of the responses is reached, these variables are simply not used to generalize to the U.S. population. Therefore, the official table of results does not provide estimates of the frequency of cash withdrawals or of the total dollar value withdrawn per month. Nevertheless, the raw and cleaned data are released in the official dataset. Information about cash withdrawals can be found in the DCPC, which asks respondents to report all cash activity. 44 98% CI: Primary Location Amount (Log Scale) 2981 403 55 2014 2015 2016 2017 7 1 7 20 55 148 Frequency (Log Scale) 98% CI: All Other Locations Amount (Log Scale) 2981 403 55 2014 2015 2016 2017 7 1 7 20 55 148 Frequency (Log Scale) Figure 7: 98th percentiles of estimated bivariate Normal distribution for cash withdrawal data (on the log scale) from 2010 to 2015. Source: Authors’ calculations 6.2.3 Preprocessing: Cash Holdings The SCPC also collects the dollar value of cash holdings. This concept is collected as two variables: the value of cash holdings on person and the the value of cash holdings stored at home (or other locations). We treat each variable separately, as there is no obvious relationship that one would expect to exist between the two. For the dollar values, we adopt the one-dimensional version of Algorithm 2 used to clean the cash withdrawal variables. Because other than in dimension the algorithms are identical, we do not provide more information about the procedure or delve into any details. Figure 8 shows the distribution of the right tails of cash holdings for each of the two variables. As indicated, this cleaning procedure results in no edits to the cash holdings on person. The maximum reported values for the four years range from $1,600 to $6,700. According to our cleaning algorithm, the presence of other observations of this magnitude suggests that there is not enough evidence to edit these values. These values are large, and it is certainly possible, maybe even likely, that an input error caused $67.00 to be coded as $6,700. At the same time, the reported values are plausible; cash transactions approaching a value of 45 $10,000 have been observed in the DCPC. 8103 ● ● ● ● ● ● ● ● 403 ● ● ● ● 20 Dollar Value (Log Scale) Cash Holdings on Person 2014 2015 2016 2017 ● ● Year ● 1202604 * * ● ● ● * ● ● ● 148 2981 Dollar Value (Log Scale) Cash Holdings in House 2014 2015 2016 ● ● ● 2017 ● ● ● Year Figure 8: Boxplots of right tails of cash holdings. The asterisk represents the only edited value. Source: Authors’ calculations For cash holdings at home, no edits were made in either year. The highest values reported were $45,000 in 2016 and $28,000 in 2017. There were two values over $20,000 in the former and one in the latter. The nature of the right tail of this distribution is less intuitive, perhaps because many individuals do not disclose the amount of cash stored at home to others. As a result, it is more difficult to say whether the observed values are reasonably likely or not. As we do not have much economic intuition, we adopt a more conservative approach and let the data and the results of the algorithm stand without imposing stricter standards. 6.3 Summary of Edited Variables In this section, we summarize the variables that are edited by the CPRC. In most cases, the edited variables are created by the CPRC as a function of various survey variables, which are any variables directly measured in the SCPC. In such cases, the underlying survey variables and any other underlying created variables that define the concept of interest are left unedited. The exceptions are the payment use variables, where the frequency-converted 46 survey variables are edited. The original payment use survey variables remain unedited and are still reported in weekly, monthly, or yearly frequencies. Any created variables that are defined by survey variables that are potentially edited have values determined by the edited version of those survey variables. For example, all variables relating to payment use, such as “csh typ,” which defines the number of cash payments, are aggregates of the lowest-level entries for payment use defined by a combination of payment method and transaction type. All statistics for payment use variables are created using the cleaned versions of data for each combination of payment method and transaction type. Thus, researchers who are interested in comparing the unedited variables must reconstruct any created variables themselves. All unedited variables are available and are denoted by “ unedited” or “ unedit” (in order to keep variable names below a certain number of characters) at the end of the variable name. For example, “csh wallet 1st” holds all edited entries for the dollar value of cash holdings on person, while “csh wallet 1st unedited” defines the unedited version of the data. Table 17 lists all variables that are edited by the CPRC. 47 Table 17: Summary of edited variables. “Underlying variables” are any survey or created variables that are used to define some created variable. Variables Cleaned (Description of Algorithm) Payment Instrument Use (Section 6.2.1) pu002 a, pu002 b, pu002 c, pu002 d, pu002 e, pu003 a, pu003 b, pu003 c, pu003 d, pu004 a, pu004 b, pu004 bmo, pu004 c, pu004 d, pu004 e, pu005 a, pu005 amo, pu005 b, pu005 c, pu005 d, pu005 e, pu006a a, pu006a b, pu006a bmo, pu006a c, pu006a d, pu006a e, pu006c a, pu006c b, pu006 bmo, pu006c c, pu006c d, pu006c e, pu021 a, pu021 b, pu021 bmo, pu021 c, pu021 d, pu021 e, pu021 f, pu008 c Cash Withdrawal Value (Section 6.2.2) csh amnt 1st, csh freq 1st, csh amnt 2nd, csh freq 2nd Cash Holdings Value (Section 6.2.3) csh wallet, csh house Notes Variables based on these variables use edited data. Underlying variables remain unedited. Population estimates based on csh freq 1st and csh freq 2nd are not generated. Underlying variables remain unedited. 48 7 Population Parameter Estimation The main goals of data collection in the SCPC are to produce estimates of consumer payment behavior for the entire population of U.S. consumers and to monitor changes from one year to the next. This section presents the model that provides a framework for achieving both of these goals. This framework will work within the assumption of a longitudinal data structure, both looking forward to the future UAS panel and matching previous expositions relating to the ALP data. The model is presented in a general way so that it can easily be applied to a variety of measured variables, ranging from binary measurements of payment instrument adoption to count data such as the typical number of monthly payments. Let Yijt be the measurement for person i, for variable j = 1, . . . , J in year t = 1, . . . , T . In the context of the number of monthly payments, for example, j could correspond to the number of payments made with payment method j. Within all observed data, the respondent identifier i ranges from 1 to N , where N represents the total number of unique respondents in all T years. In fact, for certain variables, N can be lower due to a paucity of observations. As discussed, the rate of item non-response is low in the SCPC, so estimates are simply based on the observed data, with the weights of non-responders distributed evenly across those who did respond. A natural representation for the population mean with respect to some stratification of the population into disjoint strata, indexed by s, is µjt = X fs µjt [s], (2) s where fs refers to the relative proportion of stratum s in the population (so that and µjt [s] is the average value observed in stratum s for variable j in year t. P s fs = 1), We are most generally interested in estimating µj = [µj1 µj2 . . . µjT ]T . To this end, we use the following specifications: E [Yijt ] = µjt [si ] 2 Var [Yijt ] = σjt (3) Cov [Yijt , Yijt0 ] = ρjtt0 , where si represents the stratum of individual i. Note that we do not specify a distribution for responses, so the approach can be generally applied to continuous variables, count data, and binary variables. We also make assumptions about the data dependence, most notably that 49 the variance of data within strata is fixed across all strata. We also assume independence in responses across individuals (even if they are in the same household) and within individuals, but for different variables. Such assumptions are standard for most surveys and should not affect the expected values of estimates, just the associated standard errors. In order to provide the formulas for estimating the population parameters as a function of the observed sample, we introduce the following variables. Let Njt represent the number of responses obtained for variable j in year t, and let Njtt0 represent the number of respondents P who gave responses for variable j in both year t and year t0 . Defining Nj = Tt=1 Njt , let Yj be the Nj × 1 vector with all of the responses relating to variable j over all T years. In addition, let Xj be a Nj × T matrix defined as follows. The (k, t)th element of the matrix, Xj [k, t], will be 1 if the k th element of Yj was observed in year t, and 0 otherwise. Finally, Wj is an Nj × Nj diagonal matrix such that the k th element of the diagonal corresponds to the weight of the individual corresponding to the k th element in Yj in the year when that observation was made. Then, according to established theory (Lohr 1999), the estimates of the population vector µj will be µ̂j = XTj Wj Xj −1 XTj Wj Yj . (4) Before we proceed, note that the population estimates calculated from the model, given in (4), correspond to the natural, design-based estimates given by the SURVEYMEANS procedure in SAS (SAS Institute Inc. 1999). Namely, if we define Sjtt0 to be the index of all respondents who provided a valid data entry for variable j in year t and t0 , then P µ̂jt = i∈Sjtt P wit yijt i∈Sjtt wit . (5) To see how the estimate in (5) mimics the model form in (2), we rewrite (5). Let mst represent the number of respondents who belong to stratum s, and thus have identical weights. Then, let wst represent the weight associated with each individual in stratum s. In other words, wst = wit for all i such that si = s. Then, we let ȳsjt = 1 X yijt 1 [si = s] mst i∈S jtt be the observed sample mean for all respondents in stratum s. Then, grouping individuals 50 by strata leads to µ̂jt = X s w m P st st ȳsjt . s wst mst (6) Naturally, the sample mean, ȳstj , serves as an estimate of the true stratum mean µjt [s] for each s, and the relative weights assigned to each stratum are designed to have expectation equal to fs , the true frequency of stratum s in the population. If one assumes independence between the weights and the sample observations, the implication is that E [µ̂jt ] = µjt . It should also be noted that although the point estimates of the µj are the same as those in a weighted least squares analysis, we are conceptually fitting a regression model with weights designed to scale the sample data to generate estimates for a finite population (see Lohr 1999, section 11.2.3). Therefore, unlike in the weighted-least squares case, the covariance of the estimates, Λj = Cov(µj ) will be estimated by Λ̂j = XTj Wj Xj −1 XTj Wj Σ̂j Wj Xj XTj Wj Xj −1 , where Σ̂j is the Huber-White sandwich estimator of the error variances, Var(Yj ) (Eicker 1967; Huber 1967; White 1980). In this context, this means that 2 σ̂jt = X 1 (ykjt − µ̂jt )2 Njt − T k∈S jtt and ρ̂jtt0 = X 1 (ykjt − µ̂jt ) (ykjt0 − µ̂jt0 ) . Njtt0 − T k∈S jtt0 In addition to the important population means µ̂j , the analysis above gives the estimates’ covariances Λ̂j . The square roots of the diagonal entries of Λ̂j correspond to the standard errors of the yearly mean estimates. The standard errors for the population estimates corresponding to the 2008 – 2017 SCPC are available at https://www.frbatlanta.org/ banking-and-payments/consumer-payments.aspx. 7.1 Panel Effects The model in (4) easily allows for the introduction of panel effects. To do so, we introduce a new variable, pi , that indicates which panel individual i is from. For example, p = 1 might 51 correspond to the ALP, and p = 2 might represent the UAS. The most general manifestation of panel effects on first moment estimates is by extending (4) to: E [Yijt ] = µjt [si , pi ], so that the expected response depends on the panel itself. The way in which the panel selection affects the expectation can vary, but a relatively simple model is one in which it has an additive effect that is fixed across strata, but not time: E [Yijt ] = µjt [si ] + λjt [pi ], where λjt [p] represents an additive bias associated with panel p. Under such a model, the weighted estimate given in (5) applied to data from panel p will be such that E [µ̂jt [p]] = µjt + λjt [p]. This unknown panel effect can make it difficult to compare estimates from different panels. A simple example is seen in Figure 9, which shows a time-series of estimates for payment instrument adoption rates based on data from the ALP and UAS from 2013 to 2015. Likely panel effects are most obvious in 2014, since temporal changes cannot be used to explain differences between the two panels. While adoption of certain instruments, such as credit cards (cc adopt), shows a fair amount of consistency across panels, most yield non-overlapping confidence intervals for 2014 estimates. Developing a better understanding of panel effects and developing a comprehensive way to generate realistic trend estimates that incorporates all years of data is a high priority for the CPRC. Although certain economic measures, such as share of payments made by payment instrument, show much more consistency across panels, until a broad methodology for assimilating estimates from different panels is adopted, the CPRC refrains from making comparisons of estimates across different panels. However, a general approach to such an endeavor would be to assign a distribution to the panel effects, likely on a log-scale so that λjt [p] or perhaps its log are draws from a Normal(0, σp2 ), and simultaneously estimate σp with other population means from the data. Greater discrepancies between panel results will correspond to greater uncertainty about the true mean, which manifests itself in larger standard errors. 52 chk_adopt cc_adopt 0.95 0.80 0.90 0.75 dc_adopt svc_adopt 0.90 banp_adopt 0.75 0.70 0.85 ● 0.85 ● ● 0.70 ● ● 0.60 2013 2014 2015 0.50 0.75 2013 chk_adopt 2014 2015 cc_adopt 2014 ● ● ● 0.60 2015 dc_adopt 2014 2015 0.20 ● ● ● 0.50 2013 svc_adopt ● 0.55 ● 0.55 2013 ● ● ● ● 0.45 2013 0.60 0.65 0.55 0.65 0.75 ● ● 0.80 0.25 ● 0.60 ● 0.80 0.65 ● 0.70 ● 0.30 0.75 0.65 ● ● mon_adopt 0.70 ● ● ● obbp_adopt 0.80 2014 2015 2013 banp_adopt 2014 0.15 2015 2013 obbp_adopt 2014 2015 mon_adopt 0.95 0.80 0.90 0.75 0.80 0.70 0.30 Figure 9: Estimates of payment instrument adoption rates and 95 percent confidence intervals based on ALP data in 2013 – 2014 and UAS data in 2014 – 2015. 0.70 0.75 ● 0.90 0.75 0.65 0.85 0.65 0.25 ● Source: Authors’ calculations. 0.70 ● ● 0.85 ● ● ● ● 0.70 ● ● 7.2 ● 0.60 ● ● 0.80 ● ● 0.65 ● 0.80 ● 0.55 Functions of Population Means ● 0.65 ● 0.50 0.60 ● ● ● ● ● 0.55 ● 0.20 ● 0.60 ● 0.75 0.60 2013 2014 2015 0.75 2013 2014 2015 0.45 2013 2014 2015 0.55 2013 2014 2015 ● 0.50 2013 2014 2015 2013 2014 2015 0.15 2013 2014 2015 While the most interesting population parameters are the per capita population means, µjt in (2), we are also interested in some variables that are functions of these population parameters. Perhaps the two most illuminating functions from an economic standpoint are the growth rates and the shares. In this work, we choose to work with the macroeconomic definition of each, meaning that we consider the growth rate of averages rather than the average of the individual growth rates. We thus let gjt = µj,t+1 − µjt µjt (7) be the growth rate of variable j from year t to t + 1, and µjt sjt = PJ k=1 µkt (8) be the share of variable j in year t. The macroeconomic definitions used in (7) and (8) should be contrasted with their microeconomic alternatives. The former involve defining individual shares for each variable, s ijt = yijt PJ and estimating sjt by applying (5) to this individual variable. The macroeconomic k=1 yikt approach is statistically sounder, as, under most models that treat individuals as independent, it will give the maximum likelihood estimates of the parameters in question. For example, if the total number of payments for person i at time t is Yit modeled as a Poisson random variable and the number assigned to variable j, Yijt is a binomial distribution conditional on YP it with probability pjt , then the maximum likelihood estimates for the pjt will P Yijt Y be given by Pi Yijt rather than i N Yit (in this example, we have made all weights equal to it i 53 simplify the equations). Thus, throughout this analysis, we generally use the macroeconomic definitions. 7.2.1 Generating U.S. Aggregate Estimates The term µjt in (2) represents a per capita average in year t. For example, if the variable of interest is the number of payments made in a typical month with cash, then µjt represents the average of this value with respect to all U.S. adult consumers. In theory, if µ̂jt is an estimate of this mean, then a corresponding estimate for the aggregate number among the entire population would be µ̂jt multiplied by the size of the population. However, such calculations must be taken with caution. The estimates of µjt from the SCPC are likely to be fairly variable due to the relatively small sample size and variation in the post-stratification weights. Thus, while the estimates might be unbiased, any one estimate based on a particular sample is potentially a relatively poor estimate of µjt . Any difference between µ̂jt and µjt is magnified when multiplied by the U.S. population, making the resulting estimate a potentially poor estimate of the population aggregate. The high degree of error in these aggregate estimates is the reason we recommend that such methodologies be employed with caution. Issues of bias in the estimates could arise as a result of the sampling instrument and potential measurement errors. For example, the SCPC asks respondents for their personal rather than household payment choices. Inability to clearly delineate all payments related to the household, such as bills, could lead to systematically inaccurate responses. 7.2.2 Data Suppression Many population estimates in the SCPC are based on a subset of the sample. For example, estimates for adopters of payment instruments are naturally based only on respondents who claimed to be adopters of the payment instrument in question. In some cases, the set of eligible respondents can be quite small, resulting in an unreliable estimate. As a result, in the data tables found in the SCPC report, estimates that are based on a small number of responses are suppressed. The CPRC uses two thresholds: one for categorical data and one for numerical data. The threshold for categorical data is 20 while that for numerical data is 50. That is, if the number of respondents is lower than the corresponding threshold, the estimated population average is not reported in the tables. Numerical data are given a higher threshold because many of the variables, such as those relating to dollar amounts or number of uses, are heavy-tailed 54 and therefore highly variable. Thus, a larger number of responses is required to produce reasonably reliable estimates. As can be seen in Klein et al. (2002), which details rules for suppression in various surveys, the thresholds adopted by the CPRC are comparable to those adopted by other U.S. government agencies. 55 References Angrisani, Marco, Kevin Foster, and Marcin Hitczenko. 2013. “The 2010 Survey of Consumer Payment Choice: Technical Appendix.” Federal Reserve Bank of Boston Research Data Report 13-3. Angrisani, Marco, Kevin Foster, and Marcin Hitczenko. 2014. “The 2011 – 2012 Survey of Consumer Payment Choice: Technical Appendix.” Federal Reserve Bank of Boston Research Data Report 14-2. Angrisani, Marco, Kevin Foster, and Marcin Hitczenko. 2015. “The 2013 Survey of Consumer Payment Choice: Technical Appendix.” Federal Reserve Bank of Boston Research Data Report 15-5. Angrisani, Marco, Kevin Foster, and Marcin Hitczenko. 2016. “The 2014 Survey of Consumer Payment Choice: Technical Appendix.” Federal Reserve Bank of Boston Research Data Report 16-4. Angrisani, Marco, Kevin Foster, and Marcin Hitczenko. 2017. “The 2015 Survey of Consumer Payment Choice: Technical Appendix.” Federal Reserve Bank of Boston Research Data Report 17-4. Baltagi, Badi H. 2008. Econometric Analysis of Panel Data. Hoboken, New Jersey: John Wiley and Sons. Bollen, Kenneth A., and Robert W. Jackman. 1990. “Regression Diagnostics: An Expository Treatment of Outliers and Influential Cases.” In Modern Methods of Data Analysis, eds. John Fox and J. Scott Long, 257–291. Newbury Park, CA: Sage. Bricker, Jesse, Arthur B. Kennickell, Kevin B. Moore, and John Sabelhaus. 2012. “Changes in U.S. Family Finances from 2007 to 2010: Evidence from the Survey of Consumer Finances.” Federal Reserve Bulletin 98(2). Bureau of Labor Statistics. 2013. “Consumer Expenditures and Income.” In BLS Handbook of Methods. BLS Publishing. CES. Various Years. “Consumer Expenditure Survey.” http://www.bls.gov/cex/home. htm. Chambers, Raymond L., and Ruilin Ren. 2004. “Outlier Robust Imputation of Survey Data.” The Proceedings of the American Statistical Association. 56 Cook, R. Dennis. 1977. “Detection of Influential Observations in Linear Regression.” Technometrics 19(1): 15–18. Cook, R. Dennis, and Sanford Weisberg. 1982. Residuals and Influence in Regression. New York, New York: Chapman and Hall. Daamen, Dancker D. L., and Steven E. de Bie. 1992. “Serial Context Effects in Survey Interviews.” In Context Effects in Social and Psychological Research, eds. Norbert Schwarz and Seymour Sudman, 97–113. Springer-Verlag. DCPC. Various Years. “Diary of Consumer Payment Choice.” Federal Reserve Bank of Boston. De Leeuw, Edith D. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys.” Journal of Official Statistics 21(5): 233–255. Deming, W. Edwards, and Frederick F. Stephan. 1940. “On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Tables are Known.” The Annuals of Mathematical Statistics 11: 427–444. Duncan, Greg J., and Graham Kalton. 1987. “Issues of Design and Analysis of Surveys Across Time.” International Statistical Review 55: 97–117. Eicker, F. 1967. “Limit Theorems for Regression with Unequal and Dependent Errors.” Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability 59–82. Foster, Kevin. 2014. “SCPC User’s Guide: 2011 – 2012.” Technical report. Consumer Payment Research Center, Federal Reserve Bank of Boston. Foster, Kevin. 2018. “SCPC User’s Guide: 2016 – 2017.” Technical report. Consumer Payment Research Center, Federal Reserve Bank of Boston. Foster, Kevin, Scott Schuh, and Hanbing Zhang. 2013. “The 2010 Survey of Consumer Payment Choice.” Federal Reserve Bank of Boston Research Data Report 13-2. Frees, Edward W. 2004. Longitudinal and Panel Data: Analysis and Applications in the Social Sciences. Cambridge, UK: Cambridge University Press. Friedman, Hershey, Paul Herskovitz, and Simcha Pollack. 1994. “Biasing Effects of ScaleChecking Style in Response to a Likert Scale.” Proceedings of the American Statistical Association Annual Conference: Survey Research Methods 792–795. 57 Gelman, Andrew, and Hao Lu. 2003. “Sampling Variances for Surveys with Weighting, Post-stratification, and Raking.” Journal of Official Statistics 19(2): 133–151. Hitczenko, Marcin. 2013. “Modeling Anchoring Effects in Sequential Likert Scale Questions.” Federal Reserve Bank of Boston Working Paper 13-15. Hitczenko, Marcin. 2015. “Identifying and Evaluating Selection Bias in Consumer Payment Surveys.” Federal Reserve Bank of Boston Research Data Report 15-7. Huber, Peter J. 1967. “The Behavior of Maximum Likelihood Estimates Under Nonstandard Conditions.” Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability 221–233. Kennedy, Courtney, Andrew Mercer, Scott Keeter, Nick Hatley, Kyley McGeeney, and Alejandra Gimenez. 2016. “Evaluating Online Nonprobability Surveys.” Technical report. Klein, Richard J., Suzanne E. Proctor, Manon A. Boudreault, and Kathleen M. Turczyn. 2002. “Healthy People 2010 Criteria for Data Supression.” Technical report. Centers for Disease Control and Prevention. Little, Roderick J. A., and Donald B. Rubin. 2002. Statistical Analysis with Missing Data. New York, New York: Wiley. Lohr, Sharon L. 1999. Sampling: Design and Analysis. California: Brooks/Cole Publishing. Lynn, Peter. 2009. Methodology of Longitudinal Surveys. Hoboken, New Jersey: John Wiley and Sons. Press, William H., Saul A. Teukolsky, William T. Vetterlin, and Brian P. Flannery. 2007. Numerical Recipes: The Art of Scientific Computing. New York, New York: Cambridge Univeristy Press, 3rd ed. SAS Institute Inc. 1999. SAS/STAT User’s Guide, Version 8. SAS Institue Inc., Cary, NC. SCF. Various Years. “Survey of Consumer Finances.” http://www.federalreserve.gov/ econresdata/scf/scfindex.htm. Stavins, Joanna. 2016. “The Effect of Demographics on Payment Behavior: Panel Data with Sample Selection.” Technical report. Valliant, Richard, Jill A Dever, and Famke Kreuter. 2013. Practical Tools for Designing and Weighting Survey Samples. New York, NY: Springer. 58 Wang, Wei, David Rothschild, Sharad Goel, and Andrew Gelman. 2009. “Forecasting Elections with Non-Representative Polls.” Public Opinion Quarterly 73(5): 895–916. White, Halbert. 1980. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4): 817–838. 59