Shedding Light on Survey Accuracy—A Comparison between SHED and Census Bureau Survey Results

Finance and Economics Discussion Series Federal Reserve Board, Washington, D.C. ISSN 1936-2854 (Print) ISSN 2767-3898 (Online) Shedding Light on Survey Accuracy—A Comparison between SHED and Census Bureau Survey Results Kabir Dasgupta, Fatimah Shaalan, Mike Zabek 2025-010 Please cite this paper as: Dasgupta, Kabir, Fatimah Shaalan, and Mike Zabek (2025). “Shedding Light on Survey Accuracy—A Comparison between SHED and Census Bureau Survey Results ,” Finance and Economics Discussion Series 2025-010. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2025.010. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.

Shedding Light on Survey Accuracy—A Comparison between SHED and Census Bureau Survey Results Kabir Dasgupta,1 Fatimah Shaalan,2 and Mike Zabek3 February 2025 Abstract: The annual Survey of Household Economics and Decisionmaking (SHED) receives substantial research attention for topics related to household finances and economic well-being. To assess the reliability of data from the SHED, we compare aggregate statistics from the SHED with prominent, nationally representative surveys that use different survey designs, sample methodologies, and interview modes. Specifically, we compare recent statistics from the SHED with similar questions in U.S. Census Bureau surveys, including the Current Population Survey (CPS) and the American Community Survey (ACS). Overall, aggregate responses to the SHED benchmark well against nationally representative surveys, particularly for questions with nearly identical wording. However, we also note that subtle differences in wording of survey questions for broadly similar indicators can prompt moderate variations across data sources. Keywords: SHED; survey methodology; Census Bureau: CPS; ACS; demographic; employment; homeownership; health insurance; food insufficiency JEL Classification: C80, C81, C83 Note: The results and opinions expressed in this paper reflect the views of the authors and should not be attributed to the Federal Reserve Board or the Federal Reserve System. 1 Dasgupta (email: kabir.dasgupta@frb.gov): Senior Economist, Division of Consumer and Community Affairs (DCCA), Federal Reserve Board. 2 Shaalan (email:fatimah.j.shaalan@frb.gov): Research Assistant, DCCA, Federal Reserve Board. 3 Zabek (email: mike.zabek@frb.gov): Senior Economist, DCCA, Federal Reserve Board.

1. Introduction The Federal Reserve Board’s annual Survey of Household Economics and Decisionmaking (SHED) has become a widely used data source in academic and policy research that focuses on the financial wellbeing of U.S. households and the economic challenges experienced by the low-income population. However, relative to some of the well-established nationally representative surveys such as the Census Bureau’s Current Population Survey (CPS) and American Community Survey (ACS), the data recorded in the annual SHED are based on online interviews administered to a much smaller sample of individuals. As such, while the SHED sample is drawn using address-based sampling and designed to be representative of the adult population living in the U.S., it is important to assess how confident researchers should be regarding the quality of the SHED data and the empirical findings derived from the survey. While online interviews (like in the SHED) can be conducted at a much lower cost compared with largescale surveys like the CPS and ACS—which employ various methods, including mail, internet, telephone, and in-person interviews—participants’ responses may vary based on the mode of interviews used (Keeter et al., 2015). In addition, there are other survey characteristics that distinguish the SHED from the widely used Census Bureau surveys. For instance, while SHED participants are drawn from a preexisting nationally representative probability-based online panel—KnowledgePanel, managed by the consumer research firm Ipsos—the considerably larger samples for the CPS and ACS are selected through more laborious processes.4 We broadly highlight some of the key differences with respect to the overall scope, sample design, and survey completion rate for the SHED, the CPS, and the ACS in table A.1. Given the differences across the surveys, our analysis explores whether the survey methodology used in the SHED generates comparable estimates for similar or identical questions that are also asked in the Census Bureau surveys. As such, providing a comparison of overlapping indicators from the SHED and other wellknown surveys would help establish credibility for the quality of results generated from indicators that are unique to the SHED. It is also important to note that the annual SHED has undergone several important changes since the first time it was fielded in 2013. For instance, the sample size in the 2017 survey (about 12,000 people) almost doubled the number surveyed in the previous years and remained relatively stable thereafter. In addition, the SHED questionnaire has gone through various revisions over time, as changes in broader economic conditions, including the COVID-19 pandemic, prompted inclusion of new survey questions and removal of less relevant questions. The response rates for both the KnowledgePanel and the SHED have also fluctuated over the survey years.5 In addition, while the pandemic had a substantial effect on survey 4 Ipsos recruits members of the KnowledgePanel (from which the SHED sample is drawn) by using address-based sampling methods to ensure sufficient coverage of households nationwide; more information is available on the Board’s website at https://www.federalreserve.gov/publications/2024-economic-well-being-of-us-households-in- 2023-description-survey.htm. 5 The specific information on the SHED’s completion rates over the years can be found in the yearly “Description of the Survey” section of each annual SHED report on the economic well-being of U.S. households, which can be found on the Board’s website at https://www.federalreserve.gov/publications/report-economic-well-being-ushouseholds.htm. Also see background materials of a report by California Health Care Foundation (2019) for details on the KnowledgePanel methodology.

response rates in general, the response rates in some of the Census Bureau surveys had been declining over time even before the pandemic started.6 An earlier study by Devlin-Foltz, Larrimore, and Schmeiser (2015) used surveys conducted in 2014 to investigate whether aggregate responses provided in the SHED’s online interviews benchmark well against responses to surveys like the CPS and ACS. While the overall results revealed that the SHED yielded similar estimates for overlapping questions included in the Census Bureau’s surveys, the more recent developments introduce an important scope for our analysis to examine whether the annual SHED continues to provide an accurate representation of U.S. households’ economic and financial well-being. We consider a wide range of demographic characteristics as well as socioeconomic indicators, including employment and homeownership status, health insurance coverage, food sufficiency, and banking. As will be noted in our subsequent discussions, while most of our results are based on surveys conducted in 2022, in a few cases we consider data from other years, conditional on the data availability of the comparable surveys considered in this analysis. To maintain consistency with the SHED sample, all our results are based on adult populations aged 18 and over. In the following sections, we highlight our key findings. 2. Demographic characteristics We begin by looking at some of the basic demographic attributes, including age, race, ethnicity, household income, educational attainment, and marital status. We compare the 2022 SHED with the 2022 Annual Social and Economic Supplement of the CPS (CPS-ASEC) and the 2022 ACS. As shown in table 1, the SHED, CPS-ASEC, and ACS have nearly identical distributions of age, race and ethnicity, and household income. For each grouping of age, race and ethnicity, and income, the share is within 1 percentage point across the various surveys.7 However, this similarity is by design, since the SHED’s sample is stratified and then weights are computed to ensure that the distribution of these characteristics matches estimates from the CPS-ASEC.8 Additionally, the SHED has roughly comparable distributions of educational attainment and marital status.9 The weighted proportions reported in table 1 show generally similar values for both education and marital status. The largest departures are that the share of respondents with less than a high school degree is 3 percentage points lower than in the two other surveys and that the marriage rate is 3 percentage points higher in the SHED.10 Additionally, the share of individuals that completed some college but less than a 6 For example, see U.S. Census Bureau (2024) for information on CPS modernization efforts. Also see U.S. Census Bureau’s information on ACS annual response rates at https://www.census.gov/acs/www/methodology/sample-sizeand-data-quality/response-rates/. For pandemic-related effects on the Census Bureau surveys, see Rothbaum et al. (2021) and Ward and Edwards (2021). 7 Given the SHED’s sample size of 11,667 people in 2022, a 95 percent confidence interval for population-level shares will be slightly below 1 percentage point. Confidence intervals will be wider for subgroup analyses due the smaller size of the groups. 8 The SHED weights were computed using a raking procedure based on the distribution of age, race, ethnicity, census region, household income, education crossed with race and ethnicity, education crossed with sex, and education by itself. While there are some differences, the specific categories used for each variable generally match those here. 9 Unlike other variables in table 1, the marriage rate is not used to compute the SHED’s weights. 10 For comparability with the other surveys, table 1 shows educational attainment based on original responses to question ED0 without a recoding of people who subsequently say that they ever enrolled in a college program. This recoding increases the share reporting some college and decreases the share reporting only high school or a GED (General Educational Development certificate) in the released data. Note also that a different “panel variable” that is included in the SHED’s data files is the basis of the SHED’s weights, not question ED0. This panel variable is the

bachelor’s degree is 4 percentage points higher in the SHED sample compared with the CPS-ASEC sample, but only 1 percentage point higher than the ACS sample. These gaps are modest overall. Table 1: Demographics and weighting variables Survey Characteristic SHED CPS-ASEC ACS Age group 18 to 29 20 20 20 30 to 44 26 26 26 45 to 59 24 24 24 60+ 30 30 31 Race and ethnicity Hispanic 17 17 17 White 62 62 61 Black 12 12 11 Asian 6 7 6 Household income Less than $25,000 12 12 12 $25,000 to $49,999 16 16 16 $50,000 to $99,999 29 29 30 $100,000 to $149,999 18 18 19 $150,000 or more 25 25 24 Educational attainment Less than high school 7 10 10 High school or GED 27 29 27 Some college 30 26 29 B.A. or more 36 35 34 Marital status Married 55 52 52 Note: The table gives the share of the population (in percent) with the specified demographic characteristic, by row, according to the 2022 SHED, the March 2022 CPS-ASEC, and the 2022 ACS, by column. For comparability, educational attainment in the SHED is based on original responses to question ED0 without a recoding of people who subsequently say that they are enrolled in a college program, which differs from the publicly released data. Shares are weighted using person-level weights for each survey. response to a separate question asking about educational attainment in previous surveys of all panel members that Ipsos conducted.

3. Employment The employment rate and the share of adults who are self-employed are largely comparable between the SHED and the Census Bureau surveys. In particular, as shown in table 2, the employment rates among surveyed adults are similar for the SHED and the CPS-ASEC (61 percent), but slightly below the ACS level (by 2 percentage points).11 There are small differences in question wording, as the SHED’s questions are about employment in the past month, while the CPS-ASEC and ACS each ask about employment in the previous week. To unpack some of the differences, we use a version of the CPS-ASEC’s employment question that the survey vendor, Ipsos, asks of all members of the KnowledgePanel, from which the annual sample for the SHED is drawn. When we look at the answers that SHED respondents previously gave to this question, we find a slightly higher employment rate of 63 percent for the SHED sample, similar to the ACS level. As such, there is some evidence that the longer reference period in the SHED question could be leading to a slightly lower share of respondents saying that they were employed.12 Additionally, the share of SHED adults who are self-employed (7 percent) is similar to the share in the CPS-ASEC sample and just 1 percentage point higher than the ACS share. The share of adults in the SHED who are employed as part-time workers (those who work less than 35 hours per week) exceeds the shares from the other two Census Bureau surveys, especially the CPS-ASEC. Part-time workers account for 14 percent of adults in the SHED, which exceeds the corresponding ACS and CPS-ASEC shares by 2 percentage points and 4 percentage points, respectively. While there are substantial differences in the way part-time employment is defined between the SHED and the other two Census Bureau surveys, they are unlikely to be leading to the larger share of part-time workers that we observe in the SHED. Specifically, table A.3 separates out the different concepts available in each survey, including several different approaches to defining part-time work based on various concepts asked in the CPS-ASEC. The main reason why we suspect the values are not due to conceptual differences is that both the 2022 SHED and the value reported for the CPS-ASEC are defined quite similarly, based on usual hours in the person’s main job. Unlike the SHED, the ACS asks about part-time status in the past week (rather than usual hours) at all jobs (rather than main job). Table A.3 also shows how part-time shares vary depending on the reference period and whether the question is related to someone’s main job or all of the jobs a person holds. Shifting to asking about hours worked in the past week tends to increase the share of people who report part-time status, presumably because they worked unusually low hours in the week prior to the survey. Asking about all jobs tends to decrease the share of people who are working part time, since some people have multiple part-time jobs that they work in for 35 or more combined hours. The higher share working part-time in the ACS than in the CPS-ASEC is largely due to these conceptual differences. Hence, it appears that while the SHED has similar total employment to the CPS-ASEC, there are differences at work intensity between the survey samples beyond those which can be explained purely from conceptual differences. 11 This difference is unlikely to reflect the different timings of the surveys, since the employment-to-population ratio (reported on the Federal Reserve Bank of St. Louis’s website at https://fred.stlouisfed.org/graph/?g=1gL0k) was relatively stable through 2022, though the higher rate in the ACS compared with that in the CPS-ASEC could be partially due to ACS respondents being interviewed a few months later on average. 12 The employment question asked by Ipsos and reported here is one of the panel variables that are included in the SHED’s data files. Since they were not asked concurrently, there could be additional differences due to the different timing of the interviews.

Table 2: Employment characteristics Characteristic SHED CPS-ASEC ACS Ipsos Employed 61 61 63 63 Part-time 14 10 12 14 Self-employed 7 7 6 Note: The table gives the share of the population (in percent) with the specified characteristic, by row, according to the 2022 SHED, the March 2022 CPS-ASEC, the 2022 ACS, and questions asked of the SHED sample by the survey vendor, Ipsos, by column. Shares are weighted using person-level weights for each survey. 4. Homeownership and renting Similar to the Census Bureau surveys, the SHED asks about individuals’ homeownership status and other living arrangements. However, the homeownership question in the SHED asks about a fundamentally different concept from that covered in the Census Bureau surveys. The SHED asks whether the spouse or their partner owns their home, while the Census Bureau surveys ask about whether anyone in the household owns their home.13 As such, a dependent adult who lives at their parents’ home without paying rent would be classified as neither owning nor renting in the SHED. In contrast, for the other two surveys, a respondent would be classified as neither owning nor renting only if no one in their household either owned the house or paid rent. This distinction is particularly important for younger adults, who are more likely to live with parents (Fry, Passel, and Cohn, 2020; Board of Governors of the Federal Reserve System, 2024).14 Consistent with these differences in the reference unit for the homeownership question, panel A of table 3 shows a higher fraction of the SHED sample neither owns nor rents when compared with the CPS-ASEC and the ACS. Specifically, the SHED’s share of adults who neither own nor rent a home (10 percent) is 8 to 9 percentage points greater than the corresponding levels in the CPS-ASEC and ACS data. The share of homeowners (and renters) in the SHED is 6 percentage points (and 2 to 3 percentage points) below the corresponding shares in the CPS-ASEC and ACS samples. To control for the differences in the share of adults who neither own nor rent, we restrict our analysis to only people who either own or rent. When doing so in panel B of table 3, the homeownership rate is the same as in the CPS-ASEC and the ACS. However, a slightly larger share of people report having a mortgage in the SHED than do in the ACS and, especially, the CPS-ASEC. Finally, we compare individuals’ homeownership status and rental arrangements for a sample where the household and family definitions are expected to be identical across the various compared surveys. With that intention, in a separate analysis in table A.5, we focus only on people who live alone. While the SHED’s share of those who neither own nor rent declines marginally (by 3 percentage points) and the corresponding Census Bureau surveys’ shares increase slightly (by 1 percentage point), the differences seen in table 3 qualitatively persist. The reason for this high share who neither own nor rent, but also live alone warrants further study. 13 We present a detailed overview of the overlapping survey questions considered in our analysis in appendix table A.6. 14 In line with those findings, table A.4 shows that neither owning nor renting is more common among younger respondents in the SHED, since the share that neither own nor rent is only 3 percent among people aged 40 and above and 10 percent among people aged 18 and above.

Table 3: Living arrangements and homeownership Category SHED CPS-ASEC ACS Panel A: All adults Homeowner 63 69 69 With a mortgage 42 40 44 Without a mortgage 21 29 26 Renter 27 30 29 Neither own nor rent 10 1 2 Panel B: Those who either own or rent Homeowner 70 70 70 With a mortgage 46 40 44 Without a mortgage 24 29 26 Renter 30 30 30 Note: The table gives the share of people (in percent) in the specified category among all adults and conditional on either renting or owning (excluding people who neither rent nor own), by row, according to the 2022 SHED, the March 2022 CPS- ASEC, and the 2022 ACS, by column. Shares are weighted using person-level weights for each survey. 5. Health insurance coverage Overall health insurance coverage is strikingly similar across the SHED, the CPS-ASEC, and the ACS. Table 4 shows that 91 percent of adults are covered under a health insurance plan, according to each survey. Additionally, all three surveys show that 35 percent of adults were covered by either Medicare or Medicaid. As shown in table 4, private insurance coverage rates are also quite similar across the surveys, with the SHED’s share of 67 percent lying between the CPS-ASEC’s 66 percent and the ACS sample’s 69 percent. Additionally, the SHED’s result that 56 percent of people are covered by an employer plan matches the corresponding ACS share exactly. This 56 percent in the SHED and the ACS is higher than the CPS-ASEC’s 53 percent, which could be due to a small difference in question wording—the SHED and the ACS both ask about health insurance through an employer or union, while the CPS-ASEC does not explicitly mention union coverage.

Table 4: Health insurance Item SHED CPS-ASEC ACS Any insurance 91 91 91 Private insurance 67 66 69 From an employer 56 53 5 6 From a marketplace 4 4 Medicare or Medicaid 35 35 35 TRICARE or VA benefits 6 4 5 Note: The table gives the share of the population (in percent) with any health insurance and health insurance from the specified source at the time of the interview, by row, according to the 2022 SHED, the March 2022 CPS-ASEC, and the 2022 ACS, by column. TRICARE refers to a program that provides civilian health benefits for U.S Armed Forces military personnel, military retirees, and their dependents. VA benefits refer to health services provided by the US Department of Veterans Affairs. Shares are weighted using person-level weights for each survey. 6. Comparison with other Census Bureau surveys In the 2023 SHED, a new question was added to document households’ food sufficiency. In particular, respondents were asked whether their own households had enough food to consume in the month before the survey. This question is based on the food sufficiency question from the Census Bureau’s Household Pulse Survey (HPS).15 The HPS was initially fielded as a weekly survey after the onset of the pandemic in 2020 and continues to provide high-frequency data on household finances.16 To be consistent with the survey timing of the 2023 SHED, which was fielded from October 20 to November 5, 2023, we chose the HPS that was conducted from October 18 to October 30, 2023. Table 5 shows that the share of adults in the 2023 SHED who reported their households to have sufficient food in the recent past (93 percent) exceeded the HPS share (88 percent) by 5 percentage points. This gap could be due to the difference in the time reference (recall period) specified in the food sufficiency questions asked in the surveys. More specifically, in contrast to the SHED’s “prior month” reference, the more frequently fielded HPS asks respondents about their households’ food affordability in the seven days before the survey. Recent evidence from the U.S. Department of Agriculture based on survey-based information on food security suggests that the share of households experiencing food security can vary by the length of the recall period (also see Villacis et al., 2023).17 Unlike our findings on food sufficiency, 15 The HPS is one of the Census Bureau’s experimental data products that was designed to measure emergent social and economic issues facing U.S. households. The HPS is a 20-minute online interview that started collecting information from April 2020 (after the onset of the COVID-19 pandemic). More information is available on the Census Bureau’s website at https://www.census.gov/data/experimental-data-products/household-pulse-survey.html. 16 HPS technical documentation—namely, source and accuracy statements—can be found on the Census Bureau’s website at https://www.census.gov/programs-surveys/household-pulse-survey/technical-documentation/sourceaccuracy.html. 17 The concepts of food sufficiency and food security are related but distinct from each other. Additional details on the frequency of food insecurity are available on the U.S. Department of Agriculture’s website athttps://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-u-s/frequency-of-foodinsecurity/#:~:text=The%20food%20security%20survey%2C%20which,during%20the%20previous%2030%20days .

however, the prevalence of food security tends to be lower for longer recall periods. As such, while the share of individuals who experience food sufficiency is likely to vary by the reference periods specified in different surveys, the direction of the inequality appears to differ depending on the specific measure considered to evaluate households’ access to food and nutrition, thereby motivating a relevant scope for future research to explore. Consistent with the overall shares, for each racial and ethnic group, the fraction of respondents experiencing food sufficiency continues to be higher in the SHED sample compared with the HPS sample. However, the gaps between the two surveys vary by race and ethnicity. While the gap in the shares is the narrowest for the Asian population (2 percentage points), the difference in the food sufficiency shares between the two surveys is the largest for the Black population (11 percentage points). Additionally, as shown in figure 1, when broken down by age group, individuals aged 45 to 59 appear to have the largest gap in food sufficiency across the two surveys, while the gap varies between 3 and 5 percentage points. Table 5: Food sufficiency Demographic SHED Pulse Enough food in household 93 88 By race and ethnicity White 95 89 Black 90 79 Hispanic 87 82 Asian 96 94 Note: The table gives shares of households (in percent) that experience food sufficiency for the overall sample as well as for each racial and ethnic group, by row, using the SHED 2023 data and the Household Pulse Survey (conducted from October 18 to October 30, 2023), by column. Both surveys were recoded to combine the response categories “Enough of the kinds of food we wanted to eat” and “Enough, but not always the kinds of food we wanted to eat” as “food sufficient.” The response categories “Sometimes not enough to eat” and “Often not enough to eat” were combined as “food insufficient.” The shares (percent) are weighted using person-level weights for each survey.

Figure 1: Food sufficiency, by age group 100 97 95 93 92 90 89 90 86 85 85 85 80 75 18 to 29 30 to 44 45 to 59 60+ SHED HPS Note: The figure compares shares of respondents (in percent) who reported having food sufficiency in the 2023 SHED and the October 18–30, 2023, Household Pulse Survey (denoted as “HPS” in the graph), by age group. For more details, see table 5 notes. Finally, we use an overlapping banking-related question that the SHED has in common with the Federal Deposit Insurance Corporation (FDIC) National Survey of Unbanked and Underbanked Households.18 For consistency, we use surveys conducted in the year 2021.19 We compare responses to a question that asks participants whether they have a bank account (e.g., a checking or savings account). We report the survey-specific shares of households with a bank account in table 6. It is important to note that while the FDIC’s survey question on possession of a bank account is worded to document information about either the respondent or anyone else living in the respondent’s household, the SHED’s question is meant to document similar information about only the survey participant or their spouse or partner.20 The SHED’s weights are representative of U.S. adults who are over 18, and the FDIC survey’s weights are representative of U.S. households. This slight difference in weighting and the questionnaires plausibly explains why the SHED sample’s share of those with a checking or savings account is 2 percentage points lower than the share observed in the FDIC data, with the highest difference being 3 percentage points for 18-to-29-year-old participants. Additionally, as indicated in figure 2, the corresponding shares of those with a bank account are lower in the SHED data compared with the FDIC sample (by 2 percentage points) for the Black and Hispanic or Latino populations; the SHED statistic marginally exceeds the FDIC proportion (by 1 percentage point) for the white and Asian populations. 18 The FDIC National Survey of Unbanked and Underbanked Households has been conducted biennially since 2009. The survey is administered in collaboration with the Census Bureau and collects banking as well as nonbanking information, including bank account ownership; use of prepaid cards and nonbank online payment services; use of nonbank money orders, check cashing, and money transfer services; and use of bank and nonbank credit. 19 The 2021 survey was the most recent FDIC survey at the time of our analysis. 20 Unlike the FDIC questionnaire, which asks respondents about only a checking or savings account, the SHED’s question additionally asks about a money market account.

Table 6: Checking or savings account Characteristic SHED FDIC Overall 94 96 By age group 18 to 29 92 95 30 to 44 92 95 45 to 59 94 95 60+ 98 97 Note: The table gives the share of the overall adult population as well as shares by different age groups, by row, according to the 2021 SHED and the 2021 FDIC National Survey of Unbanked and Underbanked Households, by column. The SHED sample shares (in percent) are weighted using person-level weights and are meant to be representative of the U.S. adult population aged 18 and above. The FDIC survey shares (in percent) are weighted using household-level weights and are meant to be representative of all U.S. households. Figure 2: Checking or savings account, by race and ethnicity 100 98 99 98 97 95 91 89 89 90 87 85 80 75 Asian Black Hispanic White SHED FDIC Note: The figure compares shares of surveyed samples (in percent) that reported having a bank account, by each racial and ethnic group, according to the 2021 FDIC National Survey of Unbanked and Underbanked Households and the 2021 SHED. For more details, see table 6 notes.

7. Concluding remarks Despite some of the differences discussed in this study, the aggregate responses to the SHED continue to benchmark well against Census Bureau data sources. For questions that are identical in terms of content, reference window, and units of observation, we generally see quite similar results. One exception is part-time status, though the differences are only 2 to 4 percentage points. Our analysis indicates that for some broadly similar indicators, subtle differences in wording of survey questions may contribute to differences in findings. In particular, differences in the question texts across surveys in terms of the way a survey indicator is defined, time references used, presence of specific references to respondents’ household members, as well as response options provided to survey participants can be relevant in survey responses.21 For instance, the share of the SHED sample that neither own nor rent their home seems to be visibly higher than the shares observed in the CPS-ASEC and ACS samples. It is however important to note that while the SHED question on homeownership status captures information of survey respondents and their partners, the CPS-ASEC and ACS ask about homeownership status of anyone living in the survey participants’ household. Another example is that we observe a higher share of adults living in households that are food sufficient over the prior month in the SHED as compared with the prior week in the HPS. While differences in wording of survey questions can prompt important variations in survey responses, it is possible that there are other underlying sources that can also explain a part of the differences observed in some of the broadly similar indicators across surveys. As such, our analysis motivates the relevance of further investigations to identify other plausible sources of the differences observed between the SHED and the Census Bureau surveys. Another informative extension is a comparison of how the various survey questions behave in the time series. Since many of the SHED’s results provide timely information about current economic conditions, it would be particularly useful to know the extent to which changes in results from the SHED can give similar information to that furnished by changes in statistics from other well-established surveys. This extension is of particular interest because the SHED provides a unique allotment of indicators that are frequently released more quickly than those that many other large surveys supply. 21 See table A.6 for a description of these differences.

REFERENCES Board of Governors of the Federal Reserve System (2024). Economic Well-Being of U.S. Households in 2023. Washington: Board of Governors, available at https://doi.org/10.17016/8960. California Health Care Foundation (2019). “Help Wanted: Californians’ Views and Experiences of Serious Illness and End-of-Life Care.” Oakland, Calif.: CHCF, https://www.chcf.org/wpcontent/uploads/2019/10/SurveyAdultsSeriousIllnessEndLifeMethodology.pdf. Devlin-Foltz, Sebastian, Jeff Larrimore, and Maximilian D. Schmeiser (2015). Should You Trust Things You Hear Online? Comparing SHED and Census Bureau Survey Results, technical report. Washington: Board of Governors of the Federal Reserve System. Fry, Richard, Jeffrey S. Passel, and D’Vera Cohn (2020). "A Majority of Young Adults in the U.S. Live with Their Parents for the First Time since the Great Depression." Washington: Pew Research Center, available at https://www.pewresearch.org/short-reads/2020/09/04/a-majority-of-youngadults-in-the-u-s-live-with-their-parents-for-the-first-time-since-the-great-depression. Keeter, Scott (2015). "From Telephone to the Web: The Challenge of Mode of Interview Effects in Public Opinion Polls." Washington: Pew Research Center, https://www.pewresearch.org/methods/2015/05/13/from-telephone-to-the-web-the-challenge-ofmode-of-interview-effects-in-public-opinion-polls. Rothbaum, Jonathan, Jonathan Eggleston, Adam Bee, Mark Klee, and Brian Mendez-Smith (2021). "Addressing Nonresponse Bias in the American Community Survey during the Pandemic Using Administrative Data," American Community Survey Research and Evaluation Report Memorandum Series ACS21-RER-05. Washington: U.S. Census Bureau. U.S. Census Bureau (2024). "Current Population Survey (CPS) 2023 Modernization Efforts," webpage, https://www.census.gov/programs-surveys/cps/about/modernization.html. Villacis, Alexis H., Syed Badruddoza, Ashok K. Mishra, and Joaquin Mayorga (2023). "The Role of Recall Periods When Predicting Food Insecurity: A Machine Learning Application in Nigeria," Global Food Security 36: 100671. Ward, Jason M., and Kathryn Anne Edwards (2021). "CPS Nonresponse during the COVID-19 Pandemic: Explanations, Extent, and Effects," Labour Economics 72: 102060.

APPENDIX Table A.1: Survey characteristics of the SHED, the CPS-ASEC, and the ACS Characteristic SHED CPS-ASEC ACS Sample size and design Final sample: 11,667 adults Final sample: 89,197 households (152,732 people) Final interviews: About 2 million housing units The SHED sample is selected from a nationally ASEC includes a probability sample. The monthly CPS (plus about 150,000 people in group quarters) representative probability-based online sample, samples are multistage stratified samples and use a (4- The ACS employs a two-phase, two-stage sample known as the KnowledgePanel, administered by 8-4) rotating panel design. Households are interviewed design. At different times, independent housing Ipsos. Ipsos selects the KnowledgePanel respondents for four consecutive months, rotate out of the sample unit address samples are selected for all counties, based on address-based sampling. The SHED sample for the next eight months, and are then interviewed including the District of Columbia. The ACS is is drawn from the KnowledgePanel by ensuring that again for the next four months. The ASEC sample not a panel survey. the sample behaves as an “equal probability of includes a regular monthly CPS sample, an oversample selection method” sample. A subset of respondents in of Hispanics, and a Children’s Health Insurance each survey are also surveyed in the previous survey. Program (CHIP) sample to improve state estimates of Around one-third of the participants in the 2022 children’s health insurance coverage. SHED were also surveyed in the 2021 survey. Weighting The entire KnowledgePanel is first weighted to the The final weight, which is constructed to improve the The household- or person-level weights represent benchmarks in the latest CPS-ASEC along various representativeness of survey indicators at the a fixed number of housing units or people and are geodemographic dimensions. After survey collection, population level and adjust for non-interview biases, used to make the sample representative of the Ipsos further adjusts weights in a post-stratification goes through various adjustments. Starting from the U.S. population and correct for under- or overprocess. These weights are estimated using the basic weight for each person, which represents the coverages and non-interview biases. The weights geographic and demographic distribution of the CPS- probability of selection for the survey, the final weight are constructed using a raking-ratio estimation ASEC as benchmarks. This is to correct for any incorporates additional adjustments for special procedure, producing a weight for each person as survey nonresponse or undercoverage, or under- or sampling situations and failure to obtain interviews well as a weight for each housing unit. The oversampling errors in the study design. from eligible households, followed by a two-stage ratio housing unit and household person weighting estimation procedure. For the ASEC data, a supplement uses two types of geographic areas for weight is constructed for the expanded sample and to adjustments: weighting areas and subcounty make sure married and cohabiting couples receive the areas. same weight (for the CHIP sample). Data collection method Administered online entirely Computer-assisted telephone interviewing or computer- Combination of mail-out/mail back, internet, and assisted phone interviewing computer-assisted personal interviews Survey participation/ Voluntary/64 percent Voluntary/61 percent Mandatory/84 percent Response rate Starting from recruitment of the Ipsos KnowledgePanel to completion of the SHED, the cumulative response rate was 3.6 percent in 2022. Note: The sample sizes and the participation/response rates are reported for the survey year 2022 for the SHED, the CPS-ASEC, and the ACS.

Table A.2: Education, by age Characteristic SHED CPS-A SEC AC S Age 18 or older Less than high school 7 10 10 High school or GED 27 29 27 Some college 30 26 29 B.A. or more 36 3 5 3 4 Age 25 or older Less than high school 7 9 10 High school or GED 27 28 26 Some college 28 25 28 B.A. or more 38 38 36 Note: The table gives the share of the population with the specified level of education for the specified age group, by row, according to the 2022 SHED, the March 2022 CPS-ASEC, and the 2022 ACS, by column. For comparability, educational attainment in the SHED is based on original responses to question ED0 without a recoding of people who subsequently say that they are enrolled in a college program, which differs from the publicly released data. Shares (in percent) are weighted using person-level weights for each survey. Table A.3: Different measures of part-time work SHED CPS-ASEC CPS monthly ACS Item (March 2022) Main job Usually 14 10 Last week 11 All jobs Usually 9 Last week 13 Usually last year 11 12 Note: This table gives the share of all adults (in percent) who worked part time, by row, according to the specified definition for the SHED, the CPS-ASEC, the March monthly CPS, and the ACS as of 2022, by column. Statistics are weighted using person-level weights appropriate for each survey.

Table A.4: Share of adults who neither own nor rent, by age Neither own nor rent SHED CPS-ASEC ACS Age 18+ 10 1 2 40+ 3 1 2 30 to 59 5 1 1 Note: The table gives the share of the population (in percent) who neither owns nor rents in the specified age range, by row, according to the 2022 SHED, the 2022 CPS-ASEC, and the 2022 ACS, by column. Shares are weighted using person-level weights for each survey. Table A.5: Homeownership and mortgage status for people living alone Category SHED CPS-ASEC ACS Ipsos Homeowner 47 53 52 56 With a mortgage 26 22 24 Without a mortgage 21 31 27 Renter 46 45 46 41 Neither own nor rent 7 2 3 3 Note: The table gives the share of people (in percent) who live alone in the specified category of housing tenure, by row. Statistics are presented for the 2022 SHED, the March 2022 CPS-ASEC, the 2022 ACS, and a question asked by Ipsos of SHED respondents, by column. Each uses person-level weights.

Table A.6: Topic wise survey variables considered from the SHED and Census Bureau surveys Topic SHED CPS-ASEC ACS Employment “Last month, did you do any work for either pay or Individuals who reported to be “at work” (i.e., working “Last week, did this person work for pay at a profit?” for either pay or profit or working at least 15 hours job (or business)?” Response options: Yes/No without pay in a family business or farm) or to “have a Response options: Yes/No job, but not at work” during the week preceding the survey are considered employed. Part-time “Still thinking about your main job, do you usually Based on usual number of hours per week the “During the past 12 months, in the weeks worked, employment work:” respondent reports being at their main job how many hours did this person usually work each Response: Responses range from 0 to 99 hours (top-coded at 99). week?” -Full-time (35 or more hours per week) People are coded as part time in the table if they give Responses range from 0 to 99 hours (top-coded at -Part-time (less than 35 hours per week) less than 35 hours. 99). People are coded as part time in the table if they give less than 35 hours. Homeownership “Do you [and/or spouse or partner]:” Created from a combination of responses to questions in “Is this house, apartment, or mobile home:” Response options: the main CPS survey and the ASEC. In the monthly CPS Response options: -Own your home with a mortgage or loan for the housing unit, people are asked if the dwelling is: -Owned by you or someone in this household with a -Own your home free and clear (without a mortgage -Owned or being bought mortgage or loan? Include home equity loans or loan) -Rented for cash -Owned by you or someone in this household free -Pay rent -Occupied without payment of cash rent and clear (without a mortgage or loan) - Neither own nor pay rent -Rented As a part of the ASEC Supplemental Poverty Measure, -Occupied without payment of rent respondents are asked: “Not counting home equity loans, do you or any other member of this household have a mortgage, deed of trust, contract to purchase, or similar debt on THIS property?” Response options: Yes/No

Table A.6 (continued) Topic SHED CPS-ASEC ACS Health insurance “Are you currently covered by any of the following The CPS-ASEC includes multiple questions to identify Health insurance question series: types of health insurance or health coverage plans?” if some, all, or no members in a household are “Is this person currently covered by any of the Response options: Yes/No (to the following categories) currently covered by any health insurance, following types of health insurance or health -Insured through employer/union employment-based health insurance, direct-purchase coverage plans?” -Directly from insurance company health insurance, health insurance from Marketplace, Response options: Yes/No (to the following -Medicare or Medicaid Medicaid, Medicare, TRICARE, VA Care, CHAMPVA, categories) -Veterans Affairs Health Care (VA) or TRICARE or the Indian Health Service. -Insurance through a current or former employer or -Purchased through insurance exchange union (of this person or another family member) -Any other health insurance -Insurance purchased directly from an insurance company (by this person or another family member) -Medicare -Medicaid -VA or TRICARE -Indian Health Services -Any other type of health insurance coverage SHED HPS FDIC Food sufficiency “In the past month, which of these statements best “In the last 7 days, which of these statements best describes the food eaten in your household?” describes the food eaten in your household?” Response options: Response options: -Enough of the kinds of food we wanted to eat -Enough of the kinds of food we wanted to eat -Enough, but not always the kinds of food we wanted to -Enough, but not always the kinds of food we wanted to eat eat -Sometimes not enough to eat -Sometimes not enough to eat -Often not enough to eat -Often not enough to eat Bank account “Do you (and/or your spouse or partner) currently have “Do you (or anyone else in your household) have a a checking, savings or money market account?” checking or savings account now?” Response: Yes/No Response: Yes/No Note: The table provides details on the question wording used in the different surveys for the indicators used in our analysis. We exclude the standard demographic characteristics from table 1.