Since 1950, the U.S. Census Bureau has incorporated a testing, evaluation, and experimental program into the overall enumeration effort. The 2020 Census Evaluations and Experiments (EAE) operation is one component of our 2020 Census Data Quality efforts, designed to document and evaluate 2020 Census programs and operations, as well as to test new methods suggested from previous research. Results documented in the 2020 Census EAE operation serve as the background or basis for designing, testing, and implementing the 2030 Census.
The 2020 Census Evaluations and Experiments (EAE) consists of more than 65 studies. The studies consist primarily of operational assessments, but also include evaluations, experiments, quality control results, and a topic report. The list of planned studies by type, descriptions of each type, and tentatively scheduled completion dates are viewable below. The results from some of these studies will not be released publicly because of the proprietary or restricted nature of the information. The first reports from these studies were released on August 18, 2022.
These studies aim to describe the effectiveness of census components and the impact that they have on topics such as data quality and coverage. These reports present analysis and interpretation of quantitative and qualitative data pertaining to decennial census operations, processes, systems, and auxiliary data collections.
This evaluation was conducted prior to the 2020 Census to test the effectiveness of 2020 Census television and radio advertisements using quantitative procedures. It analyzed factors such as message recall, message comprehension, behavioral intention to respond to the census, and the Census Bureau’s corporate image. The results will help the Census Bureau understand how quantitative creative testing might be utilized in intercensal years where ad copy is used, as well as in preparation for the 2030 Census.
The 2020 Census was accompanied by a diverse array of paid advertisements, partnership outreach programs, news coverage, and more designed to increase knowledge about the census and motivate households to self-respond, specifically through the internet mode. This evaluation compares self-response rates from the 2020 Census and the 2019 Census Test, which was void of advertisements, to estimate the impact of these programs—collectively referred to as the decennial environment.
The purpose of the 2020 Census Tracking Survey was to track U.S. public sentiment concerning matters that may bear upon 2020 Census participation and to examine how attitudes and perceptions change during the census outreach period. This report is an overview of key findings from the 2020 Census Tracking Survey, a survey designed as part of the overall 2020 Census Integrated Communications Campaign (ICC) to provide insight into the campaign by identifying emerging issues and trends, helping to understand results from other data inputs, and informing recommendations for campaign refinements.
The purpose of the Matching 2018 Census Barriers, Attitudes, and Behaviors Study (CBAMS) Survey Sample to the 2020 Census, was to evaluate the relationship between intended response behaviors and actual response behaviors. This report focuses on using the 2018 CBAMS to examine 2020 Census response through an exploratory model to understand individual and household characteristics for survey cooperation.
The purpose of the Investigating Digital Advertising and Online Self-Response report was to evaluate the role of digital advertising in directing respondents to complete the census online. The results of the research presented in this report provide insight into how respondents came to the Internet Self-Response instrument and highlight the useful role that mailed materials sent from the Census Bureau, online search results, and paid advertising had in directing respondents to the online self-response option.
The purpose of the Group Quarters Advance Contact (GQAC) evaluation report was to assess the performance of the student housing screening question used in GQAC, and identify potential changes to the screening definition and to the Group Quarters (GQ) type definitions on which it is based.
The purpose of this evaluation was to assess the redesign of the Address Canvassing operation, including the use of in-office address canvassing and interactive review. In an effort to reduce costs and improve quality for the 2020 Census, a preliminary in-office component was added to reduce the magnitude of work conducted in the field.
The purpose of this evaluation was to assess the redesign of the Address Canvassing operation. In an effort to reduce costs and improve quality, the U.S. Census Bureau reengineered address canvassing for the 2020 Census to include a suite of In-Office and In-Field Address Canvassing operations
By June 2025
Will research if dual system estimates could be generated without conducting an independent post-enumeration survey. The census would continue to serve as the first source, but administrative records would serve as the second source, rather than the post-enumeration survey results.
By June 2025
Will conduct both an evaluation and an experimental research project about historically hard-to-count populations such as non-English speakers and complex household residents.
By June 2025
Will assess respondents’ privacy and confidentiality concerns about responding to the census generally, as well as assessing concerns of certain types of respondents with a follow-up questionnaire.
By June 2025
Will study paradata from the internet self-response instrument to learn about issues associated with language.
By Sept 2025
Will use a panel of survey respondents to measure intent to participate in the census over time in order to evaluate the 2020 Census Communications Campaign.
Evaluation of Large Technology Platforms - Selected Digital Partnerships
While the Census Bureau plans to conduct this evaluation, it does not plan to externally release the evaluation report due to the proprietary nature of the information included. (As of May 2022)
The goal of this project was to evaluate the specific question wording, instructions, layout, and other features of the 2020 Census stimuli related to the household roster. The target population was households at risk of under-reporting young children, and the research design involved interviewing multiple adults within a household. Both these factors, and experience during the pilot testing stage, indicated in-person interviewing was needed. With the onset of the pandemic, field data collection was suspended soon after the main stage project had begun. The project could not be completed within the necessary timeframe for a 2020 Census evaluation. However, these research objectives will be carried forward and incorporated into plans for 2030 Census research related to historically undercounted populations, including the coverage of young children.
These studies identify potential topics for early 2030 Census life cycle research and testing. Experiments are quantitative or qualitative studies that must occur during a decennial census to produce results that are meaningful for the planners of the next one. In general, decennial census experiments involve comparisons (usually of response rates) between a control group that reflects 2020 Census production methods or procedures and a treatment group(s) that tests modifications to them.
This experiment attempted to tap into the unique environment surrounding the decennial census through the materials we use to contact households and request self-enumeration. Specifically, we tested the effect on self-response rates of a wearable nonmonetary insert (i.e., stickers) that promoted the 2020 Census, mailing materials that incorporate messaging developed by the 2020 Census communications campaign, and the addition of an Every Door Direct Mail (EDDM) piece sent on Census Day.
This experiment combined 31 types of administrative record and third-party sources to produce 2020 population estimates with the same reference date, April 1, 2020, and within the same timeframe as the 2020 Census of Population and Housing. The sources and methodology are designed to improve coverage of historically undercounted population groups.
Publication | October 31, 2023 | Rachel Gliozzi, Sarah Konya, Julia Coombs, and Michael Shaw
This experiment sought to understand the net impacts of the Optimizing Self-Response (OSR) innovation area in the 2020 Census. In particular, the focus was to evaluate the quantitative impacts of the mailing strategy, the overall influence of the internet response option, and the impact of the communications campaign on internet and phone response. Therefore, the following experimental treatments were tested:
Publication | March 10, 2025 | Mikelyn Meyers, Renee Ellis, Andrew Raim, Patricia Goerman, Kathleen Kephart (U.S. Census Bureau); Kim Aspinwall, Patricia LeBaron, Emilia Peytcheva (RTI International)
This experiment evaluated the impact of an experimental Spanish-speaking enumerator training on case outcomes, enumerator efficiency, and data quality during the Nonresponse Followup (NRFU) operation for the 2020 Census. This experiment’s results found that the experimental training helped Spanish-speaking enumerators to administer the 2020 Census interview in Spanish more accurately and efficiently, and that implementing this training and adapting it for additional non-English languages would benefit future bilingual enumerators.
These studies document final volumes, rates, and costs for individual operations or processes using data from production files and activities and information collected from debriefings and lessons learned. They do not include evaluative analyses. Operational assessments present planned versus actual variances as they relate to budget, schedule, and workloads (production and training). Depending on the operation, the assessment may include frequency distributions and standard demographic or address tables.
The In-Field Address Canvassing operation is a dependent listing activity conducted in the field and limited to the most challenging areas of the nation to identify where people live, stay, or could live or stay. Field staff compare what they see on the ground to the existing Census Bureau address list and either verify or correct the address and location information, adding addresses to the list as necessary. Field staff classify each living quarter as a housing unit or a group quarters.
The In-Office Address Canvassing operation uses empirical geographic evidence (e.g., imagery, comparison of the Census Bureau’s address list to partner-provided lists) to assess the current address list. This operation detects and identifies change from high-quality administrative and third-party data sources to reduce and then determine the In-Field Address Canvassing workload.
The Archiving operation coordinates the storage of materials and data. This operation is responsible for providing the official permanent records data from the 2020 Census (including files containing the individual responses to the 2020 Census) to the National Archives and Records Administration. It also provides similar files to the National Processing Center to use as source materials for the Age Search Service. Finally, the Archiving operation coordinates the storage of data to cover in-house needs.
The 2020 Local Update of Census Addresses (LUCA) operation allowed tribal, state, and local entities the opportunity to review and update the Census Bureau’s residential address list for their jurisdiction. The Census Bureau relies on a complete and accurate address list to reach every living quarter and associated population for inclusion in the decennial census.
The 2020 Post-Enumeration Survey (PES) Independent Listing operation was the first field operation PES conducted. During this operation, field staff independently visited and listed all housing units and potential housing units in each PES sample basic collection unit without using any previously collected address information.
The Census Questionnaire Assistance (CQA) program provided telephone support by answering frequently asked questions and helping callers fill out the census questionnaire. In addition to the inbound call capabilities, CQA conducted the outbound Coverage Improvement (CI) telephone operation, which aimed to resolve potential coverage-related issues that occurred during the initial census data collection.
This assessment presents the item nonresponse and imputation rates for household-level and person-level items from the 2020 Census, and it also compares the 2020 item nonresponse and imputation rates to the 2010 rates. These rates are quality metrics for response data. Item nonresponse occurs when a respondent provides some information but does not respond to all census questions. Imputation is a statistical technique used to fill in missing information and considers both respondent cooperation and response validity.
The Count Review operation (CRO) was implemented to enhance the accuracy of 2020 Census housing units and group quarters address files and conduct a coordinated review of the 2020 Census counts. CRO activities for the 2020 Census consisted of two phases: 1) the Federal-State Cooperative for Population Estimates (FSCPE) Address Review, and 2) Census Count and File Review (CCFR).
The purpose of the 2020 Post-Enumeration Survey (PES) was to measure the coverage of the 2020 Census. The Initial Housing Unit Matching operation was the first PES matching operation. This operation was designed to resolve the match status of sample addresses relatively early in the decennial census life cycle.
The purpose of the 2020 Post-Enumeration Survey (PES) was to measure the coverage of the 2020 Census. The Initial Housing Unit Followup operation was the second PES field operation conducted. In this operation, addresses that met predetermined criteria from the Independent Listing operation and the preliminary census address list were followed up in the field to collect additional information to determine their unit status, match status, or both.
The purpose of the 2020 Census Federally Affiliated Count Overseas (FACO) operation was to obtain counts, by home state, of U.S. military and federal civilian employees stationed or assigned overseas and their dependents living with them at the overseas post/duty station as of April 1, 2020. Home state is a person’s state of residence in the United States, while they are stationed or assigned overseas, based on the employer’s administrative records. Overseas was defined as anywhere outside of the 50 states and the District of Columbia. Counts were obtained from administrative records and were used to allocate the federally affiliated population living overseas to their home state for the sole purpose of using these counts when apportioning seats in the U.S. House of Representatives. Forty U.S. federal departments/agencies provided certified counts, by home state, for their federally affiliated employees and their dependents living with them overseas.
This assessment report presents the results of the Non-Identification (Non-ID) Processing operation conducted during the 2020 Census. A non-ID respondent was a person who provided responses to the questionnaire by either Internet Self-Response (ISR) or an interview with a Census Questionnaire Assistance (CQA) customer service representative (CSR) without a preassigned census identification number (Census ID). Respondents typically received Census IDs from 2020 Census mailings or from 2020 Census questionnaires dropped off at houses in non-mailable areas. Non-ID respondents provided location address information that helped the Census Bureau to match their address to the decennial census living quarters address inventory.
The purpose of the 2020 Post-Enumeration Survey (PES) was to measure the coverage of the 2020 Census. The PES was designed to measure the coverage of housing units and people―excluding group quarters, people residing in group quarters, and remote areas of Alaska. The PES provided estimates of net coverage and components of coverage including correct enumerations, erroneous enumerations, whole-person census imputations, and omissions.
The purpose of the 2020 Nonresponse Followup (NRFU) operation was twofold: 1) to determine the housing unit status for addresses that did not complete a census questionnaire, and 2) to enumerate the housing units that were occupied on April 1, 2020. Because of its scale, NRFU has historically been the largest and most expensive field operation of the decennial census.
The purpose of the 2020 Island Areas Censuses operation was to plan and manage the 2020 Census in American Samoa, the Commonwealth of the Northern Mariana Islands (CNMI), Guam, and the U.S. Virgin Islands (USVI). The Census Bureau partnered with the Island Areas governments through a contract agreement between the Census Bureau and the Office of the Governor of each Island Area to conduct the operation.
The 2020 Post-Enumeration Survey (PES) Coverage Measurement Design and Estimation operation developed the survey design and sample for the 2020 Census PES. This operation produced estimates of census coverage based on the PES. It also reported estimates of net coverage error and components of census coverage for demographic groups, geographic areas, and key census operations.
The 2020 Post-Enumeration Survey (PES) Final Housing Unit Matching operation processed any updates made since the Initial Housing Unit Matching operation to census addresses within the PES sample basic collection unit (BCU) search areas (this included the sample BCU and a ring of BCUs surrounding each sample BCU). The sample BCUs were in the 50 states, District of Columbia, and Puerto Rico.
The purpose of the 2020 Census Update Leave (UL) operation was to update addresses and their locations and to digitally link and leave a 2020 Census Questionnaire Package at every housing unit to allow the household to self-respond. The UL was designed to occur in geographic areas where the majority of housing units (HUs) either did not have mail delivered to the physical location of the HU, or the mail delivery information for the HU could not be verified. UL also occurred in areas affected by major or natural disasters.
The purpose of Geographic Programs (GEOP) was to provide geographic foundation in support of 2020 Census data collection and tabulation activities within the MAF/TIGER system. The MAF/TIGER System serves as the national repository for all the spatial, geographic, and residential address data needed for census and survey data collection, data tabulation, data dissemination, geocoding services, and map production.
The purpose of the 2020 Person Interview (PI) operation was to obtain information about the residents at the sample housing unit at the time of the PI interview and on Census Day (April 1, 2020) to determine where each person should have been counted on Census Day.
The purpose of the 2020 Person Matching operation was to determine which people rostered at Independent Listing (IL) addresses selected for the PI should be included in the P sample, if the P-sample person was enumerated by census, and if the E-sample person enumerated by census was correctly enumerated.
The 2020 Census New Construction program was a component of the 2020 Census Local Update of Census Addresses (LUCA) operation. It used the expertise of tribal, state, and local governments to improve the accuracy and completeness of the address list used for the 2020 Census.
The 2020 Census Self-Response and Return Rates Assessment documented self-response and return rates that resulted from the 2020 Census mailing strategy, digital advertising campaign, and implementation of the internet self-response option.
The purpose of the Person Followup (PFU) operation involved interviewers contacting addresses for status verification. Field staff were assigned cases in which they were required to interview knowledgeable respondents regarding people identified as requiring additional information to resolve one or more of the following: match status, person duplication, Census Day residence status, enumeration status and address information.
The purpose of the Redistricting Data Program (RDP) operation was to provide states the opportunity to identify the geographic areas for which specific tabulations are needed for the purpose of redistricting, and to deliver those tabulations to the states within one year of Census Day.
The purpose of the 2020 Census Count Question Resolution operation (2020 CQR) was to provide the opportunity for governments to request that the Census Bureau review their boundaries and/or housing counts by block to correct any in-scope error(s).
The 2020 Census Update Enumerate (UE) operation assessment documents the results of UE as well as the suboperation, Remote Alaska (RA).The UE operation occurred in areas that required updating the address frame while enumerating. The majority of the operation occurred in remote geographic areas that have unique challenges associated with accessibility. This release is the executive summary of the assessment report.
Publication | April 10, 2025 | Rosa Diaz Rivera, Marisa Hotchkiss, Kuopei White, Lily Kapaku, and Belkines Arenas Germosen
The purpose of the Content and Forms Design (CFD) operation was to successfully develop, refine, and finalize the content, questionnaires, and respondent materials for the 2020 Census.
By June 2025
By June 2025
By June 2025
By June 2025
By June 2025
By June 2025
By June 2025
By June 2025
By Sept 2025
By Sept 2025
By Sept 2025
By Sept 2025
By Sept 2025
By Sept 2025
By Sept 2025
By Sept 2025
While the Census Bureau conducted this operational assessment, it will not externally release the accompanying report due to the proprietary nature of the information appearing in it. (March 2025)
While the Census Bureau conducted this operational assessment, it will not externally release the accompanying report because of the small universe for this population and concerns with potentially revealing military vessel locations when combined with other information. (March 2025)
These studies provide results from the quality control activities associated with a specific operation. These reports present a more in-depth discussion on the topic of quality control than what appears in the corresponding operational assessment report.
The 2020 Census In-Field Address Canvassing (IFAC) Quality Results Report provides the results of the 2020 Census IFAC Quality Control (QC) program, as well as recommendations for future census listing operations. The In-Field Address Canvassing operation is a dependent listing activity conducted in the field and limited to the most challenging areas of the nation to identify where people live, stay, or could live or stay.
The 2020 Update Leave Quality Results Report provides the results of the 2020 Update Leave (UL) Quality Control program, as well as recommendations for future census listing operations. The 2020 Update Leave operation updated the address and coordinate location for an assigned area. A 2020 Census questionnaire package was then left at every housing unit identified in that area.
The 2020 Nonresponse Followup (NRFU) Quality Assurance Results report provides the results of the 2020 NRFU Quality Control operation, as well as recommendations for future census enumeration operations. This operation worked to detect and deter falsification by identifying poor quality work and taking steps to correct it.
This report highlights the most significant changes stemming directly from the pandemic, rather than from natural disasters or litigation that also impacted how the operations unfolded. Please note, this document is not intended to make statements on the exact level of quality or accuracy of 2020 Census data.