In January 2013, the Census Bureau began offering respondents the option to complete the American Community Survey (ACS) over the Internet. The ACS is a national survey that collects demographic, housing, social and economic characteristics from all household members. The Census Bureau tested the feasibility of using the Internet mode during two tests that took place in 2011 (Matthews et al. (2012) and Tancreto et al. (2012)). Census staff evaluated the data from these tests, and the high response rates and potential cost savings encouraged Census to implement the Internet mode in 2013.

The addition of the Internet mode required the American Community Survey Office (ACSO) to modify many operations, resulting in potential risks to the survey. Before the Internet mode went into production, a Strategic Project Integration Team, along with a series of sub-teams, was established and the new and modified operations were subject to a series of tests. The purpose of the testing was to insure that all operations were working as expected. ACSO staff created test datasets mocking production data to process through the operations. While the tests were thorough and designed to catch processing errors, there are always limitations associated with a test environment. For example, the test datasets may not be all-inclusive and some types of errors could be missed. Alternatively, the operation could be performing as specified, but producing unintended outcomes.

The purpose of this evaluation was to perform an additional check to see how the new and modified operations performed in a production environment. By reviewing the response distributions (including missing data rates) for every item on the survey, we were able to identify and research issues occurring in several operations. This study was designed to be a last check and serve as an early warning system for detecting and repairing processing problems early in the production cycle. It was not a study on mode effects or a study comparing Internet and mail responders.