The American Community Survey (ACS) is a household survey conducted by the United States Census Bureau that is designed to meet the needs of federal government agencies for that require data on the American public. As part of the initial step for the next ACS Content Test, the U.S. Census Bureau developed a Statement of Work designed to pretest new and revised questions that were proposed by Sponsoring Agencies. The Pretest design involved cognitive testing methods to evaluate the current ACS question wording against experimental versions of the questions and to evaluate alternative versions of new questions under consideration for the ACS. The results of the cognitive interviews conducted as part of the pretest were used to inform the decision regarding the versions of the questions that will be tested in a production version in the field in the fall of 2010. This report documents all aspects of the pretest contract, Cognitive Testing of the American Community Survey Content Test Items, which was conducted from March 20, 2009, to August 12, 2009.

The primary objective of the pretest cognitive testing was to evaluate eight question topics as part of the ACS survey. Six of the eight question topics were current questions in the ACS while two were new question topics that were proposed as additions for a future version of the ACS. These question topics that were tested were organized into modules for the cognitive interviews. The modules that were proposed as new questions for the ACS were the following two modules: Computer and Internet Usage and Parental Place of Birth. The modules with revised questions were the following: Veterans Identification, Veterans Period of Service, Food Stamps, Public Assistance Income, Wages and Salary Income and Property Income (Interest and Dividends).

The modules selected by the Census Bureau for the pretest were chosen for a variety of reasons. In general, legislation and quality issues identified during a review of earlier ACS data led to the identification of some of the question revisions that are proposed as well as to the need for additional data to be collected. In the Housing section of the ACS, there were two modules that were identified for testing. The Computer and Internet Use questions are proposed additions because the Federal Communications Commission (FCC) is interested in tracking deployment of broadband use and ownership of computers. The food stamps questions are being tested because there was a name change to this program and the new name, Supplemental Nutritional Assistance Program, was included in the questions in different ways to determine which was most widely recognized and would lead to the most accurate reporting. In the Detailed Person section of the ACS, there were six module tested. The Parental Place of Birth questions have been included in a previous Census questionnaire, but have never been part of the ACS. The Veterans Identification and Period of Service questions were revised because data users wanted to simplify the categories for periods of service. The Wages and Salary Income, Property Income, and Public Assistance questions are being tested to determine the best way to bridge the gap between the ACS reported data and administrative records data.

As part of the pretest, two or three alternate versions of the questions were tested using three different modes (self-administered, interviewer-administered in-person, and intervieweradministered by telephone) and two languages (English and Spanish). In the final recommendations based on the cognitive testing, the best of the alternative versions were selected to be administered in the next phase of the Content Test, the field test. For modules that are currently included in the ACS, the recommendation made in this report is for the one best alternative version of the questions to be evaluated against the current version. In the case of the new question topics, two versions are recommended for evaluation in the field test.

To implement this pretest of the new and revised questions, the U.S. Census Bureau contracted with RTI International* (RTI) to conduct the cognitive interviews and make the recommendations for the versions to be tested in the field. In order to maintain the schedule for this work, RTI subcontracted with two other research organizations, Research Support Services (RSS) and Westat. Each organization conducted a specified number of the cognitive interviews and contributed to the analysis and reporting.

The Cognitive Testing of the ACS Content Test Items included a total of 220 cognitive interviews, 115 interviews of which were conducted in English and 105 interviews of which were conducted in Spanish. Cognitive interviews were conducted primarily in May and June 2009 with a small number of interviews conducted in early July 2009.