Test Objective

• In January through March of 2006, the American Community Survey (ACS) conducted the first test of new and modified content since the ACS reached full implementation levels of data collection. The results of that testing will determine the content for the 2008 ACS.
• The objectives of the alternative questions on number of rooms and bedrooms were to provide a definition of a “separate room” and improve the respondent’s understanding that rooms and bedrooms are related; that is, the total number of bedrooms is a subset of total rooms, especially in the case of an efficiency apartment which has only one room and no bedroom. The analyses of the results from the Census 2000 and the American Community Survey mail questionnaires suggest that there were inconsistencies between the count of rooms and bedrooms. The Census 2000 Reinterview Survey indicated that the index of inconsistency for bedrooms was high (57.1).

Methodology

• The Content Test compared two versions of the rooms and bedrooms question set. The control version replicated the current ACS question.
• The test version included the following changes to the question on total number of rooms:
• Adding the word “separate” to the question stem.
• Adding the instruction that defines a “room.”
• Adding an instruction to include bedrooms and kitchens in the count of rooms.
• Modifying the exclusionary instruction by removing the “half-room” concept.
• Using an open-ended/write-in field rather than categories. The test version included the following changes to the question on number of bedrooms:
• Adding language that explicitly links the total count of rooms and the count of bedrooms.
• Provide the heuristic/rule to use for defining a bedroom as part of the instruction.
• Providing instructions for efficiency/studio apartments.
• Using an open-ended/write-in field rather than categories.

Research Questions/Results

• Research Questions 1 and 2: Will the changes to the room question improve the accuracy of the question? Will the changes to the bedrooms question improve the accuracy of the question? The results support the hypothesis that the net difference rate for the test panel was equal to or better than the net difference rate for the control panel for rooms and bedrooms.
• Research Questions 3: Will the changes to the question and response field improve the under-reporting of total rooms? The test panel resulted in a larger median number of rooms at the national level (5.7 for the test panel vs. 5.3 for the control) and within the high (5.9 vs. 5.5) and low (5.0 vs. 4.6) response areas. Therefore, the test version reduced the under-reporting of rooms.
• Research Questions 4 and 5: Will the changes to the question and the response field impact the item nonresponse rate for the total rooms question? Will the changes to the question and the response field impact the item nonresponse rate for the bedrooms question? Although there was no evidence to suggest that the test version of the room/bedroom questions improved response, the results did support the hypotheses that the response rates for the test version were similar to those using the control version. The only exception was in the low response area where a higher percentage of respondents did not answer the test version question on bedrooms.
• Research Question 6: Will the changes to the question and response field change the distribution of bedrooms? The results support the hypothesis that the reporting of “0” bedrooms was higher in the test panel than the control panel and illustrate the shifting of “1-bedroom units” to “0-bedroom” units, and “2-room” to “1-room” units.
• Research Question 7: Will the combination of changes to both the rooms and bedrooms question improve the internal consistency of responses to the two questions? The results support the hypothesis that there was more inconsistency in the reporting of rooms and bedrooms in the control panel than in the test panel at the national level (3.8 percent vs. 6.7 percent on the control) and within high and low response areas.

Summary of Empirical Results

Changes to the questions reduced underreporting of rooms, increased reporting of “0” bedrooms, and improved consistency between rooms and bedrooms while maintaining the item response rate. The net difference rate was also maintained or improved in the test panel.

Systematic response error was equal or less for the test version.

The systematic response error was larger for some room categories in the test version. Empirically, the test version performed better than the control version.