Test Objective

• In January through March of 2006, the American Community Survey (ACS) conducted the first test of new and modified content since the ACS reached full implementation levels of data collection. The results of that testing will determine the content for the 2008 ACS.
• Improvements to questions about household facilities to identify specific components of plumbing and kitchen facilities are needed for editing purposes, particularly for vacant units. These data will also be useful for evaluating the quality of housing in Puerto Rico and certain areas within the United States. The telephone availability question needs improvement to account for increased cell phone usage. Other objectives are to improve the reliability based on the Census 2000 Reinterview Survey, and to improve the underreporting of complete plumbing/kitchen and telephone.

Methodology

• The Content Test compared two versions of the facilities questions. The test version modified the current questions by breaking out the specific components into seven “yes/no” questions and modernized the language (e.g. replaced “piped water with “running water.”) An instruction was added to the telephone availability question to include cell phones. The control version replicated the current ACS questions.

Research Questions/Results

• Research Question 1: Will the changes to the kitchen and plumbing facilities questions impact item non-response?
  The results support the hypothesis that item non-response rates for the test version were equal to or less than those for the control version. There were no significant panel differences in item non-response rates at the national level and in the high or low response areas for any of the facility questions. Non-response rates for the questions on plumbing and kitchen facilities were less than three percent and around 12 to 13 percent for the question on telephone service in both the test and control versions.
• Research Question 2: Will the changes to the kitchen and plumbing facilities questions increase the percent of complete plumbing facilities and complete kitchen facilities?
  While the percent of complete kitchen and complete plumbing facilities didn’t increase, the percent was maintained and therefore met the selection criterion. The results show that there were no significant panel differences at the national level and within high and low response areas in the percentages of households with complete plumbing facilities and for those with complete kitchen facilities (both about 98 percent). The relatively small sample included in the 2006 ACS Content Test makes it very difficult to detect change in rare characteristics like the estimated number of households lacking complete plumbing and complete kitchen facilities. Results from the 1990 Census and Census 2000 indicate that units lacking these facilities are much more likely to be vacant and/or located in rural areas. The design of the 2006 test did not facilitate the over-sampling of vacant housing units and housing units in rural areas.
• Research Question 3: Can we improve reliability of the data by making changes to the kitchen and plumbing facilities questions: reformat the series to ask about each component separately as a yes/no response and change the wording to further distinguish (a) and (d) in the test version?
  The results support the hypothesis that the gross difference rate was equal to or less than the control version. No significant panel differences in the gross difference rates were shown for complete plumbing and complete kitchen facilities.
• Research Question 4: Do the changes to the telephone question (adding instruction, including it as part of the reformatted facilities series) impact item non-response?
  The results support the hypothesis that the item non-response rate was equal to or less than the control version. There were no significant panel differences in item nonresponse rates for available telephone service. This was true at the national level, and in high and low response areas.
• Research Question 5: Does the percent of available telephones change by including an explicit instruction to include cell phones and including it as part of the yes/no facilities series?
  The results support the hypothesis that the percent of telephone availability was equal to or higher than the control version. The percentages for households with available telephone service were higher nationally, and in high and low response areas.

Summary of Empirical Results

The following factors support the use of the test questions on plumbing and kitchen facilities and telephone service availability on the 2008 ACS questionnaire:

• The test and control versions show similar percentages and non-response rates.
• The follow-up operation shows that the reliability of the responses is similar between the test and control.
• Editing of the plumbing and kitchen facilities items will be greatly enhanced. The component approach followed in the test version will allow us to investigate whether vacant units that only lack refrigerators should be classified as lacking complete kitchen facilities. This approach will also allow housing analysts to evaluate the relationship between substandard housing and individual components of plumbing and kitchen in the US and Puerto Rico.

Empirically, the test version performed better than the control version.