The purpose of the Confidentiality Notification Experiment was to determine the effect of confidentiality-related messages, written in language that respondents can easily understand, embedded in the cover letter that accompanied the initial census questionnaire mailing. The effect of two messages are of interest to this study: an administrative records-use message, which has not previously appeared in decennial production cover letters, and an alternative to the “standard” statistical purposes statement that was used in 2010 Census production materials. With regard to a administrative records data-linkage message, there is no definitive legal requirement that the U.S. Census Bureau provide respondents with a data-linkage notification; however, policy considerations come into play and stakeholders have recommended the inclusion of such a statement.

The Confidentiality Notification Experiment includes two experimental panels and a control panel. Each panel includes experimental cover letters used in the initial questionnaire mailing and the replacement questionnaire mailing. These experimental letters were sent to a sample of households in lieu of the production 2010 Census letters, which were the same as the experimental Control. All three panels were mailed according to the 2010 Census schedule. Two different treatments examine the main effects of the notification treatments. The first panel tests an added administrative records-use message within the cover letter while the second panel tests an added administrative records-use message in conjunction with an alternative statistical purpose message within the cover letter.

The results show that the experimental treatments affected neither mail return rates nor mail response rates. The experimental messages did not have an effect on respondent cooperation at the unit level. In terms of data quality, form completeness rates and item nonresponse rates results indicate that the experimental treatments had some effect on item nonresponse but we have no hypothesis-driven explanation for these results.

The item nonresponse rates for race were higher, in general, within the Medium Response Stratum (delineation of the sample into the high, medium, or low stratum was based on response propensity). The higher race item nonresponse for the “standard” statistical purposes message panel was driven by the item nonresponse for race for persons of Hispanic origin within the Medium Stratum. In addition, race item nonresponse was higher for the “alternative" statistical puposes message panel for persons of Hispanic origin in the Low Response Stratum but we are unable to determine any clear, logical relationship to the treatments.

The administrative records-use message also appears to have increased item nonresponse to relationship, primarily within the Medium Response Stratum (each of the two treatment panels significantly differed from the Control). However, it is important to note that the item nonresponse for the relationship item is relatively low across all panels in that it is less than one percent.

The item nonresponse for the household-level items was not statistically significantly different across panels. In addition, the average household size across the experimental panels, as measured by the population count data item on the first page of the questionnaire, was not significantly different among the panels at any level.

In terms of public reaction, to our knowledge, there were no discernible confidentiality concerns exhibited in 2010 via print media. Audio public reaction via media is more difficult to monitor. However, as far as we know, there were no consequential statements voiced over the air related to confidentiality concerns or the uses of census data.

Conducting the experiment in a census environment produced results that more closely predict the 2020 Census results compared to mid-decade tests. Although internal and external stakeholders may change the approach to informed consent messaging within the decade leading up to the 2020 Census, the results from this experiment can provide a benchmark for developing informed consent messages on record linking. While it may be useful to conduct mid-decade research on the underlying causal factors of the unexpected item nonresponse results, it is important to keep in mind that these item nonresponse differences are generally less than one percent (in most cases less than half a percent) and may not be of practical significance. We recommend moving forward toward 2020 with the revised statistics statement to describe uses of census data and the inclusion of the administrative records-use message on census materials. In addition, it is important to test how these messages affect response and data quality in other census data collection modes where the dissemination of this information may be more active (i.e., Internet respondents are required to click that the text was read) than passive (i.e., paper letter included in the mailout/mailback materials).