To evaluate the level of investment in the paid advertising component of the 2010 Census Integrated Communication Campaign, we conducted an experiment in eight matched-pair geographic areas known as designated market areas. These pairs were matched on criteria such as race and ethnic composition, hard-to-count scores, presence of media outlets, and mail return behavior from Census 2000 and the American Community Survey. On average, the areas represented population sizes with adults aged 18 and older of around 500,000. We randomized selection of one area from each pair to receive a “heavy-up” dosage of paid television and radio advertising during the awareness and motivation phases of the campaign. The goal was to increase the treatment area media spending by 100 percent compared to the control areas.

We used two primary data sources to evaluate the experiment: a pre and post-census sample survey and operational Census data from the sixteen designated market areas. The sample survey interviews were part of another 2010 Census evaluation – the Census Integrated Communication Campaign Evaluation. The sample for our evaluation consisted of 1,924 pre-Census and 2,046 post-Census interviews located within the sixteen designated market areas (referred to as wave 1 and wave 3 interviews). The interviews measured levels of self-reported exposure to the Integrated Communication Campaign as well as knowledge, attitudes, and opinions about the Census. Addresses from the sample interviews were matched to Census operational data to create an additional variable indicating whether the household completed and mailed back a census form.

The second data source, the Census operational extract, included all households located within the counties that comprise the sixteen designated market areas. It contained information such as Census mail return status, cooperation during Nonresponse Followup, whether a household was designated to receive a replacement form, and the 2010 Census data collected for the household (for example, race of householder, age of householder, and household size).

Prior to addressing the major research questions of interest, we sought to understand how successful we were in increasing the paid advertising in the treatment areas. The heavy-up experiment was limited to increased advertising as part of the Diverse Mass audience plan and relied on local (as opposed to national) media buys to deliver the extra dosages. A media buy is a purchase of media with specific, controlled requirements for schedule through the day and over weeks, audience characteristics, media characteristics, and program genre characteristics. This placed restrictions on the experiment. Analysis of post-Census expenditure and gross rating point audit data indicated the experiment mostly fell short of the goal of increasing paid advertising by 100 percent in the treatment areas. The audit data also revealed that, even in the control sites, the combined total audience paid advertising reach and frequency was very high in some sites – a result of advertising from the targeted Black audience plan, the national Hispanic audience plan, and last minute media buys as part of the Rapid Response Program. These two factors – potential saturation and inability to double the advertising dosages – are limitations that likely affected our results.