U.S. flag

An official website of the United States government

Skip Header


Methodology and Theory for Design-Based Calibration of Low-Response Household Surveys with Application to the Census Bureau 2019-20 Tracking Survey

Written by:
RRS2022-03

Abstract

Motivated by the Census Bureau's 2019-2020 Tracking Survey, conducted in two modes as a telephone probability survey and also as a nonprobability web survey, this report studies methodological issues concerning household survey estimation following weight-adjustment. The issues of interest include: the creation of base-weights for telephone RDD (random-digit-dialing) surveys; the adjustment of base-weights in probability and nonprobability surveys by generalized raking methods to calibrate respondents to national population targets for so-called `post-stratification' variables; the handling of these base-weighting and calibration methods when some of the respondents' post-stratification variable values are missing; the mathematical theory justifying the large-sample behavior and variance estimation for survey estimates; and assessment of the success of weight-adjustment in making survey respondents represent the US population. Well-established theory (Deville and Särndal, 1992 JASA) says that when inverse-inclusion-probability survey weights are calibrated to true totals in a probability survey with complete response, the design-based estimates are design-consistent and have reduced variances, and the weights move very little. Yet in raking real low-response surveys like the Tracking Surveys, the movement of most weights is large. New design-based theory provided in this report justifies generalized raking in settings where the correct weights (defined here for the first time in a design-based framework) satisfy a parametric model, and large-sample theory is established for adjusted-weight survey estimators and their variance estimates whether such a model holds or not. This theory makes precise a sense in which weight calibration and variance estimation in surveys with either low response or non-probability designs is unavoidably model-based. To assess the validity of survey weight adjustments made by potentially misspecified models, survey methodologists often compare with known national targets the survey-weighted estimated proportions with respect to benchmark variables not used in the weight adjustment. In the Tracking Survey, this is done using the raked weights for several benchmark variables and also for augmented demographic benchmarks obtained by cross-classifying demographic variables whose marginal totals have been used in calibration.

Page Last Revised - June 22, 2022
Is this page helpful?
Thumbs Up Image Yes Thumbs Down Image No
NO THANKS
255 characters maximum 255 characters maximum reached
Thank you for your feedback.
Comments or suggestions?

Top

Back to Header