The Annual Social and Economic Supplement (ASEC) of the Current Population Survey asks respondents to report the annual earned income from their main job. Respondents can report their income in a periodicity that is convenient for them (i.e., for the year, monthly, twice a month, bi-weekly, or weekly). When redesigning the interviewer-administered questionnaire for web administration, the ASEC team was concerned that respondents would not know, or correctly enter, the total number of payments covering a full year without an interviewer present to help navigate any uncertainty. This experiment, on a preliminary version of the ASEC web questionnaire, tested respondents’ receptivity to a checkbox that indicated an “all year” response to the question asking how many paychecks were received during the year. In addition to taking away the need to know how may pay periods are in a year, the checkbox was intended to reduce burden and mitigate missing data for those who were paid for the entire year. Three-fifths of respondents were shown the checkbox version of the question and their response patterns were compared to a control version without a checkbox.

Results showed that the majority of respondents offered the experimental version of the question used the checkbox, with nearly three-fourths of those minimizing their burden by using the checkbox only. The checkbox question significantly reduced item missing data, finding that the control question was three times more likely to have missing data. However, there was evidence that the checkbox may encourage satisficing: the mean number of “all year” responses for the experimental question (67.4%) is significantly higher than for the control (47.8%). Survey designers considering using the checkbox need to weigh the tradeoff between these two errors -- fewer responses or more data with more measurement error. As the checkbox question produced “cleaner” data with less variety of responses to recode, implementing it, but with some changes, is recommended.