The Update Enumerate operation assigned enumerators to interview and update addresses in selected rural areas. These interviews were completed on paper questionnaires and collected information on housing unit status, population count at the address, and demographic data about the people living at the housing unit. All completed questionnaires were returned to the Local Census Office for check-in and subsequently shipped to a data capture center. The Update Enumerate quality assurance program ensured the Update Enumerate enumerators understood and followed interviewing and listing procedures through enumerator observations, a separate reinterview operation of enumerators to detect and deter falsification, a Dependent Quality Control Check of production listing work, and through Delete Verification. A Final Delete Verification was also conducted to verify addresses deleted for the first time by a quality control enumerator.

Enumerator observations were required for all production and quality control enumerators to ensure the enumerators understood and followed operational procedures. Observations should have been conducted and observation checklists were supposed to be filled out for all 15,082 production and 5,884 Quality Control enumerators. However, we received a little less than half of the observation checklists we expected. Of the forms received, we saw that 89.83 percent of enumerators passed the observation and 0.86 percent failed. The remaining enumerators had “Other” or no outcome marked, likely because they left the operation before an observation could be completed. An automated observation checklist would eliminate these data issues and ensure all enumerators were observed in a timely manner.

The Update Enumerate reinterview assigned a separate staff of enumerators to revisit a sample of Update Enumerate addresses and verify select data from the original interview. All reinterview paper forms were shipped to data capture centers, and all Update Enumerate and Update Enumerate reinterview data were delivered to the Matching, Review, and Coding System for comparison and final reinterview outcome coding. Cases were selected for reinterview by five different methods:

• Random – a sample of all eligible cases was automatically selected for each enumerator
• Outlier – additional reinterview was automatically selected for enumerators whose work differed significantly from all work within their Crew Leader District
• Supplemental – additional reinterview could have been manually selected by the Local Census Office Quality Control staff
• Hard fail – when an enumerator received a “Hard Fail” outcome (either through reinterview failure, or non-reinterview failure), all eligible cases completed by that enumerator were selected for reinterview
• Vacant – any Housing Unit with a status of Vacant – regular was automatically selected for reinterview

The total number of cases selected for reinterview was 232,276 (16.74 percent) of the 1,387,564 Update Enumerate cases that were eligible for reinterview. Random reinterview accounted for 74,932 (5.40 percent) of the cases eligible for reinterview. The random reinterview selection rate was higher than the expected four percent because it was not a simple random sample, but was instead a systematic sample starting within the first three cases for each enumerator (i.e., it was stratified by enumerator). Therefore, enumerators who only worked a small number of Update Enumerate cases had a higher percentage of their cases selected for random reinterview. This design was used in an effort to ensure all enumerators had at least one case checked in reinterview.

Of the 215,833 reinterview cases received by the Matching, Review, and Coding System, 175,247 (81.20 percent) were assigned a final reinterview outcome of “Pass,” which means the original data were collected with no critical mistakes or intentional falsification by the enumerators. Another 2,698 (1.25 percent) were found to have unintentional mistakes, and 80 (0.04 percent) were found to have intentional data falsification. This falsification was found for 33 enumerators, which is 0.22 percent of all Update Enumerate enumerators. Another 21,845 cases (10.12 percent) had no meaningful final outcome because the majority of their data were received after the coding effort closed. This indicates that the reinterview operation needed more time to account for shipping and data capture delays after the field work was concluded.

The Dependent Quality Control Check was a dependent verification of the listing activities conducted by Update Enumerate enumerators. A sample of the listing actions with every assignment area was verified by the quality control enumerators. The Dependent Quality Control Check was successful in identifying listing errors. Only 4.55 percent of all assignment areas failed and needed to be recanvassed. The Dependent Quality Control Check results showed an outgoing error rate of 0.18 percent. In Delete Verification, which was implemented to provide verification for 100 percent of all “delete” actions, 15.40 percent of the “delete” actions were reversed by the quality control enumerators. Final Delete Verification, where deletes introduced by quality control enumerators were double checked, showed that 3 of the 11 deletes (27.27 percent) that were first found by a quality control enumerator were reversed after being checked a second time.

We recommend using an automated instrument for these data collection components in the future. An automated form would assist in conducting Delete Verification, because 32.32 percent of the write-in box responses that were supposed to contain the corrected action code instead contained invalid codes, while 26.42 percent were simply blank.