Purpose: The purpose of this standard is to ensure that methods are established and implemented to promote the collection of high quality data from respondents.

Scope: The Census Bureau’s statistical quality standards apply to all information products released by the Census Bureau and the activities that generate those products, including products released to the public, sponsors, joint partners, or other customers. All Census Bureau employees and Special Sworn Status individuals must comply with these standards; this includes contractors and other individuals who receive Census Bureau funding to develop and release Census Bureau information products.

In particular, this standard applies to establishing and implementing data collection methods for data programs that obtain information directly from respondents, including reimbursable surveys and surveys in which interviewers collect information from establishments.

Exclusions:

In addition to the global exclusions listed in the Preface, this standard does not apply to:

• Administrative records data acquired under agreements with other organizations and not collected by interviewers.

Key Terms: CAPI, CATI, coverage, data collection, dress rehearsal, fax imaging, field test, load testing, mail-out/mail-back, measurement error, nonresponse bias, nonresponse follow-up,reinterview, response error, response rate, supplemental reinterview, systems test, and touch-tone data entry (TDE).

Requirement B1-1: Throughout all processes associated with data collection, unauthorized release of protected information or administratively restricted information must be prevented by following federal laws (e.g., Title 13, Title15, and Title 26), Census Bureau policies (e.g., Data Stewardship Policies), and additional provisions governing the use of the data (e.g., as may be specified in a memorandum of understanding or data-use agreement). (See Statistical Quality Standard S1, Protecting Confidentiality.)

Requirement B1-2: A plan must be developed that addresses:

1. Data collection methods (e.g., interview mode, use of incentives, and reference periods), systems, and procedures.
2. Definitions for what constitutes an interview or response (i.e., a complete interview, a sufficient partial interview, or an insufficient partial interview).
3. Verification and testing of the data collection methods, systems, and procedures. (Statistical Quality Standard A2, Developing Data Collection Instruments and Supporting Materials, addresses questionnaire content pretesting and instrument testing.)
4. Training for staff involved in the data collection effort.
5. Monitoring and evaluating the quality of the data collection operations.
   
   Note: Statistical Quality Standard A1, Planning a Data Program, addresses overall planning requirements, including estimates of schedule and costs.

Requirement B1-3: Data collection methods must be designed and implemented in a manner that balances (within the constraints of budget, resources, and time) data quality and measurement error with respondent burden.

Sub-Requirement B1-3.1: Systems and procedures must be developed to implement the data collection.

Examples of data collection activities for which systems and procedures should be developed include:

• Listing possible sampling units.
• Producing paper questionnaires and related materials (e.g., printing and assembling mail-out packages). (Statistical Quality Standard A2, Developing Data Collection Instruments and Supporting Materials, addresses the design of questionnaires and materials.)
• Providing OMB-required notifications to respondents.

• Providing telephone questionnaire assistance for mail-out/mail-back data collection.
• Transmitting information (by mail, electronically, the Internet, TDE, fax imaging, or other method) between respondents or interviewers and the Census Bureau.
• Formatting CAPI/CATI output files to be compatible with processing systems.
• Conducting interviews.
• Conducting nonresponse follow-up operations.

Sub-Requirement B1-3.2: Data collection systems and methods must be verified and tested to ensure that all components function as intended.

Examples of verification and testing activities include:

• Verifying that the specifications and procedures reflect the requirements of the program.
• Verifying that the materials used for data collection operations meet specifications (e.g., ensure that forms are printed properly).
• Verifying the physical assembly of mailing packages (e.g., ensure that mailing pieces fit properly in the envelopes).
• Testing the electronic data management systems (e.g., the systems used to manage cases and data between headquarters and the interviewers and between headquarters and the data processing systems) for accuracy, capacity (e.g., load testing), and reliability.
• Conducting a systems test to verify the functioning of the data collection instrument in combination with the data management systems.
• Conducting a field test to test systems and methods under realistic conditions (e.g., the dress rehearsal for the decennial census).

Sub-Requirement B1-3.3: Training for field and headquarters staff involved in the data collection effort (as identified during planning) must be developed and provided.

Examples of training topics include:

• Relevant Census Bureau policies (e.g., Data Stewardship Policy DS016, Respondent Identification Policy).
• The goals and objectives of the data collection.
• Survey specific concepts and definitions.
• The uses of the data.
• Techniques for obtaining respondent cooperation.
• Follow-up skills.

Sub-Requirement B1-3.4: Systems and procedures must be developed and implemented to monitor and evaluate the data collection activities and to take corrective actions if problems are identified.

Examples of monitoring and evaluating activities include:

• Tracking unit response rates, progress in completing interviews, and costs of the data collection, and taking corrective action when goals are not met.
• Tracking returned cases to ensure all cases are accounted for and investigating to locate missing cases.
• Verifying that interviewers follow interviewing procedures and do not falsify data (e.g., by conducting field observations, conducting reinterviews, or monitoring telephone center interviewers) and, if necessary, taking appropriate corrective action (e.g., retraining, reassigning, or dismissing interviewers).
• Collecting, tracking, and analyzing interviewer performance statistics (e.g., refusals, completed interviews, refusal conversions, login hours, and completed interviews per login hour), and providing feedback or other corrective action when necessary.
• Verifying that analysts follow data collection review procedures, and providing feedback when necessary.
• Reviewing response data for accuracy and completeness, and taking appropriate corrective action when necessary to improve accuracy or completeness.
• Reviewing response data for unexpected results and unusual patterns (e.g., a pattern of an unusually high number of vacant households) and taking corrective action when needed (e.g., providing feedback, retraining interviewers, or conducting supplemental reinterviews).
• Conducting evaluation studies (e.g., nonresponse bias analysis, coverage evaluation study, and response error reinterview study).

Requirement B1-4: Documentation needed to replicate and evaluate the data collection methods must be produced. The documentation must be retained, consistent with applicable policies and data-use agreements, and must be made available to Census Bureau employees who need it to carry out their work. (See Statistical Quality Standard S2, Managing Data and Documents.)

Examples of documentation include:

• Plans, requirements, specifications, and procedures for the data collection.
• Test designs and results.
• Instructions to respondents and interviewers about the data collection instrument.
• Quality measures and evaluation results. (See Statistical Quality Standard D3, Producing Measures and Indicators of Nonsampling Error.)

Notes:

1. The documentation must be released on request to external users, unless the information is subject to legal protections or administrative restrictions that would preclude its release. (See Data Stewardship Policy DS007, Information Security Management Program.)
2. Statistical Quality Standard F2, Providing Documentation to Support Transparency in Information Products, contains specific requirements about documentation that must be readily accessible to the public to ensure transparency of information products released by the Census Bureau.