Purpose: The purpose of this standard is to establish quality criteria for releasing information products.

The OMB’s Statistical Policy Directive No. 3 and Statistical Policy Directive No. 4 describe requirements for notifying the public of the release of information products. The Census Bureau’s Product Release Notification Policy and Policies and Procedures Manual (Chapter B-13 – Clearance and Release of Public Information Materials) describe procedures for notifying the PIO about information products to be released to the public.

Note: Statistical Quality Standard F2, Providing Documentation to Support Transparency in Information Products, contains specific requirements about documentation that must be readily accessible to ensure transparency in information products released outside the Census Bureau.

Scope: The Census Bureau’s statistical quality standards apply to all information products released by the Census Bureau and the activities that generate those products, including products released to the public, sponsors, joint partners, or other customers. All Census Bureau employees and Special Sworn Status individuals must comply with these standards; this includes contractors and other individuals who receive Census Bureau funding to develop and release Census Bureau information products.

Exclusions:
In addition to the global exclusions listed in the Preface,

1. Requirements F1–2 and F1–3 of this standard do not apply to:
   • Professional papers, presentations, and similar information products.
   • Information products delivered to sponsors or clients (e.g., data files and tabulations).
2. Requirements F1–7 through F1‑10 of this standard do not apply to:
   • Professional papers, presentations, and similar information products.

Key Terms: Coefficient of variation (CV), coverage ratio, dissemination, estimate, information product, metadata, nonresponse bias, nonsampling error, releases of information products, response rate, sample design, and sampling error.

Requirement F1–1: Neither protected information nor administratively restricted information may be released outside the Census Bureau, except as allowed under applicable federal laws (e.g., Title 13, Title 15, and the Confidential Information Protection and Statistical Efficiency Act) and data-use agreements.

Sub-Requirement F1–1.1: Throughout all processes associated with releasing information products, unauthorized release of protected information or administratively restricted information must be prevented by following federal laws (e.g., Title 13, Title 15, and Title 26), Census Bureau policies (e.g., Data Stewardship Policies), and additional provisions governing the use of the data (e.g., as may be specified in a memorandum of understanding or data-use agreement). (See Statistical Quality Standard S1, Protecting Confidentiality.)

Requirement F1–2: Information products released to the public by the Census Bureau must be released according to a dissemination plan that addresses:

1. What information product(s) are planned for release.
2. The release schedule. The release schedule for all regular or recurring information products for the upcoming year must be published on www.census.gov before January 1 of that year.(See OMB Statistical Policy Directive No. 4.)
3. The reviews and approvals needed before releasing the information products to the public.
4. The mode of release by the Census Bureau.

Requirement F1–3: Policies and procedures for disseminating information products, including those related to any planned data revisions or any corrections for data quality issues identified after an information product has been released, must be documented and published on the Census Bureau’s Internet Web site.

Requirement F1–4: Information products must not be released outside the Census Bureau until they receive the appropriate reviews and approvals. (See Statistical Quality Standard E3, Reviewing Information Products.)

Requirement F1–5: Embargoed news releases and data files must not be released to the public by any means (including print, broadcast, Internet, podcast, blogs, or in any other form) before the specified date and time of release. (See the U.S. Census Bureau Embargo Policy.)

Requirement F1–6: Information products must comply with the Census Bureau’s statistical quality standards and must be free of serious data quality issues in order to be released outside the Census Bureau without restrictions.

1. Serious data quality issues related to sampling error occur when the estimated coefficients of variation (CV) for the majority of the key estimates are larger than 30 percent.
   
   

   Notes:

   1. This requirement does not apply to secondary estimates. For example, if the estimated month-to-month change is the key estimate, and the monthly estimates are secondary, the requirement applies only to the estimated month-to-month change.
   2. Statistical Quality Standard A1, Planning a Data Program, provides requirements for identifying key estimates.
2. Serious data quality issues related to nonsampling error occur when:
   1. All products:
      1. The data suggest that the primary survey concepts are not clearly defined or that measurement of the concepts failed for some reason.
      2. The key estimates are inconsistent with our base of knowledge about the characteristic being estimated.
      3. Issues that are serious enough to raise concerns about the accuracy of the data occur in sample design, sampling methods, questionnaire or forms design, data collection, data processing, estimation procedures, or the underlying assumptions of a model.
   2. Products derived primarily from census or survey data:
      1. Unit response rates for surveys or censuses, or cumulative unit response rates for panel or longitudinal surveys, are below 60 percent.
      2. Sample attrition from one wave to the next wave in panel or longitudinal surveys is greater than five percent.
      3. Item response rates or total quantity response rates on key items are below 70 percent.
      4. Coverage ratios for population groups associated with key estimates are below 70 percent.
      5. Combined rates for key estimates (e.g., computed as unit response item response coverage) are below 50 percent.

   Notes:

   1. These thresholds are provided because bias is often associated with low response rates or with low coverage ratios. If nonresponse bias analyses or other studies show that the bias associated with nonresponse is at an acceptable level, or that steps taken to mitigate nonresponse bias or coverage error are effective, these thresholds do not apply.
   2. The Census Bureau conducts a few surveys that do not use probability samples. Generally, they are establishment surveys that select the largest units in the target universe and do not attempt to collect data from the small units in the universe. For these surveys, the above thresholds do not apply. These surveys have serious data quality issues if the responding units do not comprise at least 70 percent of the target universe, based on the unit response rate or the total quantity response rate, as appropriate.
   3. Statistical Quality Standard D3, Producing Measures and Indicators of Nonsampling Error, specifies requirements on computing response rates.

Sub-Requirement F1–6.1: Information products with data free from the serious data quality issues described in Requirement F1–6 may be released outside the Census Bureau with no restrictions, subject to confidentiality constraints.

Sub-Requirement F1–6.2: Information products with data that have any of the serious data quality issues in Requirement F1–6 may be released outside the Census Bureau only under the restrictions described below.

1. Restrictions for information products with serious data quality issues related to sampling error:
   
   The information product must:
   1. Note that the CV exceeds 0.30 for a majority of the key estimates.
   2. Note that data users should exercise caution when using estimates with high sampling error.
   3. Indicate why the data are being released (e.g., aggregates of the estimates may be useful or the knowledge that the estimates have extremely high magnitude or extremely low magnitude may be useful).
2. Restrictions for information products with serious data quality issues related to nonsampling error:
   1. Products that are Census Bureau publications or regular or recurring products (i.e., products governed by Statistical Policy Directive No. 3 or Statistical Policy Directive No. 4):
      
      
      1. The program manager must obtain a waiver before releasing the information product.
      2. The information product must summarize any nonsampling error issues related to Requirement F1–6, item 2a (1 through 3).
      3. If response rates, coverage ratios, or the combined rates fall below the thresholds in Requirement F1–6, item 2b:
         1. The key estimates affected must be identified.
         2. A table must be included that provides the response rates or coverage ratios for key estimates in enough detail to allow users to evaluate how the issue may affect their use of the data. Other quantitative measures of the impact of the issue should be included to the extent feasible.
      4. The information product must include details about the potential impact of the quality issues on the data.
      5. The information product must include the URL of the complete documentation on the nonsampling error issues.
   2. Products released to sponsors:
      1. The information product must summarize any nonsampling error issues related to Requirement F1–6, item 2a (1 through 3).
      2. If response rates, coverage ratios, or the combined rates fall below the thresholds in Requirement F1–6, item 2b:
         
         1. The key estimates affected must be identified.
         2. A table must be included that provides the response rates or coverage ratios for key estimates in enough detail to allow users to evaluate how the issue may affect their use of the data. Other quantitative measures of the impact of the issue should be included to the extent feasible.
      3. The information product must include details about the potential impact of the quality issues on the data.
      4. The delivery of the product to the sponsor must include the complete documentation on the nonsampling error issues or a URL where the documentation is accessible.
   3. Products that are not Census Bureau publications or are not regular or recurring products (e.g., custom tabulations, data files, professional papers, working papers, technical reports, and similar products):
      1. Release to the public is not allowed, except as noted in item 2) below. The information product may be released only on request. If released on request, the information product must:
         
         
         1. Include this disclaimer: “These data are being released on request, despite concerns about their quality. The Census Bureau’s policy is not to withhold data that are available, unless releasing such data would violate confidentiality requirements. The Census Bureau recommends using these data only for research or evaluation purposes, and not to make statements about characteristics of the population or economy because they do not meet the criteria outlined in the Census Bureau’s Statistical Quality Standard: Releasing Information Product.”
         2. Summarize the nonsampling error issues.
         3. Include summary metadata describing the issues and the impact on the data.
         4. Provide the URL of the complete documentation on the nonsampling error issues.
      2. Release is permitted only for information products whose purpose is not to report, analyze, or discuss characteristics of the population or economy, but whose purpose is to:
         
         • Analyze and discuss data quality issues or research on methodological improvements, or to
         • Report results of evaluations or methodological research
      
      
      3. External researchers at the Census Research Data Centers may not have access to confidential data that are affected by serious data quality issues, except to analyze the data quality issues, including developing potential solutions. If the researcher has corrected the data quality issues and the Census Bureau has determined that the researcher’s solutions are appropriate, the revised data may be used for subject-matter (e.g., poverty) analyses.

Requirement F1–7: When a data quality issue that might be serious is suspected in a previously released information product, the program manager must notify Census Bureau senior management of the issue immediately after it has been identified. At a minimum, the senior managers to be notified include:

1. The Division Chief(s) responsible for the program with the suspected data quality issue.
2. The Associate Director responsible for the program with the suspected data quality issue.
   
   Note: These senior managers will decide whether the issue should be escalated to the Deputy Director and provide guidance on the appropriate actions to take and the specific stakeholders or organizations to notify regarding the suspected data quality issue.

Requirement F1–8: When serious data quality issues are identified in a previously released information product, a notification must be disseminated to alert the public. If the product was released to a sponsor, the notifications must be made to the sponsor.

1. The notification must be disseminated immediately after identifying a serious data quality issue, even if the issue is not yet fully understood.
   
   1. If appropriate the data affected by the data quality issue must be removed from the Census Bureau’s Internet Web site at this time.
2. The notification must include the following components, with additional information that facilitates understanding the issue and its effects as appropriate:
   
   1. A description of the issue.
   2. A description of what is known about the effect on the data.
   3. A description of what is known about the cause.
   4. A statement indicating the data have been removed until the issue has been fixed (if appropriate).
   5. Plans for addressing the issue.
   6. Expected release dates of revised products.
3. If the notification is disseminated before the issue is fully understood, it must be updated when a more complete understanding is achieved.
   
   Note: Program managers must notify the responsible Division Chief(s) and Associate Director (Requirement F1–7) before making notifications to the public or sponsors.

Requirement F1–9: Any serious error or data quality issue identified in a previously released information product must be addressed appropriately.

Examples of appropriate actions to address serious errors and data quality issues include:


• Correct the error and re–release the product.
• Release an "errata" document for the product, describing the error and the correction.
• If it is not feasible to correct an error, release a description of the error and its likely effects on the program’s estimates and results.
• If a data user or a sponsor reported the error, acknowledge the report and indicate when the issue is expected to be resolved. If the error will not be corrected, respond and explain to the user why it will not be corrected and what actions will be taken to address the error.

Sub-Requirement F1–9.1: Serious errors or data quality issues identified in a previously released information product must be documented by completing the Dissemination Incident Report found in Appendix F1 and submitting it to the Quality Program Staff.

Requirement F1–10: Information products approved for release to the public must be published on the Census Bureau’s Internet Web site and must adhere to the requirements of Section 508 of the U.S. Rehabilitation Act.