Opening Keynote Speaker

Paradata within the Total Survey Error Framework: Successes, Challenges, and Gaps
Dr. Frauke Kreuter
Associate Professor
Joint Program in Survey Methodology
University of Maryland

Paradata now play an increasing role in the survey production process within the Federal Statistical agencies. So far paradata are mostly used to address nonresponse error, though in some instances paradata are also used to examine measurement error. To expand our thinking about the roles paradata can plan, this presentation will have three parts: First, I will highlight current paradata collection activities within the Federal Statistical System and elsewhere and summarize typical applications. Second, I will discuss the challenges we have seen in the collection and use of paradata, in particular those arising through measurement error inherent in those paradata. Third, I will outline the use of paradata within the larger Total Survey Error framework. Paradata can help us understand many of the additional error sources beyond NR and ME, such as coverage and coding error. Some of the data we need for this larger viewpoint are already routinely collected, even if they are not necessarily thought of as paradata. We will discuss whether integrating these additional data into the TSE framework can help us improve the survey production process as a whole.

Frauke Kreuter is an Associate Professor at the University of Maryland in the Joint Program in Survey Methodology, and currently head of the statistical methods research department at the Institute for Employment Research in Nuremberg, Germany. She has taught several short courses on paradata, and is guest editor of a special issue of JRSSA on paradata. Right now she is organizing an edited volume on the improvement of surveys through paradata. Her research on paradata is partially funded through a 3-year ESRC Research Grant (with G. Durrant and P. Smith both at the University of Southampton: RES-062-23-2997).

Plenary Panel

• What paradata are collected?
  
• Are standardized paradata being collected across modes and/or studies?
  
• Is there a set of key indicators used to assess survey process quality?
  
• What are the challenges to:
  
  • collecting and processing paradata?
    
  • collecting standardized paradata and producing standard indicators across modes and/or studies?
    
  • providing tailored indicators to address study-specific design issues
    
  • implementing responsive designs based on process quality indicators?
• How are these being met?
  
• What challenges remain?
  
• What are the overhead costs associated with collecting and processing paradata and developing dashboards, and how are they justified?

• Ananth Koppikar, Mathematica
• François Laflamme, StatCan
• Barbara O'Hare, Census
• Andy Peytchev, RTI
• Roger Tourangeau, Westat

Recent Innovations

• Karen Davis, RTI
• Abie Riefer, Westat
• Linda Bandeh, Mathematica
• Lon Hofman, Stat Netherlands/Blaise
• Patty Maher, ISR/Michigan
• Josh Seeger, NORC
• Jack Marshall, Census

Demonstrations

New Technologies: Mobile Data Collection

Survey Uses of Metadata

Management Challenges in CAI Survey Organizations

• Challenges related to stimulating innovation and enhancing and developing staff engagement

  Innovation is a necessary ingredient for success; however, many organizations face challenges related to stimulating innovation and encouraging staff engagement in this effort. In this session, we would like to generate discussion on best practices managers/leaders use to encourage innovation from staff.

• Challenges presented by the current Federal budget environment

  Everyone is aware that there is significant pressure on federal budgets. In this session, we are interested in hearing about how that pressure is affecting the mission of various federal agencies, as well as how it might be affecting the work/projects and proposals being done by the government contractors.

• Challenges associated with managing technology change

  The research environment today presents a landscape of continually changing technologies. In this session, we are discussing how organizations integrate new technologies and proactively identify and manage the associated impacts such as budgeting and planning, assessing the impact on current processes, transitioning to the new technology, staff training and education, end-user support, communication, and staff acceptance.

• Diane Herz, Mathematica-MPR
  
• Michael Horrigan, Bureau of Labor Statistics
  
• Barbara LoPresti, Bureau of Census
  
• Patty Maher, ISR-University of Michigan.
  

Addressing and Reducing Respondent Burden to Gain Cooperation

New Technologies: New Uses of Existing Technologies

CARI: Approaches for Quality Control and Interviewer Performance Feedback

• What are the main criteria for reviewing the recording?
• How are the recorded interviews reviewed?
• Who does this and do the reviewers go through the full recorded interviews or segments?
• How are the results of the review communicated to interviewers and researchers?
• How to measure the benefit by using CARI method?
• What areas can we make improvement to use this technology?

Usability and Accessibility

Data Management

• Data management processes for preparing both public dataset and internal microdata sets
  
• Role of paradata in file development
  
• Preparing data collected through multiple modes
  
• Documenting public datasets
  
• Disclosure analysis - best practices and new approaches
  
• Contextualizing CAI results through linkage to public data sets
  
• What does data.gov mean CAI survey data management and dissemination?

New Technologies: Survey Management Technologies

Web-Based Surveys

Going Mobile: Ensuring Security of New Data Collection Platforms

Blending CASIC Designs with Administrative Records