As a scientific investigation, evaluation of web survey usability requires sound methodology. Yet, web survey usability evaluation is a relatively young field that is filled with challenges and opportunities. To promote research in this area and exchange information with colleagues in the public opinion research community, researchers at the U.S. Census Bureau will present a panel on methodological challenges and opportunities in web survey usability evaluation at the 2016 American Association for Public Opinion Research conference.

Web surveys are now widely used to gather information from the public. Like all software applications, web surveys are subject to usability or user experience issues. Usability, in this context, refers to the extent to which a respondent can self-administer a web survey effectively, efficiently and satisfactorily. Usability problems may frustrate the respondent and slow down survey completion and thereby compromising their ability to provide accurate responses. It is thus crucial to ensure adequate ease of use of web survey instruments through rigorous evaluation.

Usability evaluation can both reveal problems affecting user experiences and help address those problems. Addressing usability issues during the development lifecycle of the instrument can help to minimize usability-induced measurement or nonresponse errors in a cost-effective manner.

In our panel at AAPOR, we will introduce the general approach the Census Bureau uses to evaluate web survey usability. We will discuss five major challenges that we experience in practice: (1) conducting cognitive probing techniques, (2) utilizing and interpreting eye tracking data, (3) evaluating accessibility on mobile devices, (4) incorporating usability evaluation in the agile software development process, and (5) completing surveys on a smartphone.