Designing Effective Online Surveys
A 3-Day Remote Seminar Taught by Andrew Miles, Ph.D.
The Internet has revolutionized survey research. Web-based surveys are now used in many disciplines, and for good reason. Compared to traditional paper-and-pencil and face-to-face modes of data collection, they make it easy to randomize question order, skip irrelevant questions, include rich multimedia content, and adapt the survey experience based on respondent input. Add to this a wide range of question types and unparalleled ease of distribution, and it is easy to see why web-based surveys have become so popular.
But the Internet doesn’t just make it easy to program surveys: it also gives unparalleled access to a vast reservoir of potential research participants. With the help of third-party web panels or crowdsourcing websites, researchers can recruit respondents and complete data collection in a matter of days, sometimes hours. Fielding surveys online is also inexpensive, which makes it possible to obtain large samples even for researchers on a budget.
Online surveys are not a panacea, however. Web-based surveys make use of digital technologies that differ in marked ways from other survey modes, raising the question of how the mode of data collection influences results. Web surveys also can also be completed on both computers and mobile devices, a fact which comes with its own set of possibilities and pitfalls. And respondents recruited online are generally far from representative of most populations of interest.
This course provides a practical introduction to designing effective web surveys and working with online respondents. Particular attention is given to how choices made at the design stage affect response rates and data quality, with an emphasis on highlighting actionable design strategies. We also delve deeply into the advantages and disadvantages of using online samples and evaluate design-based and post-hoc approaches to addressing some of the challenges associated with online research.
Starting May 20, we are offering this seminar as a 3-day synchronous*, remote workshop for the first time. Each day will consist of a 4-hour live lecture held via the free video-conferencing software Zoom. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
Each lecture session will conclude with a hands-on exercise reviewing the content covered, to be completed on your own. An additional lab session will be held Thursday and Friday afternoons, where you can review the exercise results with the instructor and ask any questions.
*We understand that scheduling is difficult during this unpredictable time. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for two weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
More Details About the Course Content
This course presents the latest research on designing web-surveys and working with online respondents. Topics include maximizing response rates, writing effective survey questions, determining optimal survey length, adapting for mobile devices, correcting for non-probability sampling, reducing respondent fatigue, and creating effective visual layouts. We will also discuss ways to maximize the advantages of online data collection. Class meetings will consist of lectures interspersed with demonstrations of key ideas. Participants will be given the opportunity to apply the principles and skills in two take-home exercises.
Course demonstrations will use Qualtrics survey software. Take-home exercises can be completed using many web-survey software packages (e.g., Qualtrics, SurveyMonkey). Most of these software packages offer a free version but require a subscription or license to access their full capabilities. If you do not have a subscription, we recommend using the free version of Qualtrics, which allows access to a number of important design capabilities (https://www.qualtrics.com/free-account/).
Who Should Register?
This course is suitable for anyone who wants an up-to-date understanding of best practices in web survey design and working with online respondents. Course content will be valuable for both novices and seasoned researchers who have limited experience collecting data online.
- Types of survey error (coverage, sampling, non-response, measurement, survey mode)
- Web-based surveys vs. online samples
- Designing effective web-surveys
- Paging vs. scrolling designs
- All about questions (content, types, display)
- Visual elements (layout, images)
- Interactive features
- Survey length
- Mobile devices
- Pre-testing surveys
- Working with online samples
- Do you want an online sample?
- Probability vs. non-probability samples
- Recruiting respondents online
- Guarding against fraud
- Correcting for non-probability sampling
- Ethical considerations when using online respondents
- Maximizing the advantages of web-surveys and online samples
- Mixing sampling strategies and survey modes
- Planned missing data designs
- Using online respondents as research assistants
Reviews from "Statistics with R" taught by Andrew Miles
“This is one of the best introductory programming courses I’ve taken. The instruction was clear and well-organized. The instructor was great at teaching and providing super clear directions/explanations on what can be a complicated program to understand. I’ve found R difficult to grasp before this course and now feel I have a foundational understanding of R programming. The instructor is great at assisting students with questions and incorporating in-class exercises to break up the lecture and apply the skills you’ve learned or see what’s still tripping you up.”
Larisa Burke, University of Illinois at Chicago
“Andrew was very helpful and knowledgeable. He was patient and very good at answering questions. I had no knowledge of R prior to taking this course, and now I am confident that I can use it for my course and research (of course, with more practice at home). The textbook was also excellent.”
Aki Roberts, University of Wisconsin, Milwaukee
“A good basic course in R. The instructor was super helpful and really good at explaining complex ideas so they seem simple.”
Hlin Kristbergsdottir, Reykjavík University
“Andrew was an incredible teacher, very detail-oriented and knowledgeable. He provided exercises throughout the workshop to allow us to practice and truly understand what we were learning. He is incredibly approachable as well. 10/10 would recommend and do again.”
David Bodenstein, University of Toronto
Thursday, May 20, 2021 –
Saturday, May 22, 2021
Each day will follow this schedule:
10:00am-2:00pm ET (New York time): Live lecture via Zoom
4:00pm-5:00pm ET: Live lab session via Zoom (Thursday and Friday only)
The fee of $895 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.