(Non-)Probability Survey Samples in Scientific Practice (Online)
Date:
09/12/2021 - 10/12/2021
Organised by:
National Centre for Social Research
Presenter:
Carina Cornesse
Level:
Intermediate (some prior knowledge)
Contact:
Paige Ridd
0207 549 7194
Paige.Ridd@natcen.ac.uk
Venue: Online
Description:
This course will be hosted in the NCRM Moodle Virtual Learning Environment. In order to participate, delegates will be issued with a personal Moodle user account linked to the name and email address provided when they register for the course. Accounts will remain live for one year from the start of the course to allow continued access to the course materials, after which they will be automatically deleted. Accounts can be deleted earlier by emailing dataprotection@ncrm.ac.uk
The course covers:
- Empirical evidence and state-of-the-art (e.g., online panels, polling, pandemic)
- Assessing and comparing bias
- Theoretical frameworks and assumptions
- Common approaches to weighting and combining samples
By the end of the course participants will:
- Have an overview of the history, theoretical foundations, critical arguments, and accumulated empirical evidence surrounding the debate about probability and nonprobability sample surveys.
- Possess the necessary skills to evaluate whether any given sample is fit for the purpose of answering a particular research question.
- Be able to choose an appropriate sample type when designing their own social scientific research studies.
Course materials:
- Participants will receive written course notes electronically (by email)
- There is no need to bring any course materials.
Pre-requisites:
- No software skills are required.
For many decades, social science researchers have almost exclusively relied on probability sample surveys when aiming to draw inferences to the general population. However, probability sample surveys are expensive and data collection is often slow. With the rise of the internet in the 21st century, therefore, it became popular to conduct fast and cheap surveys via online panels, which usually rely on web-recruited nonprobability samples. In academic circles, this has led to the reignition of an old debate about whether and under which conditions data from nonprobability sample surveys can produce accurate population estimates. This debate is ongoing and concerns many areas of social science research and practice.
This short course presents the arguments raised in the (non-)probability survey sample debate and summarizes the empirical evidence that has been accumulated so far. The short course thus focuses on providing the necessary context that social science researchers and survey practitioners need to participate in the debate. In this context, the course covers real-world survey examples from areas such as election polling and social surveys during the pandemic. In addition, the short course provides hands-on advice on the conditions under which nonprobability samples may be suitable to answer a particular research question (i.e., “fit-for-purpose” designs) and when it may be necessary to rely on probability samples instead. Furthermore, the course briefly summarizes the most common weighting and sample combination strategies used on (non-)probability survey sample data and reflects on how these approaches can help or hurt survey data quality.
The course will take place on December 9 and December 10, 2021. Each day will start at 9am and end at 3:30pm, and will include several short breaks as well as a lunch break from 12:30-1:15pm.
Suggested preparatory reading:
Blom, A. G., Cornesse, C., Friedel, S., Krieger, U., Fikel, M., Rettig, T., Wenz, A., Juhl, S., Lehrer, R., Möhring, K., Naumann, E., & Reifenscheid, M. (2020). High Frequency and High Quality Survey Data Collection. The Mannheim Corona Study. Survey Research Methods, 14(2), 171-178. https://doi.org/10.18148/srm/2020.v14i2.7735
Cornesse, C., Blom, A. G., Dutwin, D., Krosnick, J. A., De Leeuw, E. D., Legleye, S., Pasek, J.,Sakshaug, J. W., Struminskaya, B., Wenz, A.(2020) A Review of Conceptual Approaches and Empirical Evidence on Probability and Nonprobability Sample Survey Research. Journal of Survey Statistics and Methodology, 8(1), 4-36. https://doi.org/10.1093/jssam/smz041
Kohler, U. (2019). Possible uses of nonprobability sampling for the social sciences. Survey Methods: Insights from the Field, 1-12. https://doi.org/10.13094/SMIF-2019-00014
Kreuter, F., Barkay, N., Bilinski, A., Bradford, A., Chiu, S., Eliat, R., Fan, J., Galili, T., Haimovich, D., Kim, B., LaRocca, S., Li, Y., Morris, K., Presser, S., Sarig, T., Salomon, J. A., Stewart, K., Stuart, E. A., & Tibshirani, R. (2020). Partnering with a global platform to inform research and public policy making. Survey Research Methods, 14(2), 159-163. https://doi.org/10.18148/srm/2020.v14i2.7761
Rinken, S., Domínguez-Álvarez, J.-A., Trujillo, M., Lafuente, R., Sotomayor, R., & Serrano-del-Rosal, R. (2020). Combined mobile-phone and social-media sampling for web survey on social effects of COVID-19 in Spain. Survey Research Methods, 14(2), 165-170. https://doi.org/10.18148/srm/2020.v14i2.7733
Sturgis, P., Kuha, J., Baker, N., Callegaro, M., Fisher, S., Green, J., Jennings, W., Lauderdale, B.E. & Smith, P. (2018). An Assessment of the Causes of the Errors in the 2015 UK General Election Opinion Polls. Journal of the Royal Statistical Society: Series A (Statistics in Society), 181(3), 757-781. https://doi.org/10.1111/rssa.12329
Valliant, R. (2020). Comparing Alternatives for Estimation from Nonprobability Samples. Journal of Survey Statistics and Methodology, 8(2), 231–263. https://doi.org/10.1093/jssam/smz003
Wi?niowski, A. Sakshaug, J. W., Perez Ruiz, D. A., Blom, A. G., (2020). Integrating Probability and Nonprobability Samples for Survey Inference. Journal of Survey Statistics and Methodology, 8(1), 120–147. https://doi.org/10.1093/jssam/smz051
Cost:
The fee per teaching day is £30 per day for students registered at UK/EU University / £60 per day for staff at UK/EU academic institutions, UK/EU Research Councils researchers, UK/EU public sector staff and staff at UK/EU registered charity organisations and recognised UK/EU research institutions / £100 per day for all other participants. In the event of cancellation by the delegate a full refund of the course fee is available up to two weeks prior to the course. NO refunds are available after this date. If it is no longer possible to run a course due to circumstances beyond its control, NCRM reserves the right to cancel the course at its sole discretion at any time prior to the event. In this event every effort will be made to reschedule the course. If this is not possible or the new date is inconvenient a full refund of the course fee will be given. NCRM shall not be liable for any costs, losses or expenses that may be incurred as a result of its cancellation of a course, including but not limited to any travel or accommodation costs. The University of Southampton’s Online Store T&Cs also continue to apply.
Website and registration:
Region:
Greater London
Keywords:
Survey Research, Survey sampling , Probability sampling methods, Non-probability sampling
Related publications and presentations:
Survey Research
Survey sampling
Probability sampling methods
Non-probability sampling