Conference on Experiments with Crowd Sourced Subjects
Date:
14/02/2013 - 14/03/2013
Organised by:
Nuffield Centre for Experimental Social Sciences, University of Oxford
Presenter:
Professor Ray Duch
Level:
Entry (no or almost no prior knowledge)
Contact:
Description:
Social science experiments using crowd sources subjects is flourishing, as crowd sourcing services provide variety of methods to implement experiments at smaller budget and shorter time. This one-day workshop will introduce the development and use of crowed sourced experiments to researchers who are interested in conducting such experiments in their fields. A stronger focus is placed on Amazon Mechanical Turk (AMT), one of the most common tools for such experiments.
The materials covered in the workshop consist of three components. First, participants will learn the general overview of crowd sourcing experiments, especially ones using AMT. Although there are obvious advantages in AMT experiments, such as low cost in recruiting subjects, smaller budgets to collect data with a large sample, and easiness to pay for subjects, there are some pitfalls inherent in crowd sourcing. Participants will learn these advantages and disadvantages; most importantly they will learn how to avoid these disadvantages. We also provide a few alternatives to AMT experiments.
Second, the workshop will include presentations of research projects that highlight the possibility of crowd-sourced experiments – these will include state-of-the-art techniques in experiments with AMT and Facebook.
The workshop will include a demonstration and instructions of AMT experiments. We will provide the practical materials about how to implement the AMT experiments: how to start up AMT as requesters, how to create the tasks for subjects, how to access the data, and also what programming environments are available for more complicated AMT experiments.
Registration is required for the conference. There is limited capacity for this workshop and priority registration will be accorded students and faculty from CESS affiliated departments at University of Oxford, (Department of Politics and International Relations, Department of Economics, and Nuffield College).
Programme
09:00-10:30 Opening Remarks (Ray Duch, Centre for Experimental Social Sciences, Nuffield College
Overview: Gabriele Paolacci (Erasmus University Rotterdam), “Handle with Care: Opportunities and Threats of Amazon Mechanical Turk Experimentation”
10:30-10:45 Coffee Break
10:45-12:15 Research Using Crowd Sourced Subjects:
Gabriel S. Lenz (University of California, Berkeley), “Substituting the End for the Whole: Why Voters Respond Primarily to the Election-Year Economy”
Justin Grimmer (Stanford University), “The Impression of Influence: How Words and Money Cultivate a Personal Vote”
12:15-13:30 Lunch Break
13:00-14:30 Practical Aspects of Designing and Running Crowed Sourced Experiments:
Gabriele Paolacci, How to Implement Amazon Mechanical Turk Experiments, Part I
14:30-14:45 Coffee Break
14:45-16:15 Gabriele Paolacci, How to Implement Amazon Mechanical Turk Experiments, Part II
16:15-17:00 Closing Remarks (Ray Duch, Centre for Experimental Social Sciences, Nuffield College)
Cost:
Free of charge
Website and registration:
http://cess-wb.nuff.ox.ac.uk/seminars.php
Region:
South East
Keywords:
Experimental Research , crowed sourced experiments , Amazon Mechanical Turk
Related publications and presentations: