You are here:

Supporting materials

An Introduction to Factorial Survey Experiments- Part 1: Introduction
Download slides
Download transcript

An Introduction to Factorial Survey Experiments- Part 2: Setting Up the Experiment and the Survey
Download slides
Download transcript

Related datasets, collections and other links


Recommended reading

  • Jasso, G. (2006) ‘Factorial survey methods for studying beliefs and judgments’, Sociological Methods and Research, 34(3), pp. 334–423.
  • Mutz, D. C. (2011) Population-Based Survey Experiments. Princeton: Princeton University Press.
  • Rossi, P. H. (1979) ‘Vignette Analysis: Uncovering the Normative Structure of Complex Judgments’, in Merton, R. K., Coleman, J. S., and Rossi, P.H. (eds) Qualitative and Quantitative Social Research: Papers in Honor of Paul F. Lazarsfeld. New York: Free Press, pp. 176–186.
  • Wallander, L. (2009) ‘25 years of factorial surveys in sociology: A review’, Social Science Research, 38(3), pp. 505–520.
  • Wallander, L. (2012) ‘Measuring social workers’ judgements: Why and how to use the factorial survey approach in the study of professional judgements’, Journal of Social Work, 12(4), pp. 364–384.
  • Sampling/Sets
  • Atzmüller, C. and Steiner, P. M. (2010) ‘Experimental vignette studies in survey research’, Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 6(3), pp. 128–138.
  • Dülmer, H. (2007) ‘Experimental plans in factorial surveys: Random or quota design?’, Sociological Methods & Research, 35(3), pp. 382–409.
  • Dülmer, H. (2016) ‘The Factorial Survey: Design Selection and its Impact on Reliability and Internal Validity’, Sociological Methods and Research, 45(2), pp. 304–347.
  • Kuhfeld, W. F. (2010) ‘Experimental Design: Efficiency, Coding, and Choice Designs’, in Marketing Research Methods in SAS. Cary: SAS Institute, pp. 53–241.
  • Kuhfeld, W. F., Tobias, R. D. and Garratt, M. (1994) ‘Efficient Experimental Design with Marketing Research Applications’, Journal of Marketing Research, 31(4), pp. 545–557.
  • Other methodological research (e.g., presentation format, number of dimensions and vignettes, order effects)
  • Auspurg, K. et al. (2015) ‘The Factorial Survey as a Method for Measuring Sensitive Issues’, in Engel, U. et al. (eds) Improving Survey Methods: Lessons from Recent Research. New York: Routledge, pp. 137–149.
  • Auspurg, K., Hinz, T. and Walzenbach, S. (2019) ‘Are Factorial Survey Experiments Prone to Survey Mode Effects?’, in Lavrakas, P. J. et al. (eds) Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment. Hoboken: John Wiley & Sons, Inc., pp. 371–392.
  • Auspurg, K. and Jäckle, A. (2017) ‘First Equals Most Important? Order Effects in Vignette-Based Measurement’, Sociological Methods and Research, 46(3), pp. 490–539.
  • Düval, S. and Hinz, T. (2020) ‘Different Order, Different Results? The Effects of Dimension Order in Factorial Survey Experiments’, Field Methods, 32(1), pp. 23–37.
  • Gutfleisch, T., Samuel, R. and Sacchi, S. (2021) ‘The application of factorial surveys to study recruiters’ hiring intentions: comparing designs based on hypothetical and real vacancies’, Quality and Quantity, 55, pp. 775–804.
  • Sauer, C. et al. (2011) ‘The application of factorial surveys in general population samples: The effects of respondent age and education on response times and response consistency’, Survey Research Methods, 5(3), pp. 89–102.
  • Sauer, C., Auspurg, K. and Hinz, T. (2020) ‘Designing multi-factorial survey experiments: Effects of presentation style (text or table), answering scales, and vignette order’, Methods, Data, Analyses, 14(1), pp. 61–90.
  • Shamon, H., Dülmer, H. and Giza, A. (2019) ‘The factorial survey: The impact of the presentation format of vignettes on answer behavior and processing time’, Sociological Methods & Research. doi: https://doi-org.proxy.bnl.lu/10.1177/0049124119852382.
  • Weinberg, J., Freese, J. and McElhattan, D. (2014) ‘Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample’, Sociological Science, 1, pp. 292–310.
  • Validation studies regarding the external validity of (factorial) survey experiments
  • Diehl, C. et al. (2013) ‘Not In my kitchen? Ethnic discrimination and discrimination intentions in shared housing among university students in Germany’, Journal of Ethnic and Migration Studies, 39(10), pp. 1679–1697.
  • Eifler, S. (2007) ‘Evaluating the validity of self-reported deviant behavior using vignette analyses’, Quality and Quantity, 41(2), pp. 303–318.
  • Eifler, S. (2010) ‘Validity of a factorial survey approach to the analysis of criminal behavior’, Methodology, 6(3), pp. 139–146.
  • Hainmueller, J., Hangartner, D. and Yamamoto, T. (2015) ‘Validating vignette and conjoint survey experiments against real-world behavior’, Proceedings of the National Academy of Sciences, 112(8), pp. 2395–2400.
  • Pager, D. and Quillian, L. (2005) ‘Walking the talk? What employers say versus what they do’, American Sociological Review, 70(3), pp. 355–380.
  • Pedulla, D. S. (2016) ‘Penalized or protected? Gender and the consequences of nonstandard and mismatched employment histories’, American Sociological Review, 81(2), pp. 262–289.
  • Petzold, K. and Wolbring, T. (2019) ‘What can we learn from factorial surveys about human behavior? A validation study comparing field and survey experiments on discrimination’, Methodology, 15(1), pp. 19–28.
  • Wulff, J. N. and Villadsen, A. R. (2019) ‘Are Survey Experiments as Valid as Field Experiments in Management Research? An Empirical Comparison Using the Case of Ethnic Employment Discrimination’, European Management Review, 17(1). doi: 10.1111/emre.12342.