/*==============================================================================
NCRM Online learning resource - An Introduction to Factorial Survey 
Experiments (FSEs)

Part III: Data Analysis - Practical examples
Date: 25 August 2021
Author of this syntax: Tamara Gutfleisch


Data source: 
The data used for this course is from the EDYPOLU recruiter survey: 
https://doi.org/10.17605/OSF.IO/ZTB6Y. The data is licensed under a Creative 
Commons Attribution-NonCommercial 4.0 International License (to view a 
copy of this license, visit https://creativecommons.org/licenses/by-nc/4.0/). 

Data documentation and codebook:
Gutfleisch, T., & R. Samuel (2021). EDYPOLU Recruiter Survey. Scientific Use File. 
Data Documentation. https://doi.org/10.17605/OSF.IO/7BP5E 

Description of experimental design: 
The EDYPOLU recruiter survey aimed at examining the impact of certain applicant characteristics on recruiters' hiring intentions. The vignette universe included 
36 vignettes (full factorial). The 36 vignettes were divided into 6 vignette sets, 
each including 6 vignettes using a D-efficent blocking technique. The respondents 
were randomly assigned to one deckl (mixed design). Three experimental variables 
were used (gender, unemployment, and nationality). Respondents were identified before data collection, and the experimental set up data included "personalised" vignette 
sets for each respondent. 

Relevant variables: 
Respondent_ID - Respondent ID
sector - Occupational field
vigid - Vignette ID (ranges from 1 to 36)
vigset - Vignette set, individual vignette sets (i.e., with randomised order)
deck - initial vignette sets (ranging from 1 to 6)
vignr - Vignette position
vigeval_1 - Vignette evaluations
vgndr - Vignette gender
vuetim - Vignette unemployment
vnat - Vignette nationality


Focus of this course: 
- Check data quality after data collection
- Analysing and interpreting the results

More practical examples that cannot be covered by this short introductory course
can be found in the supplementary material to the book by Auspurg and Hinz (2015). 
This supplementary material includes example code of how to construct the vignette
sample in SAS, examples of how to prepare the set up data and how to merge the 
set up and respondent data, and more examples of how to analyse the data.
______________________________
Auspurg, K. and Hinz, T. (2015) Factorial Survey Experiments. Los Angeles: Sage. 
Supplementary files: 
https://study.sagepub.com/auspurg_hinz/student-resources/ancillary-materials
==============================================================================*/


capture log close
log using "..\Part_III_results.log", replace // save code and results in .txt file
clear all

set scheme lean1 // set overall look of graphs 


* @0 Load the data for analysis
use "EDYPOLU_survey_SUF.dta", clear 
/* 
The data already combines the experimental set up data and the respondent data. 
*/


* @1 Select sample

* Keep only one occupational field
keep if sector==4 // keep data from the catering sector

/*
The EDYPOLU recruiter survey covered five occupational fields, which can be 
considered separate experiments. For the following demonstrations, I therefore
focus on just one occupational field. The analyses, however, could also be done
in the full data set, but you would need to control for the occupational fields
in the regression analysis. 
*/


* @2 Check characteristics of vignette experiment


** Check whether each vignette/vignette set has been evaluated
tab deck // frequency of initial vignette sets
tab vigid // frequency of vignettes


** Check whether randomsiation of vignette order has worked

// Check whether vignette position correlates with vignette ID or vignette sets
pwcorr vignr vigid, star(0.05) 
pwcorr deck vignr, star(0.05)


** Frequencies of levels of each dimension
tab vgndr // levels of vignette gender
tab vuetim // levels of vignette unemployment
tab vnat // levels of vignette nationality


** Bivariate correlations between vignette levels 

// Generate dummies of all levels of categorical vignette dimensions
tab vuetim, gen(vuetim_) // dummies for vignette unemployment
tab vnat, gen(vnat_) // dummies for vignette nationality

// Test bivariate correlations
pwcorr vgndr vuetim_1-vuetim_3 vnat_1-vnat_6 , star(.05) // Pearson's correlation coefficient, indicating statistical significance at the 5% level

** Check distribution of vignette ratings
replace vigeval_1=. if vigeval_1==-9 // define missing values
tab vigeval_1 // show frequencies
hist vigeval_1, d // draw histogram of vignette evaluations


* @3 Data analysis

** Ordinary least squared regression with robust clustered standard errors
reg vigeval_1 i.vuetim i.vnat vgndr i.vignr, vce(cluster Respondent_ID)

** Multilevel analysis
mixed vigeval_1 i.vuetim i.vnat vgndr i.vignr || Respondent_ID: , var 
// || Respondent_ID indicates that the data is clustered at the respondent level
// var indicates that variances should be shown




********************************************************************************
log close
exit