Open, Reproducible and Transparent Social Science

Description:

Open science involves making scientific methods, data, and outcomes transparent to everyone. It includes making as transparent and available as possible (1) steps taken in data collection, processing and analysis that lead to the production of results, (2) study plans, data, materials and associated processing methods and (3) the results generated by the research. Reproducible science involves the potential for others to recreate reported results by repeating the original data processing and analyses with the original data.

Transparent and reproducible science results from open science workflows that allow you to easily share work and collaborate with others as well as openly publish your data and workflows to contribute to greater scientific knowledge. Facilitating openness, reproducibility and transparency in social science is important as it advances collaboration, scientific progress, trust in science, and the reusability of research. It also touches on ethical questions, for example in navigating between the creation of science as a public good and the protection of research subjects. 

This course introduces the principles and practical steps of doing cutting-edge open, reproducible and transparent social science research. Participants will learn how to conduct research that is easy to check and understand by providing easy-to-use access to methods and data. They will also learn how to conduct reproducible research the results of which can be easily recreated using the original data and steps in data processing and analysis.

The course covers an introduction to principles and practices of transparent and reproducible social science research:

• Motivations behind and principles of open, reproducible and transparent research
• Pre-analysis plan / pre-registration / Registered Report
• Multiverse analysis, sensitivity analysis, specification curve analysis
• Meta-analysis / systematic evidence synthesis
• Transparent reporting standards and disclosure
• Replication
• Data management and data sharing
• Script/code sharing and version control (e.g., using Github)
• Reproducible workflows (e.g., using R Markdown)
• Open access (preprints and publisher models)

By the end of the course participants will:

• understand the philosophical and meta-research-based rationale for doing open, reproducible and transparent social science research;
• have conceptual and practical knowledge of the main building blocks of open, reproducible and transparent social science research:
  • transparent study planning,
  • assessment of the robustness of research results,
  • sharing of research outputs to enable easy reproduction and replication.

This course is aimed at Social science researchers of all backgrounds, disciplines and levels (junior and senior) who undertake data analysis (quantitative and qualitative).  It is essential that participants possess at least a beginner level of familiarity with R. Some basic understanding of regression modelling is also recommended.

R and RStudio will be installed on all the desktop computers available in the teaching room. However, if you bring your own laptop, we recommend installing the R and RStudio in advance. You may also want to get an account on GitHub and download a desktop version of GitHub.

Preparatory Reading

The following references provide a useful reading list covering the methods that we will see in this course. They are listed in order of relevance:

Christensen, G.S., Freese, J. and Miguel, E. 2019. Transparent and reproducible social science research: How to do open science. Oakland, CA: University of California Press. https://doi.org/10.2307/j.ctvpb3xkg  

Moody, J.W., Keister, L.A. and Ramos, M.C. 2022. Reproducibility in the social sciences. Annual Review of Sociology. 48(1), pp.65–85.

Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K.M., Gerber, A., Glennerster, R., Green, D.P., Humphreys, M., Imbens, G., Laitin, D., Madon, T., Nelson, L., Nosek, B.A., Petersen, M., Sedlmayr, R., Simmons, J.P., Simonsohn, U. and Laan, M.V. der 2014. Promoting transparency in social science research. Science. 343(6166), pp.30–31.

Freese, J., Rauf, T. and Voelkel, J.G. 2022. Advances in transparency and reproducibility in the social sciences. Social Science Research. 107, Article 102770.

Further recommended readings (in alphabetical order):

Allen, C. and Mehler, D.M.A. 2019. Open science challenges, benefits and tips in early career and beyond. PLOS Biology. 17(5), Article e3000246.

Banks, G.C., Field, J.G., Oswald, F.L., O’Boyle, E.H., Landis, R.S., Rupp, D.E. and Rogelberg, S.G. 2019. Answers to 18 questions about open science practices. Journal of Business and Psychology. 34(3), pp.257–270.

Crüwell, S., van Doorn, J., Etz, A., Makel, M.C., Moshontz, H., Niebaum, J.C., Orben, A., Parsons, S. and Schulte-Mecklenbeck, M. 2019. Seven easy steps to open science: An annotated reading list. Zeitschrift für Psychologie. 227(4), pp.237–248.

Fecher, B. and Friesike, S. 2014. Open Science: One term, five schools of thought In: S. Bartling and S. Friesike, eds. Opening science: The evolving guide on how the internet is changing research, collaboration and scholarly publishing [Online]. Cham, Switzerland: Springer, pp.17–47. Available from: https://doi.org/10.1007/978-3-319-00026-8_2.

Figueiredo Filho, D., Lins, R., Domingos, A., Janz, N., Silva, L., Figueiredo Filho, D., Lins, R., Domingos, A., Janz, N. and Silva, L. 2019. Seven reasons why: A user’s guide to transparency and reproducibility. Brazilian Political Science Review. 13(2), Article e0001.

Kathawalla, U.-K., Silverstein, P. and Syed, M. 2021. Easing into open science: A guide for graduate students and their advisors. Collabra: Psychology. 7(1), Article 18684.

Klein, O., Hardwicke, T.E., Aust, F., Breuer, J., Danielsson, H., Mohr, A.H., Ijzerman, H., Nilsonne, G., Vanpaemel, W. and Frank, M.C. 2018. A practical guide for transparency in psychological science. Collabra: Psychology. 4(1).

Masuzzo, P. and Martens, L. 2017. Do you speak open science? Resources and tips to learn the language. PeerJ Preprints. https://doi.org/10.7287/peerj.preprints.2689v1

McKiernan, E.C., Bourne, P.E., Brown, C.T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B.A., Ram, K., Soderberg, C.K., Spies, J.R., Thaney, K., Updegrove, A., Woo, K.H. and Yarkoni, T. 2016. How open science helps researchers succeed. eLife. 5, Article e16800.

Parsons, S. et al. 2022. A community-sourced glossary of open scholarship terms. Nature Human Behaviour. 6(3), pp.312–318.

Programme

Day 1: 10:00-17:00 

Session 1 (Morning): Introduction to Open Science

- Welcome and workshop overview

- What is open science, and why is it important for social scientists?

- Principles of open, reproducible, and transparent research

Session 2 (Late Morning): Transparent research planning and reporting

- Pre-analysis plans, pre-registration and Registered Reports

- Transparent reporting standards and disclosure

Lunch Break

Session 3 (Afternoon): Assessing Research Robustness 

- Multiverse analysis, sensitivity analysis, specification curve analysis

- Meta-analysis / systematic evidence synthesis

Session 4 (Late Afternoon): Data Management and Sharing

- Data management best practices

- Data sharing and repositories

- Ethical considerations in data sharing and privacy

Day 2: 09:00-16:00

Session 1 (Morning): Creating Reproducible Research Workflows

- Reproducible workflows with R Markdown

Session 2 (Late Morning): Opening Reproducible Research Workflows

- Script/code sharing and version control (e.g., using Github)

Lunch Break

Session 3 (Afternoon): Replication

- Intro to replication in the social sciences

- Best practices for social science replications in research and teaching

Session 4 (Late Afternoon): Open Access and Closing

- Open access publishing models

- How to publish in open-access journals

- Summary and key takeaways

- Q&A and closing remarks

Throughout the course students will have the opportunity to discuss questions and issues that arise from their own research.

Cost:

The fee per teaching day is: • £35 per day for students registered at University. • £70 per day for staff at academic institutions, Research Councils researchers, public sector staff and staff at registered charity organisations and recognised research institutions. • £250 per day for all other participants All fees include event materials and morning and afternoon refreshments. Fees do not include travel and accommodation costs. In the event of cancellation by the delegate a full refund of the course fee is available up to two weeks prior to the course. NO refunds are available after this date. If it is no longer possible to run a course due to circumstances beyond its control, NCRM reserves the right to cancel the course at its sole discretion at any time prior to the event. In this event every effort will be made to reschedule the course. If this is not possible or the new date is inconvenient a full refund of the course fee will be given. NCRM shall not be liable for any costs, losses or expenses that may be incurred as a result of its cancellation of a course, including but not limited to any travel or accommodation costs. The University of Southampton’s Online Store T&Cs also continue to apply.

Website and registration:

https://go.soton.ac.uk/fnv

Region:

South East

Keywords:

Quantitative Approaches (other), Quality in Quantitative Research, Frameworks for Research and Research Designs (other), Data Management, Quantitative Approaches (other), Research and Project Management

Related publications and presentations:

Quantitative Approaches (other)

Back to archive...