Introduction to Evaluation Research

Description:

Format

5-day intensive seminar; 9am-5pm with one hour for lunch from 12:30-1:30pm; approximately 8 hours per day split into two 4-hour sessions; Day 1 will consist of introductory material for those with no background on evaluation; Days 2-5 will consist of more advanced material on specific subjects; participants should sign up separately for each session.

What will be covered?

DAY 1 – Introduction

Session 1: Introduction to evaluation and evaluation design (Auty)

This session will introduce the basic theory, purpose, and politics of evaluation in the social sciences. It will also include a practical dialogue about the role of evaluation in academic settings. It will provide an introduction to the various analytical frames that can be used in evaluation research. As evaluation is fundamentally a process of valuing, different frames will inevitably generate different ways of thinking about the problem of what works, when, how, and for whom. This session focuses on the issue of the task of integrating knowledge from competing policy frames as a major issue for democratic governance.

Session 2: Core evaluative techniques (Auty)

This session will introduce and describe in some detail several mainstream and non-mainstream approaches to evaluation. Each of these approaches provides a different ways of seeing/valuing. The different approaches to evaluation frame will be disused with reference to four key assumptions; assumptions about methodology, ontology and epistemology, assumptions about the complexity of problems, assumptions regarding decisional contexts, institutional settings and policy processes and assumptions about welfare.

DAY 2 – Quantitative Analysis

Session 3: Cost-benefit analysis (Hooton)

A review of the theoretical and practical issues surrounding the quantification of evaluations using the Cost-Benefit Analysis technique; the session will review the theoretical frameworks for developing CBA studies and the various measurement issues surrounding operationalization of a study including (but not limited to) leakage, control selections, measuring additionality, the use of revealed versus stated preference valuations, and more.  Additionally the session will both review and conduct an in-class exercise on important financial and economic techniques for conducting CBA studies including the selection of appropriate valuation models, the incorporation of inflation and time discounting, and actual calculations for CBA.  Finally, the session will have students develop a CBA study for a practical situation to illustrate important real-world considerations.

Session 4: Financial and budget evaluation (Hooton)

A review of basic financial and accounting principles and theory with specific application to the evaluation of project and program pro-formas; for example, how to evaluate elements of a proposed drug prevention program and put together a financial assessment

DAY 3 – Data collection techniques

Session 5: Presenting evaluation results (Auty)

This session will review effective methods and styles for presenting evaluation research results in papers and presentations including expected reporting procedures; the session will discuss how evaluation results can and should be interpreted and how they may become open to attack; it will also review how to state assumptions used in evaluation

Session 6: Stated preferences: survey techniques for evaluation (Auty)

A review of basic survey theory and design, but then with a specific extension on how to design survey-based studies for evaluation research; for example, how does a researcher design a study investigating the benefits of living near a park using surveys of neighborhoods

DAY 4 – Spatial and Geographic Analysis

Session 7: Spatial evaluation techniques (Hooton)

A review of the theory and techniques for conducting evaluation research in a spatial context; the session will take lessons learned in previous sessions and apply them in a spatial setting with a discussion on the unique measurement issues that arise in spatial evaluations such as spatial autocorrelation, the treatment of geography, and more.  Additionally, the lectures will explain the importance of differencing techniques and methods for control/counterfactual selection.  The session will also introduce students to spatial data sources and spatial data analysis software systems to allow further independent study.  The session will finish with practical exercise in which students attempt to design a spatial evaluation.

Session 8: Econometrics for evaluation (Hooton)

A review of the specific regression models most commonly used in evaluation research including means of differences, difference-in-difference, and dynamic causal effects analysis; would be an extension of other methods courses offered by SSRMC

DAY 5 – Other topics in evaluation

Session 9: Propensity Score Matching approaches to evaluation (Auty & Jolliffe)

The session will introduce how a propensity score matching (PSM) approach can be used to evaluate the impact of interventions on program participants. PSM is a technique whereby a number of covariates are used to model the probability of participating in an intervention for the participants and the matched non-participants. PSM attempts to reduce the bias due to confounding variables that could be found in an estimate of the treatment effect obtained from simply comparing outcomes among units that received the treatment versus to those that did not.

Session 10: Feasibility Studies (Auty & Jolliffe)

This final session will provide an overview of how to conduct an appropriate feasibility study. Feasibility studies can provide essential information and aid decision making. Their central aim is to uncover the strengths and weaknesses of a proposed study, and also to judge the cost required and value to be attained. This session will detail the stages and procedures of conducting a feasibility study and how to select the appropriate data and models for use in a feasibility study.

Cost:

£

Website and registration:

http://www.ssrmc.group.cam.ac.uk/advanced-training/evaluation-research

Region:

East of England

Keywords:

Research Skills, Communication and Dissemination, Research Skills, Communication and Dissemination (other), Cambridge

Related publications and presentations:

Research Skills, Communication and Dissemination
Research Skills, Communication and Dissemination (other)

Back to archive...