Latest news from the National Centre for Research Methods https://www.ncrm.ac.uk/news/ Thu, 12 Mar 2026 08:03:45 +0000 en-uk https://www.ncrm.ac.uk/incoming/furniture/images/sitewide/NCRM_new_Logo.gif https://www.ncrm.ac.uk/news/ Latest news from the National Centre for Research Methods https://www.ncrm.ac.uk/news/show.php?article=5725 Social science researchers are being invited to apply for funding to refine recently developed methodological approaches. The Economic and Social Research Council (ESRC) has a pot of £2.8 million to support adaptations and innovations that were developed during the COVID-19 pandemic. ESRC hopes that the initiative will enable researchers to embed their new approaches in research practice and make them accessible across the social sciences. The scheme builds on NCRM’s ESRC-funded Changing Research Practices project, which carried out important work supporting researchers at the height of the COVID-19 crisis. In 2022, the NCRM team turned its attention to the sustainability of some of the methodological adaptations that social researchers were developing – the outcomes of necessity combined with creativity. This work included identifying methods adaptations in progress and with potential for the future. Professor Melanie Nind, Co-Director at NCRM and Changing Research Practices principal investigator, said: "Researchers who participated in our knowledge-exchange workshops and webinars will be pleased to see that the ESRC is investing further through funding work on refining methodological approaches developed since March 2020. "We need to learn from the response to COVID-19 and optimise the role that methods adaptations can play in future social science research." Find out more about the funding info@ncrm.ac.uk (Ed Grover) Thu, 25 Aug 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5725 https://www.ncrm.ac.uk/news/show.php?article=5724 Join NCRM's newly established Collaborative and Participatory Methodological Special Interest Group (MSIG) for the first event in its Critical Conversations series. Creating a space for critical conversations to emerge, this session will focus on how we practise ethics throughout the life of our research cycles (and beyond). Themes will be sourced from the attendees, but could include institutional processes, power, collaboration and co-production, remuneration, representation and tokenism. The session will take place on Wednesday, 12 October 2022, from 16:00 to 17:30. It will start with four lightening talks of approximately three minutes: Dr Isabelle Latham, of Hallmark Care Homes, will discuss how participatory forms of research offer the potential to enhance ethical practices in relation to consent and involvement of people living with dementia in research that affects their lives. Dr Niamh O’Brien, of Anglia Ruskin University, will talk about issues associated with the process and practicalities of participatory forms of research, focussing on ownership of data and interpretation, and how endings of a project can impact community members. Dr Julia Hayes, of Inclusion Creativa, will focus on the ethical considerations of research with children with disabilities within contexts of the global south, and the use of situated ethics, which take into account the context and culture in which one is researching. Dr Nicole Brown, of University College London, will discuss power dynamics between researchers and participants, arguing that it should not be researchers’ aim to hand over all responsibility and control. This session will be hosted on Zoom and guided by the MSIG organisers, and break-out rooms may be used. The sessions will not be recorded but the MSIG may write a blog to reflect on the key points of the session without identifying any individual contributors or attendees, unless permission is expressly given. The session duration is 90 minutes. A short break is scheduled and attendees are welcome to join and leave as they need to. Feedback on the format is welcomed. Instructions on how to join Zoom can be found on the Zoom website. Register for the event info@ncrm.ac.uk (Ed Grover) Thu, 25 Aug 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5724 https://www.ncrm.ac.uk/news/show.php?article=5723 The hard-to-measure Social scientists are often interested in theoretical constructs that elude efficient and reliable measurement, such as ‘beauty’ or ‘reading comprehension’. Developing instruments for hard-to-measure constructs can be expensive and labour intensive, and even then outcomes might not be robust when adapted to different contexts. Such measurement difficulties can thwart the advance of our disciplines. To overcome these difficulties there is growing interest in an approach to measurement in the social sciences that is based on the long-standing principle of comparative judgement. Here I provide a brief overview of the approach, along with examples from my own discipline of education. Comparative judgement Comparative judgement offers an efficient method to position a set of heterogenous objects on a linear measurement scale according to a high-level criterion. For example, the objects might be photographs and the criterion beauty. The measurement scale is constructed by presenting pairs of objects to participants and asking them to decide, for each pairing, which object has the ‘most’ of the given criterion. We collect the binary decisions of many such pairings from a group of participants, and then fit a statistical model to the binary decision data in order to produce a unique score for each object. This set of scores is our linear measurement scale and, like any set of scores, can be used for typical analytical processes such as hypothesis testing, regression analyses and so on. Comparative judgement readily and efficiently produces reliable measurement scales across a wide range of hard-to-measure constructs because it harnesses the principle that human beings are consistent at making relative judgements and inconsistent at making absolute judgements. A neat demonstration of this using shades of colour can be seen at the free-to-use online comparative judgement platform. In the context of education, people are inconsistent when marking essays using a rubric, but consistent when comparatively judging the same essays. Application to the social sciences In a programme of research at Loughborough University, we have applied comparative judgement to the measurement of a range of nebulous but important educational constructs. In one study, research mathematicians made pairwise judgements of A-level examination scripts from a historic archive and the resultant scores enabled us to track changes in the ‘difficulty’ of A-level mathematics qualifications over recent decades. In another study we delivered teaching interventions to two randomly-assigned groups of older primary students and used comparative judgement to determine which intervention led to a ‘better’ understanding of algebra. We have also applied comparative judgement to the measurement of students’ problem-solving skills, conceptual understanding and proof comprehension. Beyond education, we are currently collaborating with colleagues from other disciplines to apply comparative judgement to experimental philosophy, and to empirical literary studies. Find out more For those interested in learning more about how comparative judgement approaches might enable the construction of measure scales in their own research, I am delivering an NCRM online course coming up on 8 and 9 September 2022. You can also visit my online resource for getting started. Also, please get in touch for any advice and support about getting started with comparative judgement. Register for the training course, Comparative Judgement Methods info@ncrm.ac.uk (Dr Ian Jones, Loughborough University) Tue, 09 Aug 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5723 https://www.ncrm.ac.uk/news/show.php?article=5722 Most of the time, social science researchers analysing quantitative data will tend to fit statistical models which explore variation in a single outcome variable. For example, in my own work I look at educational outcomes, such as the number of qualifications a young person has attained or whether they attended university or not. Using models in the generalised linear modelling framework, such as linear regression or logistic regression models, helps us to understand variation in these outcomes, accounting for multiple explanatory variables that are associated with the outcome. For instance, we might be interested in knowing whether ethnic group differences in educational attainment persist once we have controlled for social class. The outcome variables we analyse are measures (or indicators) of underlying concepts. For example, an IQ test is a standardised test designed to measure the concept of intelligence. We cannot directly observe intelligence, so we have to use a measure which we think maps onto the concept. But what if we have several measures in our data that we believe relate to our concept? Are these indicators measuring the same thing? How consistent are the responses that people give to questions on a similar topic? Latent variable techniques attempt to answer these questions. A latent variable is one that cannot be directly observed but is estimated based on a series of observed variables (for example, perhaps using a battery of survey questions on a similar topic). Latent variable models can help us to understand the patterns of association between sets of variables. Another way of understanding these techniques is that latent variable models seek to explain the relationship between multiple correlated observed variables using a common underlying latent variable There will often be a high number of possible response patterns that people give to survey questions. Particular latent variable techniques allow us to examine whether there are groupings within the responses provided. Other latent variable techniques instead consider the latent variable as a continuous scale. Alternatively, the responses given to a set of survey questions might be recorded on a scale rather than a set of categories. How do we handle this? The short answer is that the type of statistical model we fit depends on how our variables are measured. This is true of latent variable models. Different models have been developed to account for how we understand either the latent variable or the observed variables to be expressed. Once we have an elementary understanding of one type of latent variable model, we can start to understand how to estimate and interpret other types of model. Learn more about latent variables If you are an empirical social science researcher with knowledge of statistical data analysis methods but want to learn more about latent variable models, then the following online one-day course (on 14 September 2022) will provide an overview of the different latent variable techniques and how they are related. The demonstration and practice exercises will use Stata. Familiarity with Stata is helpful but not essential for participants on this course. What will be covered? The course, Latent Variable Models for Social Research, covers: Introduction to latent variable models Comparison of factor analysis, latent trait analysis and latent class analysis Estimation and interpretation of latent class models using Stata Practical exercise: Estimation of latent variable models in Stata What can attendees expect to gain? By the end of the course participants will: Gain a clearer understanding of the concept of latent variable models Be able to select an appropriate latent variable model Estimate and interpret latent class models using Stata Be aware of potential issues during model estimation Meet the presenter Dr Chris Playford is quantitative sociologist at the University of Exeter working in the fields of social stratification and the sociology of education. His work has focused on modelling the role of family background on educational attainment with a substantive interest in inequality and disadvantage. He specialises in the secondary analysis of large-scale survey and administrative data. Dr Playford has methodological interests in a range of statistical techniques including generalised linear and mixed models, latent class analysis and multiple imputation of missing data. In a previous role he researched child development and emotional well-being. He has also published work on research reproducibility. Register for the course, Latent Variable Models for Social Research info@ncrm.ac.uk (Dr Chris Playford, University of Exeter) Tue, 09 Aug 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5722 https://www.ncrm.ac.uk/news/show.php?article=5721 How can you analyse large amounts of qualitative data in a way that retains the distinctive characteristics of rigorous qualitative research? This is a question increasingly faced by researchers working with qualitative data. They may have generated extensive quantities of interview material as part of a primary research project. Or they may be looking at sizeable sets of secondary qualitative data drawn from pooled sources deposited in data archives, or gathered from other resources such as social media. The breadth-and-depth method of analysis was developed as part of the National Centre for Research Methods Phase III research programme in order to address the big qual question. The method supports qualitative researchers in a research data context where computational processing tools can manipulate enormous amounts of data speedily. The research context may also involve requirements for the archiving of qualitative data from primary research projects for sharing and reuse as part of open science debates, as well as drives to promote data sharing more broadly. Working with big qual enables researchers to scope out research questions that allow for comparison, and for generalisability in the qualitative sense of how social processes work. The unique breadth-and-depth method can be used with any big qual material: primary or secondary, snapshot or longitudinal, and from single-sited, multiple-sited or separate multiple studies. It’s an iterative approach to managing and analysing large volumes of qualitative data that can be applied flexibly whatever the theoretical logic, substantive topic and nature of the qualitative data. The analytic method consists of four main steps that allow researchers to combine extensive coverage with intensive illumination, moving between the span of big qual analysis and the detail of qualitative engagement. If you’re searching for existing relevant secondary data for your research, you start from Step 1. If you already have large amounts of qualitative data, you start at the next step.   An archaeological metaphor A good way of conveying this movement between breadth and depth is using an archaeological metaphor. Step 1 is a breadth element of the breadth-and-depth method. It’s like an archaeological ariel survey, used to gain a broad overview of the landscape of available potential secondary data. The purpose is to identify several sources of existing material that might be relevant to your research topic and questions. Step 2 continues the breadth element, and archaeologically is like a geophysical survey. Geophysical surveying maps the patterning of landscape features and detects sub-surface areas of interest. This step uses computational text-mining approaches to identify areas of potential interest within the corpus and to point towards areas for preliminary deeper investigations. Preliminary analysis is the third step and is where we begin to move from breadth to depth. Drawing on our archaeological metaphor, this step is like digging a shallow test pit into the data to see whether there’s anything of interest or not. We don’t go into the data in great depth. The fourth step moves us fully into the depth element, from examining extracts of data to the deep excavation of whole cases.  Deep excavation is part of the archaeological process of working with big qual that heralds the ability to bring depth back into conversation with breadth. Learn to use the breadth-and-depth method As part of the Timescapes 10 Festival, in collaboration with NCRM, we’re excited to be running a flexible online, self-directed course on how to work with the breadth-and-depth method. It amounts to around seven hours’ study over a three-day period. The course is tailored so that participants can dip in and out of resources and activities, self-directing their study to fit in with work and life commitments. We will be making pre-recorded videos, activity sheets, background reading and other resources available to participants through a dedicated online portal, and live online sessions are scheduled once a day with the course tutors for tailored support. You can find articles explaining the breadth-and-depth method in the journal Quality & Quantity (2018) and (2021), and read an example of an application of the method in Sociological Research Online (2021). There is also a wide range of materials designed to help researchers think about, handle and analyse large volumes of complex qualitative data at our Big Qual Analysis Resource Hub. Register for the Timescapes 10 Festival info@ncrm.ac.uk (Professor Rosalind Edwards and Dr Susie Weller, University of Southampton, and Dr Emma Davidson and Professor Lynn Jamieson, University of Edinburgh) Wed, 27 Jul 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5721 https://www.ncrm.ac.uk/news/show.php?article=5720 There is a growing pedagogical culture in research methods education within UK universities, according to a new review paper published by NCRM researchers. The systematic review found that there has been a recent increase in the volume of published papers on how research methods are taught and learnt, particularly qualitative methods. The authors also reported that, within the latest literature, there is more theoretical discussion of practices, rationale and reflection on the teaching process. In previous years, researchers had flagged underdevelopment of a pedagogical culture and a lack of an effective body of knowledge to guide methods teachers. The new paper concludes that "clear intention to engage in dialogue and contribute to evidence-based practice and knowledge in research methods education is evident" and that "the ‘how to’ element is richly articulated and justified". The systematic review was conducted by Professor Melanie Nind, Co-Director at NCRM, who was assisted by doctoral researcher Angeliki Katramadou. Published in the British Journal of Educational Studies, the paper synthesises pedagogic approaches and strategies documented in 55 papers on doctoral and post-doctoral training published between 2014 and 2020. These papers feature a plethora of case studies and reflective accounts about teaching approach, strategy, tactics and tasks in research methods education. Consistent with previous studies, the review highlighted the predominant use of experiential, active learning and student-centred approaches, which often overlap or combine with other approaches. Read the full paper info@ncrm.ac.uk (Ed Grover) Wed, 27 Jul 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5720 https://www.ncrm.ac.uk/news/show.php?article=5719 Registration is open for the Timescapes 10 Festival, which takes place on 5-16 September 2022. The online event will be a celebration of progress and advances in qualitative longitudinal methods. Among the sessions at the festival will be an exclusive interview with Professor Ann Oakley of University College London about her life’s work and legacy. The full programme is available online. The festival is being jointly run by the Timescapes Archive at the University of Leeds and NCRM. It marks 10 years since the conclusion of the original Timescapes programme of research and aims to facilitate advances in qualitative longitudinal (QL) research, archiving and the re-use of QL data. For a nominal fee of £10, delegates will gain access to a range of informative, thematic events that are taking place across the two weeks of the festival. These include sessions with archives such as UKDA, Mass Observations and the Timescapes Archive, as well as presentations and training delivered by senior international and interdisciplinary scholars. Find out more and register for the Timescapes 10 Festival info@ncrm.ac.uk (Ed Grover) Fri, 24 Jun 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5719 https://www.ncrm.ac.uk/news/show.php?article=5718 Videos from keynote speeches at the 2021 Research Methods e-Festival are available on the NCRM website. These are: Fiction as Social Inquiry (Dr Ash Watson) Can Human Rights Survive Inequality in the Digital Age? (Amos Toh) Computational Social Science: where are we now? (Professor Noshir Contractor) Small Data and Big Data in the Waves of the Pandemic: Building the Boat as we Sailed it (Professor Trisha Greenhalgh) All five episodes of the Methods Matter podcast, which were first broadcast at the festival, are also available. A transcript and reflection piece have been published for the session To share or not to share: code sharing in social science. And a video recording of the session Teaching Visual Methods During COVID is also available. View the full programme from the 2021 Research Methods e-Festival info@ncrm.ac.uk (Ed Grover) Fri, 24 Jun 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5718 https://www.ncrm.ac.uk/news/show.php?article=5717 NCRM has published a new rapid evidence review that synthesises how social researchers have adapted or designed research methods during the COVID-19 pandemic. The authors concluded that researchers have found ways not just to get through a crisis, but to carry on over a prolonged period of disruption, better prepared for further contextual crises. The team – Professor Melanie Nind, Dr Andy Coverdale and Dr Robert Meckin – focused on methods used in 138 papers published in 2021, adding findings to those produced by their review of 64 papers in 2020. The new review states: "The evidence indicates that the impact of changes to the social world from Covid-19 on social research practices involves more than just the pivot to online methods. "It includes: adapting recruitment processes, innovation in methods, designing for flexibility and speed and for research from people’s homes, coping with different impacts on different groups and the potential to miss engagement of some groups, and strengthening relationships with stakeholders and within research teams." The review, Changing Social Research Practices in the Context of Covid-19: Updated Rapid Evidence Review – Synthesis of the 2021 Literature, was published as part of the Changing Research Practices project. In addition to the review, the NCRM team will be publishing on the project in a substantive editorial in their special issue of International Journal of Social Research Methodology and delivering an NCRM course in October 2022. Read the rapid evidence review info@ncrm.ac.uk (Ed Grover) Fri, 24 Jun 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5717 https://www.ncrm.ac.uk/news/show.php?article=5716 Applications to attend MethodsCon are now open. The free event, which takes place in Manchester on 13-14 September, will feature more than 30 sessions on methods in health and social science research. Attendees will be able to take part in workshops and interactive seminars, hear from a variety of experts and collaborate with researchers from across different sectors, including government, healthcare, the private sector, academia and the charity sector. Sessions will be in three main formats: Professional Development Workshops; Innovation Incubators; Interactive Seminars. The provisional programme for MethodsCon is available to view on the NCRM website. The final programme will be confirmed in the near future. Apply to attend MethodsCon info@ncrm.ac.uk (Ed Grover) Tue, 07 Jun 2022 00:00:00 +0100 https://www.ncrm.ac.uk/news/show.php?article=5716