The phrase LLM in the centre of an illustration of a human brain.Should qualitative researchers be using generative artificial intelligence (GenAI)? Recently, more than 400 internationally renowned qualitative researchers co-signed a commentary rejecting GenAI for reflexive qualitative research. The oppositional commentary has been submitted to a major qualitative research journal, calling on the journal and its readers to consider their position on the issue. Yet other qualitative researchers argue that AI, such as large language models (LLMs), is part of the academic and intellectual ecology, so the point is how qualitative researchers engage with it rather than disengagement. The topic then, is one of hot debate.
Reasons for opposing the use of AI
Primary reasons put forward by qualitative researchers who are opposed to the use of GenAI for qualitative research are that GenAI only provides an illusion of meaning making and is devoid of understanding. As such, it runs the risk of replicating the loudest, dominant voices and marginalising others. Qualitative research and explanation, they state, are about making connections with people and social processes through the intellectual and bodily humanity of the researcher. The commentary also draws attention to ethical concerns about the extractive, exploitative and colonialist practices of the major tech companies developing and profiting from GenAI and its supporting infrastructures, harming the environment and peoples.
Critically engaging with LLMs
These are, indeed, issues that qualitative researchers need to reflect upon critically. But others argue the use of large language models for analysis of qualitative data is a ship that has sailed and is docking in its destination port. So, rather than calling for a ban, we need to bring our critical faculties to bear in a different way.
At a recent online seminar considering qualitative research and LLMs – hosted by NCRM, QUEST, the South Coast Doctoral Training Partnership and the Work Futures Research Centre – Susan Halford (University of Bristol) and Les Carr (University of Southampton) argued that the critical engagement with use of LLMs required was at the level of the politics of social research methods.
This is a reframing of the discussion away from the prevalent focus on opportunities and risks for qualitative research, making comparisons between human and LLM performance, and developing guidelines for the inevitability of a future of GenAI use. Rather, we should be reflecting analytically on the fundamentals, such as ways of knowing and the knowledge economy. That is, the methods that qualitative (and indeed all) researchers use to generate data are not mere reflections of society but turn complex and fluid worlds into bounded static versions, frozen in a particular form in the specific analysis. The methods applied construct the knowledge object. They validate one way of knowing – a way of knowing that aligns with institutional power, over another way.
On the face of it, this point has resonance with an objection put forward by the oppositional commentary, that of oblivion to marginalised understandings. But it calls for a critical engagement that shapes the future use of LLMs, making the epistemological implications clear, and the powerful interests involved transparent.
Addressing time-saving claims
Susan and Les also focused on the “pragmatics” of their own project work with LLMs and a massive qualitative data set, unpacking the idea that GenAI is somehow a quick and efficient means of analysing large amounts of qualitative data. The sheer amount of time involved in preparing big qual data so that it is suitable for inputting into the programme, and is organised in a way that will meet research aims, needs to be brought into the equation (as I and colleagues can wholeheartedly attest). This seemingly small point in itself is a foundational challenge to how GenAI gets talked about and an example of the politics of methods.
Sarah Jenner’s (University of Southampton) following presentation also talked about using LLMs for analysis. She presented her evaluation of the strengths and drawbacks in a story completion study she designed, comparing various models. Again, the time-consuming management work involved in using LLMs was apparent, in terms of prompt engineering for the models and asking for explanation of the process used. Sarah pointed out that sceptical qualitative researchers were not obliged to use GenAI. How long this will remain the case, with research funders coming to expect or even require it, is a question however.
Interaction between LLMs and qualitative values
In the final presentation in the seminar , Marianne Aubin Le Quéré (Princeton University) and Casey Randazzo (University of California, Santa Barbara) discussed their collective work, exploring the epistemological issues raised in their interviews with qualitative researchers from different disciplinary backgrounds.
The central theme that Marianne and Casey addressed was: how do LLMs as a tool interact with qualitative values? This is a question that the oppositional commentary raises and answers negatively. In contrast to the body of researchers who signed the oppositional commentary, however, it was complexity and tensions that were evident among the qualitative researchers. They spoke about the blurred lines and relationship between LLMs’ external outputs and the researchers own qualitive research values and approaches, such as helpfulness as a “brainstorming buddy” as against the issue of bearing witness to participant stories.
Marianne and Casey pointed out that the permeation and ubiquity of LLMs means their presence is becoming less obvious. This is not helping towards transparency and enabling qualitative researchers to think critically about the politics of methods.
Looking ahead
As a final reflection in the vein of critical engagement, in the discussion following the seminar presentations, one point raised was the need for a different vocabulary to refer to the application of LLM processes, rather than unthinkingly continuing with the words that have long been used for what it is that qualitative researchers do. Not least, we need different terms from “intelligence”, “interpretation” and “analysis” to refer to the processing of data through GenAI models.
Personally, I can envisage scenarios where I can – and with colleagues have – worked with LLMs, as an initial entry point to a large qualitative data set that provides me with a thematic overview for a deep dive into topics that may throw light on my research aims. But getting to grips with meaning and insights through in-depth analytic engagement is one of the creative skills and joys of qualitative research that I do not wish to let go.