Anita Chandra: NCRM Impact Prize entry

The title of this application was Improving questionnaire design skills to increase accuracy of measuring the impact of Mind’s services.

It was submitted by Anita Chandra, of the mental health charity Mind, who explained that NCRM's training helped her organisation to improve the quality of data it gathers about the impact of its services.

Summary of NCRM participation

I attended the Questionnaire Design for Mixed-Mode, Web and Mobile Web Surveys research training in February 2023. One of the main reasons I chose to attend this training was because there’s a great deal of surveys and questionnaires which people run at Mind and one of the functions of my team is to review them and provide support and guidance. I wanted to increase my knowledge in this area so I could pass this knowledge onto my colleagues.

One of the other reasons I chose to attend this training was because I was developing a research training programme to rollout across Mind to help staff become better consumers and producers of research. I wanted to incorporate some of the best practices when it comes to questionnaire design and distil this to the whole organisation.

Impact achieved

Sharing knowledge with colleagues

Following my attendance of this research training, I shared my key takeaways from the session with my immediate team colleagues in the form of a shared learning session. In this session, I included things such as why we should avoid agree/disagree Likert scale (which is a common question style we tend to use at Mind), the cognitive stages people go through when answering survey questions and why ‘select all that apply’ questions aren’t effective. The training really made me question some of the advice and support we offer as a team, and it was very helpful to understand the evidence behind some of these things.

My key takeaways session made my colleagues think about how they design surveys and the question types they tend to use and to carefully think about whether the question type is the best one for the nature of the questionnaire. For example, in our standard demographic monitoring form, we ask a question which asks people to describe their experience of mental health problems and it’s in a ‘select all that apply’ format. As a result of this shared learning session, as a team we agreed that we would change the format of this question so that it’s a yes/no question because it’s easier to answer but also easier for us and the organisation to report on.

Enhancing training

In addition to the above, I also incorporated some of the key takeaways into a training module I was developing for the organisation on how to design surveys. I included really helpful resources such as The Nesstar UK Data Service so staff across Mind could look at archived questionnaires. This was something no one at Mind knew of prior to me attending this course. I’ve since rolled out my newly designed training at Mind on how to design surveys and through the feedback we’ve received, staff have let us know how helpful it’s been, particularly around understanding the pros and cons of different question types and the little things to think about when designing question e.g., setting an adequate time frame.

Since the launch of my training module, we’ve trained 30 people directly and indirectly my team have supported several teams across with their surveys and questionnaires. I’ve seen a real difference in the standard of survey and questionnaires that people have been drafting across Mind through reviewing them and my team have also shared the difference this new knowledge has had on their ability to provide support to teams.

Collecting better quality data

Ultimately, improvements in the standards of surveys and questionnaires across Mind has meant better quality data is being collected which is helping Mind show the impact its services is having on people with mental health problems.