Why is having a research protocol important in qualitative research?

Protocol for a qualitative study exploring perspectives on the INternational CLassification of Diseases (11th revision); Using lived experience to improve mental health Diagnosis in NHS England: INCLUDE study

1. Corinna Hackmann1,
2. Amanda Green1,
3. Caitlin Notley2,
4. Amorette Perkins1,
5. Geoffrey M Reed3,
6. Joseph Ridler1,
7. Jon Wilson1,2,
8. Tom Shakespeare2

1. 1 Department of Research and Development, Norfolk and Suffolk NHS Foundation Trust, Hellesdon Hospital, Norwich, UK
2. 2 Department of Clinical Psychology, Norwich Medical School, University of East Anglia, Norwich, UK
3. 3 Department of Psychiatry, Global Mental Health Program, Columbia University Medical Centre, New York, New York, USA

1. Correspondence to Dr Corinna Hackmann; Corinna.hackmann{at}nsft.nhs.uk

Abstract

Introduction Developed in dialogue with WHO, this research aims to incorporate lived experience and views in the refinement of the International Classification of Diseases Mental and Behavioural Disorders 11th Revision (ICD-11). The validity and clinical utility of psychiatric diagnostic systems has been questioned by both service users and clinicians, as not all aspects reflect their lived experience or are user friendly. This is critical as evidence suggests that diagnosis can impact service user experience, identity, service use and outcomes. Feedback and recommendations from service users and clinicians should help minimise the potential for unintended negative consequences and improve the accuracy, validity and clinical utility of the ICD-11.

Methods and analysis The name INCLUDE reflects the value of expertise by experience as all aspects of the proposed study are co-produced. Feedback on the planned criteria for the ICD-11 will be sought through focus groups with service users and clinicians. The data from these groups will be coded and inductively analysed using a thematic analysis approach. Findings from this will be used to form the basis of co-produced recommendations for the ICD-11. Two service user focus groups will be conducted for each of these diagnoses: Personality Disorder, Bipolar I Disorder, Schizophrenia, Depressive Disorder and Generalised Anxiety Disorder. There will be four focus groups with clinicians (psychiatrists, general practitioners and clinical psychologists).

Ethics and dissemination This study has received ethical approval from the Coventry and Warwickshire HRA Research Ethics Committee (16/WM/0479). The output for the project will be recommendations that reflect the views and experiences of experts by experience (service users and clinicians). The findings will be disseminated via conferences and peer-reviewed publications. As the ICD is an international tool, the aim is for the methodology to be internationally disseminated for replication by other groups.

Trial registration number ClinicalTrials.gov: NCT03131505.

• International Classification of Diseases
• ICD-11
• Personality Disorders
• Anxiety Disorders

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• International Classification of Diseases
• ICD-11
• Personality Disorders
• Anxiety Disorders

Strengths and limitations of this study

• This study is the first to gather expert by experience views on the proposed criteria to be fed into the revision process of the International Classification of Diseases.

• All aspects of the proposed study have been co-produced with experts by experience and agreed with a representative from WHO.

• Qualitative focus group data will be thematically analysed to form the basis of co-produced recommendations to be fed back to WHO.

• The themes and resulting recommendations will be limited to five diagnostic categories and will only reflect views from the UK.

Introduction

Diagnostic systems have a number of functions both from the perspective of the clinician and service user.1–3 Diagnosis offers indications for treatment, may guide expectation regarding prognosis and can help people to make sense of their experiences of living with mental health (MH) difficulties.1 2 In order for a diagnostic system to be useful, it is critical that it reflects the day-to-day experiences of people living with the symptoms. Service users have reported relief derived from diagnostic definitions that resonate with and explain their experiences.1 4 On the other hand, some feel their diagnosis does not ‘fit’ with or describe their experiences, and thus has limited utility other than being a ‘tick box’ exercise of labelling and categorising.5–7 To date, it appears that no revision of the major systems for psychiatric diagnosis (International Classification of Diseases (ICD) or Diagnostic and Statistical Manual of Mental Disorders) has sought feedback from service users prior to publication.

Diagnostic systems are designed for clinicians; despite this, service users can easily access the diagnostic criteria. Research shows that the labels, language and descriptions used in these systems can impact people’s self-perception, their interpretations of how other people view them and their understanding of the implications of having a diagnosis, including the prognosis and potential for recovery.5 8 9 These interpretations can have a direct impact on factors such as self-worth and self-stigmatisation, social and occupational functioning, recovery and service use.5 6 10 For example, service users have reported that terms like ‘disorder’ and ‘enduring’ suggest permanency, impeding their hope for recovery.5 Similarly, others have reported that the descriptions and terms used in diagnostic systems (eg, language like ‘deviant’, ‘incompetent’, ‘disregard for social obligations’ and ‘limited capacity’) can be stigmatising and unhelpful, leading to feelings of rejection, anger and possible avoidance of services.5 6 8 Clarity on the perceptions of individuals receiving a diagnosis, in terms of the language, meaning and implications of what is included in the system, may help to minimise possible negative consequences.

Evidence suggests that clinicians also have concerns regarding the validity and clinical utility of the current diagnostic systems.3 11–13 For instance, health professionals have reported that some diagnostic definitions feel arbitrary, artificial or unreflective of the typical presentations they observe in practice.11 12 Other evidence suggests that clinicians find the categories difficult to use, particularly for distinguishing between disorders.9 12 14 Clinicians have also expressed reservations regarding the terminology and associated stigma, particularly for conditions such as Schizophrenia and Personality Disorder.13 15 These findings are from studies that have been conducted after the criteria have been released. Prospective input from clinicians on the proposed criteria as part of the process of revision may therefore improve the validity and clinical utility of diagnostic systems.

The value of expertise by experience is increasingly recognised by policymakers,16–18 service providers and researchers.19 20 Many have argued that processes of diagnosis could be improved by including perspectives of those with lived experience.10 21 It has been suggested that within the diagnostic categories, "the traditional language is useful for listing and sorting but not for living and experiencing. ‘Naming' a thing is not the same as 'knowing'a thing" (p90)22 and therefore categories could be improved by viewing service users as ‘authors of knowledge from whom others have something to learn’ (p291).21 Likewise, it has been argued that diagnostic systems could be improved by addressing problems identified by practising clinicians.3

Input regarding the proposed content for the ICD-11 from service users and clinicians should be used to support the process of revision and improvement. Feedback and clarity from service users on (1) whether the content of the system is in line with their experience of symptoms and (2) their interpretations of the content and language should facilitate the development of a system that is more accurate and valid, with minimised unintended negative impact.

Aims and objectives

This research project will use a focus group methodology to ask service users and clinicians who use the ICD diagnostic tool (psychiatrists and general practitioners) their views on the proposed content for the ICD-11. Data collected through collaborative discussion in the groups will be inductively analysed, and resulting themes will be triangulated with an advisory group (involving additional service users and clinicians). The output will be recommendations for improvement to ICD-11 content that have been co-produced with a feedback group (of different service users and clinicians).

Research questions

1. What are the views and perspectives of service users and clinicians on the content of the ICD-11?

2. How could the system be improved for the benefit of service users and clinicians?

Methods and analysis

Study design

This is a qualitative study. Data will be collected through focus groups. Focus groups are an appropriate method of data collection to answer the study research questions seeking to explore views and perspectives of service users and clinicians, where our analysis will aim to define key themes and points of consensus or divergence gathered through interaction,23 24 drawing on participants own perspectives and choice of language.25 Participants will be given a copy of the proposed diagnostic criteria relevant to their diagnosis to discuss in the group. This will include both the technical version (as it is proposed for the ICD-11) and a lay translation of the criteria. Thematic analysis26–28 will be used to identify emergent recurring and/or salient themes in the focus group data. The themes will form the basis for co-produced recommendations to support the development of the ICD-11. Data collection for this study commenced in February 2017 and analyses are planned to be completed and fed back to WHO by the end of December 2017.

Co-production

The research team that developed this project includes a service user expert by experience (AG), two academics (TS, CN), two research clinicians (a consultant psychiatrist (JW) and a clinical psychologist (CH)) and two research assistant psychologists (AP, JR). A service user expert by experience research team member will be involved in all aspects of the research, including design, facilitating focus groups, analysis, write-up and dissemination.

In developing the project, team members consulted a local service user governor, service users and the service user involvement leads at the hosting National Health Service organisation. This input helped shape the design (changing and broadening the process of recruitment of service users and supporting the use of focus groups) and the initial selection of the diagnoses that were included.

Co-production with service users, clinicians and researchers will continue throughout the project. Data analysis will be co-produced through involvement of the service user expert by experience on the research team and the advisory and feedback groups.

Diagnoses under investigation

With agreement from WHO, five diagnoses have been selected for exploration: Personality Disorder, Bipolar I Disorder, Schizophrenia, Depressive Disorder and Generalised Anxiety Disorder. These diagnoses include a wide range of symptom phenomena. Personality Disorder, Bipolar I Disorder and Schizophrenia are found to be more stigmatised, rejected and negatively viewed than other diagnoses, meaning they may have a particularly negative impact and be more consistently associated with harm.29 30 Depressive Disorder and Generalised Anxiety Disorder are highly prevalent, making the largest contribution to the burden of disease in middle-income and high-income countries, including the UK.31

Lay translation

The lay translations of the criteria have been produced by members of the research team including psychiatrists and other clinicians, and approved by a representative from WHO to ensure they reflect the proposed ICD-11. Documents have been created presenting lay translations alongside the technical version as it is written in the ICD-11, so that participants are easily able to refer to either source. Copies of these are available in English for researchers wishing to replicate this study.

Recruitment

Sampling will be purposive and include a number of pathways to ensure maximum inclusivity. Recruitment of service users will be both via clinicians in a MH trust and self-referral via a number of routes. Promotion of the study will be via clinicians, service user involvement leads in a MH trust and non-governmental organisations (NGOs). Clinicians in the MH trust will be asked to identify potential participants and seek consent to be contacted by the research team. Service user involvement leads will disseminate information about the study to service users and the membership of a MH trust (which includes many previous service users), providing a telephone number and email address to self-refer if interested. NGOs will promote the study using the same materials. The study will be promoted through recruitment posters, service user involvement forums and on social media. Clinicians will be recruited via team leaders, word of mouth and email communications promoting the project.

Once self-referral or consent to contact has been established, a member of the research team will make contact, provide potential participants with a brief overview of the study, and answer any questions. If the individual wishes to be involved in the study, they will be sent a copy of the participant information sheet via post or email. This information sheet outlines the purpose and nature of the study, and the ethical safeguards regarding data protection and privacy. Potential participants will have at least 72 hours to consider whether they would like to be involved in the study. If the individual would like to take part in the study, researchers will arrange to meet them at least 1 week before the focus group to complete the consent process and give them the relevant proposed diagnostic criteria to read and consider.

Sample size

There will be two service user focus groups for each of the five diagnoses. Additionally, there will be four clinician focus groups. The ICD system is primarily used by medical doctors in the UK, although clinical psychologists have been included in this study as they also apply the system in their work.32 In this study, the diagnostic criteria presented to participants are divided into distinct discussion points. During the focus groups, these discussion points will be addressed one by one and participants will be asked for their feedback through predefined questions and prompts. This includes asking people their views of the proposed features, the language used, the positives and negatives of what is included and how the classification might be improved for the benefit of service users. In light of this, the number of groups was agreed based on research stating that using more standardised interviews decreases variability and thus requires fewer focus groups.33 In total, there will be 14 groups, containing three to six participants each. This will give a total sample of 42–84 participants (30–60 service users and 12–24 clinicians). The advisory group will comprise three to five additional service users and three clinicians. Lastly, the feedback group will comprise five service users and three clinicians. The focus group size was chosen to allow participants opportunity to discuss their views and experiences in detail, while increasing recruitment feasibility.34 The sample size should be sufficient in providing data to meet the aims and to cover a range of views. Evidence suggests that the majority of themes are discovered in the first two to three focus groups.35

Inclusion and exclusion criteria

Adult service users (18 years and older) may be included in the focus groups if they have formally received at least one of the five diagnoses under investigation and have accessed services within the last 5 years (including those currently in receipt of services). People with multiple diagnoses may only take part in one focus group, but will be given the option of which group. Clinicians will have had experience working in MH, including the use of the psychiatric diagnoses under investigation. Individuals may only participate in either one focus group, the advisory group or the feedback group.

Individuals will be excluded if they are under the age of 18 years, lack the capacity to consent, or have an inability to speak fluent English (as fluent English is required to participate in the focus groups). Individuals will also be excluded if their participation is deemed unsafe to themselves or others by their lead clinician or clinicians on the research team.

Data collection

Focus groups are the most applicable method for data collection to meet our research aims, as attitudes, opinions and beliefs are more likely to be revealed in the reflective process facilitated by the social interaction that a focus group entails than by other methods.23–25 Additionally, focus groups have proved to be a useful way of exploring stigma issues in MH,36 and service users are often familiar with group settings for discussing MH issues.

The summary of the new diagnostic guidelines and lay translation will enable participants to reflect on both the content and the language of the proposed criteria. During the groups, topic guides will encourage participants to discuss and share views of the relevant diagnostic category. This includes their overarching views, thoughts and feelings; as well as, specific reflections on areas such as the language used, aspects that may be helpful or unhelpful, and suggestions for improvement.

Each focus group will be led by an experienced and trained member of the research team and have an assistant facilitator. Service user focus groups will last 60–90 min, and clinician focus groups will last 2–2.5 hours to account for the discussion of multiple diagnoses.

Analysis

The focus groups will be audio-recorded and transcribed verbatim. The transcripts will first be read and descriptively openly coded (using the same language as participants where possible) by the lead researcher. Approximately 25% of the transcripts will be independently open coded by another member of the research team, as a validity check. Codes will be compared and discussed until consensus is reached. The five diagnoses will initially be analysed separately to produce themes that are relevant to each diagnosis. Following this, these themes will be compared with identify common themes relevant to all the diagnostic categories. Analysis of data will mainly be descriptive. We will take a critical realist epistemological stance to analysis, recognising that there are multiple individual realities, but taking a pragmatic approach to analysing data at face value, drawing on the perspectives of individuals as they choose to represent themselves through discussion.37 Thematic analysis will be used to inductively code themes that reoccur or appear important.26–28 The concept of salience will be referred to here, to guide coding that is conceptually and inherently significant, not just frequently occurring. A qualitative data management software system (NVIVO-11) will be used to facilitate data analysis.

In addition to descriptive data for thematic coding, focus groups generate data that is conversational. Analysis of this requires an inductive approach that focuses on instances in the data where there is marked agreement (consensus), disagreement or divergence. These instances will be identified as ‘critical moments’. The sample size is small and purposive. Consequently, summary quantified coding matrices will not be produced. Instead there will be a focus on the 'critical moments' to direct the analysis and eventual findings, reporting on the issues that are of central importance to the participants.

Following analysis of each focus group, a second stage analysis will be conducted to compare and contrast findings across groups. The analysis will seek out consensus, disagreement and inconsistency within service user and clinician focus groups, and between diagnoses. This second stage analysis will involve discussions within the research team to refine the themes and to develop higher level themes, that is, grouping the open codes into meaningful conceptual categories. This will allow tentative conclusions to be drawn about aspects of the diagnostic criteria which may be particularly pertinent for some groups and less important for others. It will also enable conclusions to be drawn regarding generic language or overall responses to the diagnostic criteria, in comparison to more nuanced reactions to diagnostically specific categories.

The output from the analysis will be higher level themes and categories that form the basis of recommendations for the ICD-11. These themes will be triangulated with the advisory group. The resulting themes will be discussed with the feedback group in order to co-produce the recommendations. These recommendations will be contextualised with a description of the themes and identified areas of agreement and disagreement for feedback to WHO.

Data protection

All confidential data will be kept for 5 years on password-protected computers and/or locked filing cabinets only accessible to members of the research team. During transcription, audio-recordings will be anonymised, with all identifiable information removed prior to using the software analysis tool. All audio-recordings will be destroyed immediately after transcription.

Ethics and dissemination

Ethical considerations

Written informed consent to participate and be audio-recorded will be obtained from all participants. Data management and storage will be subject to the UK Data Protection Act 1998. Ethical approval for the current study was obtained from the Coventry and Warwickshire Research Ethics Committee (Rec Ref: 16/WM/0479).

Declaration of Helsinki

This study complies with the Declaration of Helsinki, adopted by the 18th World Medical Association (WMA) General Assembly, Helsinki, Finland, June 1964 and last revised by the 64th WMA General Assembly, Fortaleza, Brazil, October (2013).

Output and dissemination

This research has been designed to obtain feedback with recommendations for the ICD-11, and to develop a methodology that can be replicated in other countries that use the ICD system. Additionally, the findings, and learning in terms of the process of co-producing and conducting research with experts by experience, will be disseminated via peer-reviewed publications, conferences, media and lay reports.

Discussion

Service user involvement in MH is a priority.19 Studies have found that both clinicians and service users have questioned the accuracy, validity and clinical utility of the ICD and other psychiatric diagnostic tools.3 8 9 11 12 38 Despite this, to date, service user and clinician feedback has not been obtained prior to revision of the ICD manual. In light of this, is not clear whether the content resonates with the experiences of people giving and receiving the diagnoses, could lack clinical utility, or even, cause harm (eg, in terms of the language used).

Limitations

This study is designed to input feedback from service users and clinicians in the forthcoming revision of the ICD. The usefulness of the data and resulting recommendations is dependent on input, that is, reflective of the views of service users and clinicians that the new system will impact. The current study will include two focus groups for each disorder in an attempt to minimise bias35 and to account for group-think processes that may occur within individual groups. Taking a critical realist epistemological stance is a pragmatic approach to work with discursive data created through the interactional context of a focus group. It is acknowledged that there are multiple competing realities and perspectives that may differ across time and context, and the analysis findings will be limited to the time and context of this study. Transferability of findings is nonetheless maximised by triangulation to ensure the inclusion of multiple stakeholder perspectives, enabled by the advisory and feedback groups of experts by experience that will co-produce the recommendations reported to WHO. Interpretation of the feedback will take into account potential limitations regarding the generalisability of the findings. The current project is exploring only five of the diagnoses that are included in the ICD-11. The ICD is internationally used, and the current project will reflect the experiences and views of service users and clinicians in the UK only. Future research may include both additional diagnostic categories and encapsulate expertise by experience and relevant clinicians in different countries.

Conclusion

The current study will use feedback from experts by experience to co-produce recommendations for the revised diagnostic system proposed for the ICD-11. This feedback aims to improve the accuracy, validity and clinical utility of the manual, and minimise the potential for unintended negative consequences. This qualitative approach has not been previously employed by any countries that use the ICD system. Our vision is that this process will become a routine feature in future revisions of all diagnostic systems.

Acknowledgments

We would like to thank the library services at Norfolk and Suffolk Foundation Trust for aiding the searching and retrieval of documents. We would like to thank Kevin James (service user governor), Lesley Drew and Sharon Picken (service user involvement leads) for their input during the development of the project. We would also really like to thank Dr Bonnie Teague who generously offered the benefit of her wisdom and proof reading skills.

References

1. 1.↵
2. 2.↵
3. 3.↵
4. 4.↵
5. 5.↵
6. 6.↵
7. 7.↵
8. 8.↵
9. 9.↵
10. 10.↵
11. 11.↵
12. 12.↵
13. 13.↵
14. 14.↵
15. 15.↵
16. 16.↵

    Department of Health. Putting People First: Planning together – peer support and self-directed support. London: Department of Health, 2010.

17. 17.↵

    Department of Health. No Health without Mental Health: A cross-government mental health outcomes strategy for people of all ages. London: Department of Health, 2011.

18. 18.↵

    Department of Health. Closing the Gap: Priorities for essential change in mental health. London: Social Care, Local Government and Care Partnership Directorate, 2014.

19. 19.↵
20. 20.↵
21. 21.↵
22. 22.↵
23. 23.↵
24. 24.↵
25. 25.↵
26. 26.↵
27. 27.↵
28. 28.↵
29. 29.↵
30. 30.↵
31. 31.↵

    World Health Organization. The global burden of disease: 2004 update. Switzerland: World Health Organization, 2008.

32. 32.↵
33. 33.↵
34. 34.↵
35. 35.↵
36. 36.↵
37. 37.↵
38. 38.↵

Read the full text or download the PDF:

Log in using your username and password

Why is having a research protocol and a research script important in qualitative research?

Why is a research protocol important?

What is protocol in qualitative analysis?