Creating CHOIS: A Recovery-Oriented PROMIS-Based Measure for Mental Health Screening and Outcomes

Keywords

Mental Health; Screening; Recovery.

Introducation

Early screening and detection of mental health disorders has a wide array of benefits for the 1 in 4 adults and 1 in 5 children who will have a mental illness within a given year, and the over 46% of people who will have a mental illness within their lifetime, especially when only 36% of adults with a diagnosable and treatable mental illness are currently getting treatment, according to statistics from the National Institute of Mental Health [1,2]. It is commonly accepted that by detecting problems earlier, we may be able to prevent them from getting worse. Untreated mental health problems can lead to hospitalization, prison, and other poor patient-centered outcomes. They can interfere with one’s ability to work, parent, maintain relationships, and enjoy a good quality of life. Sometimes, the consequences can be dire, resulting in suicide or harm to others [3,4].

Untreated mental health conditions not only cause unnecessary suffering for people who live with them, but also can have negative consequences on people’s families, workplaces, and communities. The consequences of undiagnosed or untreated mental illnesses include both an individual’s decreased quality of life as well as financial consequences exceeding 300 billion dollars per year annually [2]. Furthermore, untreated mental health and substance abuse conditions have a substantial impact in the workplace, costing employers up to 100 billion dollars annually [5]. For example, depression is related to work impairment, disability, increased absenteeism and reduced productivity on the job [6,7].

Untreated mental illness in parents has serious impacts on children and adolescents. For example, mothers who suffer from depression have been shown to influence the mental and emotional health of their children [8]. Furthermore, untreated mental disorders among younger people can interfere with critical emotional, cognitive, and physical development [9] that can predict future problems such as drug and alcohol abuse, violence, risky sexual behavior, failure in school, suicide, unemployment, underemployment, poor physical health, and high rates of utilization of social welfare and the criminal justice system [10,11].

Depression screening instruments are an effective and efficient way of detecting risk of depression in an individual and have become increasingly more important [12,13]. The importance of mental health screening procedures should not be trivialized, because they help medical practitioners make important decisions about treatment options and decisions [14]. One mental health screening measure, the Patient Health Questionnaire (PHQ), is a very popular and valid measure of screening for depression that is widely used in both clinical and research settings. While the PHQ-9 is one of the most widely used screeners for depression [15], it is not as commonly used in screening for mental health problems.

Use of mental health screening in the military has shown to provide substantial benefit, particularly in regard to military deployment. A recent study by Warner and colleagues [16] compared three brigades screened for mental health disorders prior to deployment in Iraq to three unscreened brigades. Individuals who screened positive for a mental health condition were then further assessed by mental health providers. Of the 10,678 soldiers screened, 819 received further mental health evaluation with 74 not cleared to deploy and 94 deployed under specific restrictions or conditions (restricted duties, medication, counseling etc.). After six months, soldiers from the screened brigades showed significantly lower rates of clinical contact for combat stress, psychiatric disorders, and occupational impairment as well reduced rates for the need of air evacuation for behavioral health reasons. According to Warner et al. [16], mental health screenings in the military were associated with a reduction of negative soldier outcomes, including suicidal ideations. Furthermore, the National Alliance on Mental Illness indicates that with implementation of mental health screening measures, youth suicide will be reduced [9]. It has been suggested widely that improved mental health screening might address and prevent some of the mental health problems implicated as contributing to some mass murders [17]. Therefore, mental health screenings can play a significant role in saving lives, in addition to improving quality of life.

Mental health screening is important in many other places outside traditional mental health care settings such as education and the military and good measures will contribute to those efforts at improving mental health promotion. The screening instrument is a critical component of integrating mental health care into other settings, because mental health issues don’t come with the usual blood tests and physiological measurements to help diagnose while gauging severity and recovery progress. As with many physical illnesses, important help often comes from the physician discovering and discussing a problem, rather than patient-initiated conversation about a developing health problem. Once the topic is broached, screening instrument results can help the patient understand the diagnosis and make the best decisions for their own care in partnership with their clinician. Doctors may feel more comfortable broaching the topic of mental health and patients may feel less stigmatized, even those who are already getting treatment. Normalizing the discussion of mental health through screening programs attempts to create a culture where everyone is more comfortable talking about mental health, including issues that might not be assessed on the screening. Providing normative data to help patients understand their results provides another vehicle of language for physicians to explain mental health diagnosis and treatment in language that patients can understand. Putting issues in patients’ own language is easier when they have completed questionnaires. In sensitive areas such as alcohol abuse or suicidal thoughts, the screening questionnaire can provide an easier way for the clinician to discuss the topic with their client, as well as providing a foundation for motivational interviewing and discussing treatment options.

Additionally, mental health screening provides substantial benefits to an individual’s overall physical health. Individuals experiencing psychological distress are not only more likely to become smokers and maintain a poor diet, but they are also more likely to develop hypertension, obesity, asthma, and diabetes (diabetes mortality) [18-21]. Moreover, individuals with a depressive disorder are nearly twice as likely to suffer from coronary artery disease or suffer from a stroke, and are four times more likely to experience a myocardial infarction and to die within 6 months than individuals without depression [20]. Likewise, individuals suffering from chronic or debilitating conditions such as diabetes, asthma and cancer are significantly more likely to suffer from a mental health condition, especially anxiety or depression [20]. However, studies have shown that when mental health conditions are screened for and addressed, individuals with such conditions show better rates of recovery, fewer emergency room visits, and improved rates of medication adherence [20]. According to a 2006 American Psychiatric Association report about Integrating Medical and Behavioral Health Care [22], there is a high comorbidity of depression and chronic medical disorders that has led to the development of depression management programs that integrate mental health services into primary care settings.

Although there are a variety of mental health instruments available, it was very difficult for new Prevention and Early Intervention programs in San Diego County to find an instrument that met all of the stakeholders needs for screening and outcomes measurement for settings that were not primarily focused on mental health such as primary care. The ideal instrument needed to meet all of the following criteria: comprehensive mental health assessment, accurate and sensitive to change, useful in many different settings, minimal cost, minimal burden to staff and consumers, recovery orientation, and cross-cultural. It should be appropriate across all ages from adolescents to older adults, available in multiple languages, available on both paper and computer, and provide meaningful reports to clinicians and consumers.

There are several existing screening tools designed for a variety of purposes and budgets. Other screening measures used by San Diego County programs include My Mood Monitor (M3), Brief Symptom Inventory (BSI), Patient Health Questionnaire (PHQ), Patient-Reported Outcomes Measurement Information System (PROMIS), Illness Management and Recovery Scales (IMR), Recovery Markers Questionnaire (RMQ), and the Global Appraisal of Individual Needs (GAIN). While all of these instruments met several of the criteria and some are being used successfully by some programs, none completely fulfilled the needs of a new Prevention and Early Intervention program that places additional mental health services into Alcohol and Drug Treatment Programs (ADTP) with a focus on mental health screening for participants and family members in order to provide referrals or other extra help.

The PROMIS measures a few important domains of mental health, but does not yet measure some other important psychiatric symptoms, nor does it include recovery factors and strengths such as social connectedness, self-esteem, and coping ability. San Diego County Behavioral Health has a strong recovery orientation that even drives choices of measurement instruments, so it was another important consideration that the instrument not be simply a symptom or problem inventory. Staff and clients in the system called for a section of the CHOIS that came to be called “recovery factors”, making the CHOIS a recovery-oriented screening measure for mental health. The presence of these positive strength-related items that were mostly generated through client focus groups may also reduce feelings of stigma about admitting to mental health issues on the questionnaire.

The cost of healthcare is always a critical issue under consideration. While individuals are seldom charged for mental health screening in any setting, the prospect of mass screening by healthcare or other institutions does bring concern about additional costs to the system. These costs may come from charges for using the instrument, staff time, or other additional charges to the healthcare funder that eventually filter back in the form of higher insurance premiums, taxes, or reductions in other services. The goal of CHOIS is to minimize burden to staff and respondents by making it short and automated, as well as free so that more healthcare resources can be saved for treatment. The purpose of the present study is to demonstrate the psychometric validity of the CHOIS instrument, and make it freely available to the healthcare community.

Methods

Participants

Participants in this study were 3,724 people getting mental health screenings through a Mental Health Services Act Prevention and Early Intervention project that added mental health counselors to 13 Alcohol and Drug Services (ADS) treatment programs to screen for mental health issues and help clients who exhibit mental health concerns. Only the first assessment was used for each individual who had multiple assessments, and 12 questionnaires that were less than 70% completed were removed, leaving 3,712 completed instruments for the analysis.The participants had a mean age of 30.8 and 55% were male.

Measures

The CHOIS was created through a partnership between academic clinical psychologists with expertise in measure development, health outcomes experts, county mental health administrators, mental health consumers, and frontline clinicians such as psychiatrists and counselors. Most of the instrument is made up of the PROMIS short scales for Depression, Anxiety, and Anger, thus deriving some validity from previous validation of those scales which have published internal reliabilities of .98, .98, and .97 [23]. Other items come from expert opinion by psychologists studying the DSM definitions and coming up with applicable items that were then reviewed and reworded by the different stakeholders listed above. A set of positive recovery factors was also created with input from the groups.There are some pairs of items that can indicate response inconsistency (hopeful vs. hopeless and happy vs. sad) to detect suspicious response patterns. Response options for the CHOIS items are on a five-point Likert scale of Never (0),Rarely (1),Sometimes (2),Often (3),and Always (4).People were also asked “How difficult have any problems reported here made it for you to do your daily activities, work (including school), take care of things at home, or get along with other people?” and whether they would like to talk to someone about mental health problems for themselves or someone else.

Analysis
Internal reliability of the scale was calculated as Cronbach’s alpha. Item fit on scales was examined with item-total correlations and by recalculating Cronbach’s alpha with each item removed from its scale. Pearson correlations were used to test concurrent validity by examining the correlation between the CHOIS scales and thequestions about how much difficulty was caused by mental health problems and whether the respondent wanted to discuss mental health issues with someone.

Results

As shown in Table 1, most of the scales showed very good internal reliability as demonstrated by Cronbach’s alpha ranging from 0.87 to 0.93 for all the scales that had more than two items. The “Cronbach’s alpha with item removed” column shows that no item fit poorly on its respective scale, in that alpha was not significantly increased by removing any item. All scales were significantly correlated to functioning difficulties and to wanting help for themselves or a family member for mental health problems, as shown in Table 1.The mean of all 20 symptom items was strongly related to the item assessing functioning difficulties caused by the mental health issues, r =0.58, p < .0005.

The scale means would indicate that there is significant mental distress in this population getting treatment in San Diego ADS programs. This is also indicated by 53.7% reporting at least some functioning limitations from mental health problems other than substance abuse, and 30% wanting to speak with someone about mental health issues at the time of screening.

×

Table1:

Close

Table 1: Item and Symptom Scale Characteristics of the CHOIS Measure

Discussion

The CHOIS instrument demonstrated adequate psychometric validity for most scales. The reliability was lower for the two twoitem scales in part because the authors attempted to capture some breadth of the construct in all scales, and this differentially affects reliability of smaller scales. While these two item scales may be less effective psychometrically, they capturekey unique features that may indicate serious mental illness such as psychosis. Many items should be considered important in isolation, such as suicidality or hallucinations. The scale scores were significantly related to functioning difficulties caused by the mental health issues. The entire scale showed the highest correlation, along with the depression and anxiety scales.This paper also demonstrated that there was a significant need for mental health services in this ADS population. Many people said they wanted help with mental health issues, and CHOIS scores indicated that people involved in ADS experienced more mental distress than the general population. This suggests there is a great need for integrating mental health services with ADS in order to address concurrent, or even causal, problems that interfere with recovery.

Participants of the current study were already being assessed for substance use by other instruments, given they were in ADS, and the PROMIS alcohol/substance scales were not yet available.The authors recommend using the PROMIS alcohol and substance abuse items with the CHOIS when they become available, or some other measure of substance abuse. It is another limitation of this studyis that the population was all ADS clients, and more validation work is needed in other settings. Further validation work is needed to establish its sensitivity to changes in symptoms and to further refine the items and factors.

The CHOIS is a useful tool for allowing valid and comprehensive use of the still-developing PROMIS system in a variety of settings as a mental health screening and outcomes instrument. It works seamlessly as a supplement to the entire PROMIS outcomes system that measures other important health constructs, and creates a more recovery-oriented focus for healthcare systems, even in non-psychiatric settings. The items are available in a number of languages. The authors hope that the healthcare community finds the CHOIS to be a useful tool in their efforts to improve mental health and quality of life for all people.

Acknowledgements

The authors acknowledge San Diego County people receiving services, clinicians, and other staff of alcohol and substance abuse programs for providing input on the measure and Marshall Lewis, Ted Ganiats, Erik Groessl, Steven Tally, Bill Sieber, and Emily Trask for input on question wording. We thank Alma Correa for her help in managing the data, Kimberly Center and Elizabeth Miles for reviewing the article, and Kyle Choi and Neil Yetz for assistance with reference articles.

References

1. Healthy People 2020. Mental health across the life stages. (2013) www.healthypeople.gov/2020/LHI/mentalHealth.aspx?tab=determinants
2. National Institute of Mental Health (NIMH). The numbers count: Mental disorders in America. (2013) www.nimh.nih.gov/health/publications/the-numbers-count-mental-disorders-in-america/index.shtml.
3. Rockville (2001) U.S. Surgeon General Mental Health: A Report of the Surgeon General, Government Printing Office.
4. Neufeld E, Perlman CM, Hirdes JP (2012) Predicting inpatient aggression using the Inter RAI risk of harm to others clinical assessment protocol: A tool for risk assessment and care planning. The Journal of Behavioral Health Services & Research 39(4): 472-480.
5. Partnership for Workplace Mental Health. An employer’s guide to workplace substance abuse: Strategies and treatment recommendations. (2014) www.workplacementalhealth.org/Business-Case/An-Employers-Guide-to-Workplace-Substance-Abuse-Strategies-and-Treatment-Recommendations-.aspx
6. Elinson L, Houck P, Marcus SC, Pincus HA (2004) Depression and the ability to work. Psychiatric Services 55(1):29-34.
7. Jain G, Roy A, Harikrishnan V, Yu S, Dabbous O, et al. (2013) Patientreported depression measured by the PHQ-9 and impact on work productivity: Results from a survey of full-time employees in the United States. Journal of occupational and environmental medicine 55(3):252-258.
8. Campbell SB, Morgan-Lopez AA, Cox MJ, McLoyd VC (2009) A latent class analysis of maternal depressive symptoms over 12 years and offspring adjustment in adolescence. Journal of Abnormal Psychology 118(3):479-493.
9. National Alliance of Mental Illness (NAMI). Achieving the promise: Transforming mental health care in America. (2014) www.nami.org/Content/NavigationMenu/Inform_Yourself/About_Public_Policy/New_Freedom_Commission/Goal_4_Early_Mental_Health_Screening.htm.
10. Columbia University Teen Screen Program: Mental Health Check-ups for Youth (2003). Mental illness and suicide in youth: Why screening is needed. (2014) www.nami.org/Content/ContentGroups/CAAC/Teenscreen-Brochure.pdf.
11. Wickrama T,Wickrama KA (2010) Heterogeneity in adolescent depressive symptom trajectories: Implication for young adults’ risky lifestyle. Journal of Adolescent Health 47(4):407-413. dx.doi.org/10.1016/j.jadohealth. 2010.02.013.
12. Kroenke K, Spitzer RL, Williams JB (2001) The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine 16(9):606-613.
13. Picardi A, Adler AD, Chang H, LegaI,GigantescoA,et al. (2012) Development and preliminary validation of the PC-SAD5, a screener-derived short depression severity measure. Journal of the European Academy of Dermatology Venereology 26:185-171.
14. U.S. Preventive Services Task Force (2009) Screening for Depression in adults: U.S. preventive services task force recommendation statement. Annals of Internal Medicine 151(11): 784-792.
15. Substance Abuse and Mental Health Services Administration (SAMHSA). Screening tools. (2014) www.integration.samhsa.gov/clinical-practice/screening-tools
16. Warner CH, Appenzeller GN, Parker JR, Warner CM, Hoge CW (2011) Effectiveness of mental health screening and coordination of In-theater care prior to deployment to Iraq: A cohort study. The American Journal of Psychology 168(4):378-385.
17. Brent DA, Miller MJ, Loeber R, Mulvey EP, Birmaher B (2013) Ending the silence on gun violence. Journal of the American Academy of Child & Adolescent Psychiatry 52(4): 333-338.
18. Brown AF, Ettner SL, Piette J, Weinberger M, Gregg E, et al. (2004) Socioeconomic position and health among persons with diabetes mellitus: A conceptual framework and review of the literature. Epidemiologic Reviews 26:63-77.
19. Brumby S, Chandrasekara A, McCoombe S, Kremer P, Lewandowski P (2012) Cardiovascular risk factors and psychological distress in Australian farming communities. Australian Journal of Rural Health 20:131-137.
20. Sederer LI, Silver L, McVeigh KH, Levy J (2006) Integrating care for medical and mental illnesses. Preventing Chronic Disease 3(2):1-3. www.cdc.gov/pcd/issues/2006/apr/05_0214.htm
21. Saydah SH, Imperatore G, Beckles GL (2013) Socioeconomic status and mortality: Contribution of health care access and psychological distress among U.S adults with diagnosed diabetes. Diabetes Care 36:49-55.
22. Azocar F, Ciemins E, Kelleher D, (2006) Behavioral health outreach: Integrating medical and behavioral health care.Psychiatric Services 57(12):1807-1808.
23. Cella D, Riley W, Stone A, Rothrock N, Reeve B, et al. (2010) The Patient- Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology 63(11):1179-94.