English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Research has shown that people with any type of cancer frequently experience levels of psychological distress during the course of their illness, diagnosis, and treatment. The prevalence of mood disorders is around 38% (the range is from 28% to 49%) (Grassi, Caruso, Mitchell, Sabato, & Nanni, 2018; Mitchell et al., 2011). In the last decade, psychological distress has become the sixth vital sign in cancer care and, consequently, numerous international organizations and professional societies consider routine screening for it to be a standard requirement, for example the Joint Action on Cancer Control (CanCON, 2017), the International Psycho-Oncology Society (IPOS) (Watson & Bultz, 2010), and the National Comprehensive Cancer Network (NCCN, 2017). In this context, selecting an adequate assessment tool is crucial for the effectiveness of screening programs. To date, many instruments have been developed to measure emotional distress in cancer patients (Mitchell, 2010; Vodermaier, Linden & Siu, 2009). The Brief Symptom Inventory-18 (BSI-18) (Derogatis, 2001) is one of the most common checklists used in this context, not only to screen for psychological distress but also as a criterion measure to validate other measures of distress. The BSI-18 is considered a "short-tool" because it consists of only 18 items (Vodermaier et al., 2009). It was designed to assess three clinically relevant psychological symptoms (anxiety, depression, and somatization), and to provide an indicator for the overall level of psychological distress, which is known as the Global Severity Index (GSI). Furthermore, the BSI-18 includes normative data of cancer populations and allows "casesness" to be identified based on a criterion proposed by Derogatis (2001).
The psychometric properties of the BSI-18 have been widely examined in the general population (e.g., Franke et al., 2017) as well as for different subgroups: Latinos (Torres, Miller, & Moore, 2013), women (Wiesner et al., 2010), home-bound older adults (Petkus et al., 2010), drug users (Wang et al., 2010), psychiatric outpatients (Andreu et al., 2008), temporomandibular disorder patients (Durá et al., 2006), and Parkinson disease patients (Abraham, Gruber-Baldini, Harrington, & Shulman, 2017), using mainly the English version of the instrument (e.g., Franke et al., 2017; Wiesner et al., 2010) followed by the Spanish version (e.g., Andreu et al., 2008; Durá et al., 2006). Several studies have been conducted in cancer settings. Zabora et al. (2001) analysed the psychometric properties of BSI-18 in a large sample of patients who were actively undergoing cancer treatment. Principal components analysis revealed a four-factor structure; the first three factors correspond to somatization, depression, and anxiety scales (Derogatis' theoretical proposal); the fourth factor had only one significant item related to thoughts of ending one's life (item 17). Based on a sample of Spanish patients with breast cancer and using confirmatory factor analyses (CFA), Galdón et al. (2008) tested the fit of several models: a single-factor model measuring a single dimension of psychological distress, a three-factor model (the theoretical proposal), and a four-factor model (the empirical structure proposed by Derogatis (2001), based on community sample: somatization, depression, general anxiety, and panic). The results demonstrated that the improved three-factor model of the Spanish version of the BSI-18 was superior. Likewise, although the correlations between three factors reflected a possible second-order factor representing overall psychological distress, this model was not tested. Recklitis et al. conducted a series of studies among adult survivors of childhood cancer (Recklitis et al. 2006; Recklitis & Rodriguez, 2007; Merport & Recklitis, 2012). The factor structure of the BSI-18 has also been studied in a large and heterogeneous sample using both exploratory and confirmatory factor analysis (Recklitis et al. 2006). Exploratory factor analysis (EFA) supported the theoretical three-factor structure of the BSI-18. CFA was used to test the model's fit from the EFA and alternative models, including a single-factor model (general psychological distress), a four-factor model (somatization, depression, general anxiety, and panic) and hierarchical three- and four-factor models. Results showed that fit statistics for the two hierarchical models were similar. However, authors considered the hierarchical three-factor model to be preferable. Moreover, analysis of the three-factor model showed excellent reliability and consistent fit among both male and female participants; this supported the invariance of factor structure for both sexes. A subsequent study added data about the adequate convergent validity of the BSI-18 as a result of its high correlations with the corresponding Symptom Check-list-90-R subscales and highlighted the excellent internal consistency of the instrument (Recklitis & Rodriguez, 2007).
To date, the validity of evidence based on the internal structure of the BSI-18 in cancer settings has been investigated with oncology subgroups: specific cancer-sites such as breast cancer (Galdón et al., 2008) as well as during certain phases of the illness, for example when receiving medical treatment (Zabora et al., 2001), or as a survivor of childhood cancer (Recklitis et al. 2006). However, evidence of psychometric properties of the Spanish version of the BSI-18 based on heterogeneous samples of adult cancer patients are still not available, and this is crucial because its study would yield representative and generalizable estimates for the overall population. On the other hand, previous studies have used different statistical methods, for example EFA (Zabora et al., 2001) and CFA (Galdón et al., 2008; Recklitis et al. 2006), which have not always tested hierarchical structures that allow us to obtain a general score of distress.
At the same time, more advanced statistics such as factorial invariance also deserve special attention in this setting. Testing for invariance is necessary to ensure that reported mean level differences are true score differences rather than confounded by errors in measurement. When such testing is not performed, the potential for comparing groups on non-equivalent measures is increased, resulting in biased interpretations of the results (Brunet, McDonough, Hadd, Crocker, & Sabiston, 2010). About this, only the study with a sample of survivors of childhood cancer has addressed this topic; consequently, further research is necessary to properly understand the possible differences by sex in an adult cancer setting (Grassi et al., 2018).
Given this context, the goal of this study was to provide data that offers an integral vision of the psychometric properties of the BSI-18 in the adult cancer population whilst overcoming prior limitations. Specifically, the current study aims to (i) examine the factor structure of the BSI-18 in a large and heterogeneous sample of adult oncology patients to test the fit of a single-factor model as well as three- and four-factor models and their hierarchical forms; (ii) provide information on the convergent validity and the internal consistency of the instrument; and (iii) determine the factorial invariance of the BSI-18 for both men and women.
Method
Participants and Procedure
This study is part of a large research project on the detection and treatment of emotional distress in adult oncology patients at the Fundación Instituto Valenciano de Oncología (FIVO). The study was approved by the institution's Ethics Committee. Between May 2015 and July 2017, consecutive patients who visited the FIVO to receive care in any of its medical departments were invited to participate. Eligible patients were required to be over 18 years of age, be diagnosed with cancer, and be able to provide informed consent. Patients received information about the study, and those who were interested provided their informed consent and complete the questionnaire package. Of 1259 patients who were approached, 76 (6%) declined to participate. The final sample consisted of 1183 oncological patients (65% female). Participants' mean age was 57.71 years (SD=13.09; Range = 19-94 years). Most participants were living with a partner (73%) and had completed primary studies (87%). The most frequent primary cancer type was breast cancer (40%), followed by gynaecologic (15%), and prostate (12%) cancers; other localized primary cancer types included respiratory (7%), digestive (7%), melanoma (6%), and miscellaneous (9%) cancers. About 50% of participants were in a non-advanced stage of disease (stages I and II), 20% were in stage III, and 22% were in stage IV. Finally, 64% of participants were receiving medical treatment at the time of the study: surgery (10%), adjuvant treatment (47%), and hormonotherapy (7%).
Materials
Sociodemographic and medical data
A data sheet was used to obtain sociodemographic data including age, sex, marital status, education level, and employment status. Medical information including cancer type, stage of disease, phase of illness, and medical treatment at the time of the study was gathered by chart review.
Brief Symptom Inventory-18
The BSI-18 is a self-reported symptom checklist (Derogatis, 2001; Spanish version, Derogatis, 2013). Respondents rate a total of 18 items on a five-point Likert scale to reflect their distress during the previous week. The instrument provides three symptom scores: somatization (items 1,4,7,10,13,16), depression (items 2,5,8,11,14,17), and anxiety (items 3,6,9,12,15,18) as well as an overall score (GSI). Moreover, the BSI-18 allows cases of clinical distress to be identified using normative data proposed by Derogatis (2001): the scores are transformed into T-scores (T ≥ 63 in the GSI or in at least two subscales). The BSI-18 has shown adequate psychometric properties in studies of cancer patient populations (Vodermaier et al., 2009).
NCCN Problem List
The Problem List (NCCN, 2017) consists of 36 problems commonly experienced by oncology patients, which are grouped into five categories: practical (5 items: e.g. insurance/financial), family (3 items: e.g., dealing with partner), emotional (6 items: e.g., nervousness), physical (20 items: e.g., fatigue), and spiritual/religious (1 item: spiritual and religious concerns). Respondents indicate whether or not they have experienced any of the problems during the previous week. For the present study, the multi-item categories on the NCCN Problem List had adequate internal consistency.
Statistical Analysis
Data analysis was conducted in two stages. First, in the total sample, we tested the factor structure of five competitive models of the BSI-18 reported in the literature. As the assumption of multivariate normality was not fulfilled (Mardia, 1970), we tested measurement invariance of the selected model of the BSI-18 for both sexes using EQS 6.0 and robust normal theory maximum likelihood (MLR) estimation methods. To assess the goodness-of-fit for the models, we considered the Satorra-Bentler chi-squared statistic (X2) as well as other goodness-fit indexes, such as Comparative Fit Index (CFI; cutoff values close to 0.95), Root Mean Square Error of Approximation (RMSEA; cutoff values lower than 0.08), Non-Normed Fit Index (NNFI; cutoff values of 0.90), and Standardized Root Mean Square of the Residuals (SRMR; cutoff values of 0.08) as described by Hu and Bentler (1999) and Browne and Cudeck (1993). However, Hu and Bentler's (1999) guidelines are regarded as a conservative approach to the performance of fit statistics, which may vary depending on the complexity of the specified model (Beauducel & Wittman, 2005).
The evidence of multigroup invariance was based on ACFI and AX2 (Cheung & Renswold, 2002; Meade, Johnson, & Braddy (2008). Besides, partial invariance was eventually tested for some particular groups of constraints (Byrne, Shavelson, & Muthén, 1989). We estimated the configural, metric, scalar, and structural invariance of the final model of the BSI-18 for both sexes (Vandenberg & Lance, 2000). Significant latent mean differences and effect sizes were assessed using Cohen's d parameter according to Cohen (1988).
Reliability of the BSI-18 was measured using Cronbach's alpha (α) together with Composite reliability (CR) corrected by error covariances according to Brown (2015). Finally, convergent validity of the BSI-18 was assessed using a series of Pearson's correlation coefficient between the BSI-18 and the NCCN Problem List. Only two missing data were found for items 14 and 15. Pairwise deletion was used to deal with these data, which was assumed to be randomly missing.
Results
Factor structure
As shown in Table 1, five competing models were compared. All items loaded significantly on its corresponding factor. As expected, the one-factor model obtained the poorest fit to the data. Model fit for the three-factor and three-factor hierarchical models was identical as is to be expected in cases where fewer than four first-order factors are associated with a particular second-order factor (Thompson, 2004). Fit statistics for the four- and four-factor hierarchical models were very similar and slightly better than the three-factor models. In order to select the most parsimonious model consistent with the theoretical basis, the hierarchical three-factor model was eventually retained and improved, with three error covariances between items (5 & 14, 11 & 17, and 9 & 18). Post hoc model specifications were selected on the basis of modification indexes and a substantive rationale since the items belonged to the same subscale. The improved hierarchical three-factor model was retained for subsequent analysis as the best-fitting factor structure. This conclusion held for the total sample as well as for the sample groups based on sex (see Table 1).
Multigroup factorial invariance
Multisample CFA results are presented in Table 2. Once reasonable evidence of configural invariance (Model 1) across sex was achieved, equality constraints on all freely estimated first-order factor loadings were added. The assumption of full factor loading invariance (Model 2a) was untenable for sex, as shown through a significant worsening of model fit compared with the baseline model.
Thus, partial factor loading invariance was tested by relaxing the factor loading invariance assumption for items 6 and 15 from the anxiety subscale (Model 2b). Sex differences in the freely unstandardized estimated factor loadings were minor (item 6: 1.022 for women, 1.021 for men; item 15: 0.734 for women and men). Model 3b, which also allowed partial invariance in items 6 and 15 for sex, fit the data better than Models 2a and 3a, which had full invariance constraints in all factor loadings.
Subsequently, the difference between the latent factor means across the subgroups were estimated (Table 3). Model 4b obtained a good fit to the data, indicating that women obtained significantly higher scores than men in all subscales, regardless of items 6 and 15 from the anxiety subscale. The highest differences were found for anxiety, as indicated by Cohen's d, followed by the depression, and somatization subscales.
Reliability and convergent validity
As shown in the first column of Table 4, the values of Cronbach's alpha and Composite reliability for the three sub-scales of the BSI-18 and the GSI were satisfactory. Although discriminant validity could not be verified due to high second-order factor loadings, as shown in Figure 1 (λ-SOMA = 0.83, λ-DEPR = 0.91, λ-ANX = 0.95), and low values of Average Variance Extracted (AVE-SOMA =0.40, AVE-DEPR = 0.40, AVE-ANX = 0.27), goodness of fit indexes for the one-dimensional structure did not improve the excellent fit of the hierarchical three-factor structure, as exposed in Table 1.
Table 4 Reliability and convergent validity for improved hierarchical three-factors model.
** p ≤ 0.01; *** p ≤ 0.001
The correlations between the BSI-18 (GSI and subscales) with the 5 categories of the NCCN Problem List are shown in Table 4. GSI correlated the highest with emotional problems followed by substantial correlations with physical, family, practical, and the lowest with spiritual/religious. The Anxiety and Depression subscales correlated quite similar to the previous ones while Somatization showed a higher correlation with physical than emotional problems.
Discussion
This research tested the factor structure and the in-variance of the Brief Symptom Inventory-18 (BSI-18) in a heterogeneous sample of more than 1000 cancer patients. The BSI-18 is considered a reference instrument to assess cancer patients' emotional distress: it is a suitable option to be used in routine screening due to its length (it only has 18 items), it can feasibly be used in clinical settings (it takes only a few minutes to complete), and has norms for an oncological population (both general and separate by sex). Results from this study based on the total sample and subgroups defined by sex confirmed the BSI-18 factor structure through the satisfactory data fit of two hierarchical models: the improved three-factor model (the theoretical proposal) and the four-factor model (the empirical solution reported by Derogatis (2001). Nevertheless, although both achieved satisfactory adjustment indexes, the improved hierarchical three-factor model was considered the best option in cancer setting. This model, in addition to being the most consistent with the intended design of the BSI-18, is more parsimonious and easily interpretable. The three-factor model coincides with the four-factor model in terms of item distribution in the GSI as well as in the depression and somatization dimensions, but it diverges in the anxiety dimension as it has a more simple distribution: one factor vs. two factors: panic and general anxiety. Therefore, the three-factor model provides information about general distress and psychological symptomatology such as anxiety, depression, and somatization. Our results coincide with those obtained by previous studies (using CFA) undertaken with breast cancer patients (Galdón et al., 2008) and survivors of childhood cancer (Recklitis et al., 2006).
Data from this study also suggest that there is partial invariance across sex after relaxing the invariance assumption in only two items. The findings, based on factor loadings and subsequent types of invariance, showed that the majority of items were interpreted equivalently between men and women, but two items belonging to the anxiety sub-scale ("feeling tense or keyed up" and "feeling restless / can't sit still") were not invariant. Apart from this slight discordance, the current findings revealed minimal differences across sex. This means that psychological response is measured similarly across the subgroups that were considered when using the BSI-18 hierarchical three-factor model. Consequently, an important implication of these findings is that comparisons of means, based on the considered demographic variable, are valid and may help professionals to identify which subgroups of cancer patients experience higher levels of distress. Moreover, we observed variations in mean levels of BSI-18 across sex: women showed higher scores - especially in anxiety, although they also scored higher in depression and somatization symptomatology. These results may be due to a possible tendency that men under-report psychological symptoms and, thus, appear to apparently experience a lower prevalence of mental disorders (Zabora et al., 2001). However, given that these findings are based on the partially invariant construct of anxiety, assessments about the size of the differences between sexes should be interpreted with caution.
The reliability of the three original subscales and the total score of the BSI-18 was satisfactory. The Cronbach's alpha values (α = 0.72-0.91) were good to very good and had similar ranges to other studies that included large samples of cancer patients carried out by Derogatis (2001), Zabora et al. (2001), Recklitis et al. (2006), and Recklitis and Rodríguez (2007). Likewise, the Composite Reliability (CR = 0.69-0.90), an index not previously used, also supports the internal consistency of the three subscales and global scale. Oppositely, weak discriminant validity was identified for first-order factors. Consistently, future research could test a bifactor model: an in-depth examination of the contribution each item makes to both general and specific factors.
Using the categories belonging to the NCCN Problem List (NCCN, 2017), the results of the study support the BSI-18's convergent validity due to the highest correlation of sub-scales and GSI being with the emotional and physical categories of problems on the NCCN Problem List. Furthermore, both the total score and anxiety and depression subscales have a higher correlation with the category of emotional problems than with physical problems; the opposite occurs for the somatization subscale. These results endorse those presented by Clover et al. (2016), which showed that Problem List items associated with emotion, physical function, and support were significantly associated with moderate or severe distress.
The current research has both strengths and limitations. It contributes to gaining a better understanding of BSI-18's psychometric properties, but does not address other important issues such as the controversy of the cut-off point for identifying clinical distress. In addition, we are aware that the nature of partial invariance analysis is exploratory, which may be a concern in psychometric work. However, this application of invariance evaluation procedures was conducted in the absence of explicit hypothesis testing. Given the large number of constrained parameters and the large sample size, it is also possible that parameters from the anxiety subscale differ by chance. In future research it would be productive to examine differences between sex in terms of the definition of the anxiety construct. On the other hand, this is the first study to systematically examine the factor structure of the BSI-18 in a large and heterogeneous adult oncology sample. The remarkable strength of the present study is that we tested a theoretical model and empirical proposals that have previously been studied, and we also confirmed the invariance of the BSI-18 across groups based on sex. Moreover, we analyzed the internal consistency of the BSI-18 using a classic index (Cronbach's alpha) as well as a more recently-created index (Composite reliability) and support its convergent validity. Therefore, this work is a further step to validating and analyzing the psychometric properties of the BSI-18 in a cancer setting. The direct implication of these findings is that the BSI-18 may be confidently used as a reliable and valid measure of emotional distress in oncology patients; also, meaningful comparisons among men and women are possible.
