text in 
English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
What do we know about this topic?
Burnout Syndrome (BOS) is a syndrome derived from chronic stress in the workplace that was not successfully managed.
What new does this study contribute with?
In Colombia, there is no a validated instrument that can be used to measure the BOS on health human talent. Therefore, this validation and reliability study of the Spanish Burnout Inventory (SBI) for burnout assessment will contribute this validated tool for further studies.
INTRODUCTION
Burnout syndrome (BOS) is the term used to describe a response to chronic stressors at work and may occur in individuals providing patient care 1-3. This syndrome was first described in 1974 by Helbert Freudenberg in New York 4.
The World Health Organization (WHO) has highlighted the importance of human resources as critical for quality healthcare 5 and included burnout in its 11th revision of the International Classification of Diseases (ICD-11) describing it as an occupational exhaustion syndrome. The WHO defines it, not as a disease, but as a syndrome derived from chronic workplace stress that has not been successfully managed, and the term must not be applied to describe experiences in other areas of life 6.
In the world, the incidence of BOS among physicians is close to 28% 7. In Colombia, the prevalence of BOS in medicine ranges between 17.6% and 45% 8,9.
In 2006, Gil-Monte designed the questionnaire for workplace burnout assessment, which includes four domains: enthusiasm toward the job (It), understood as the achievement of goals that are perceived as creating a feeling of enjoyment and personal satisfaction; mental exhaustion (Dp), consisting of emotional and physical exhaustion derived from care-related activities; indolence (In), which refers to the presence of negative attitudes such as cynicism, indifference and callousness towards the people served; guilt (C), as the feeling that emerges as a result of the individual's own behavior or work-related negative attitudes, mainly towards people with whom relations are established at work. The SBI is summarized in the form of two different profiles, depending on whether the guilt domain is included or not 10,11.
Human resources working in health care may suffer from high levels of anxiety, depression, suicide and BOS due to work overload or stress. The SBI has been shown to be useful for diagnosing BOS in countries such as Spain, Mexico and Brazil 12-14; however, it has not been validated yet in Colombia.
The objective of this study was to assess content and construct validity (CV) and reliability of the SBI in surgical specialists working in healthcare institutions in Antioquia, Colombia in 2018.
MATERIALS AND METHODS
The SBI validation and reliability study was approved by the Ethics Committee of the Universidad de Antioquia Medical Research Institute as stated in Minutes No. 088 of May 24, 2018, and by the committees of each of the participating institutions. The study complied with the current regulations on human research, in particular Resolution 8430 of 1993 of the Colombian Ministry of Health and the 2013 Declaration of Helsinki on human research. The study was classified as minimum risk.
The SBI copyrights were purchased directly from the TEA Ediciones representative in Colombia; the author gave permission to use the tool. Data were collected in a database using a Google docs form, with doble entry in Windows Excel. The SPSS 23 and MPlus 8 statistical software packages were used for data analysis 15.
Participants
The participants were Colombian physicians from eight surgical specialties working in 14 healthcare institutions located in the department of Antioquia, Colombia. participants were recruited between June 27 and September 4, 2018.
Sample selection
Of the 415 surgical specialists working in the 14 healthcare institutions, a sample of 220, stratified by specialty, was selected; convenience sampling was used among those who agreed to participate.
Inclusion criteria: Colombian nationals practicing a surgical specialty such as anesthesiology, surgery, pediatric surgery, obstetrics and gynecology, neurosurgery, orthopedics, urology and otolaryngology, with more than six months of experience in their surgical specialty.
Exclusion criterion: surgical specialist working only in an administrative role.
The sample size was estimated based on the need to recruit at least 10 subjects for each item as proposed by Kline, and at least 200 as the ideal minimum for performing a factorial analysis, as suggested by Ferrando and Anguiano 16,17. For reliability of the sample size based on internal consistency, a number ranging between 5 and 20 participants for each questionnaire item was estimated. In this case, considering that the SBI has 20 items, the sample of 200 participants was sufficient for the analysis 18-20. Potential data attrition was estimated at 10%, for a total of 220 participants.
Pilot test
The pilot test was conducted with 11 volunteer healthcare professionals not included in the study, under the supervision of a psychologist with training in the use of the tool.
Tool
The SBI is a questionnaire used to assess cognitions, emotions and attitudes related to individual workplace experiences. It consists of 20 items covering four domains: enthusiasm toward the job, psychological exhaustion, indolence and guilt. Each item is assigned a frequency of 0 to 4, where 0 is never; 1 is rarely, sometimes during the year; 2 is sometimes, at times during the month; 3 is frequently or sometimes during the week; and 4 is very frequently, every day 10,21. Direct scores for each domain and total scores are found in healthcare personnel table of rates of the SBI Manual and are used to derive percentiles for each domain and the total BOS. The Guilt domain is interpreted independently and its items are not part of the total SBI score. The aim is to distinguish between two profiles: Profile 1 which indicates BOS symptoms without guilt regarding patient care; and Profile 2 that reflects a critical BOS level with Guilt (percentile ≥ 90).
The SBI uses four cutoff points for interpreting the scores, based on percentiles: very low <11; low, 11 to 33; medium, 34 to 66; high, 67 to 89; and critical >89 10.
A psychologist supervised full completion of the tool by the specialists who were working at the time of the visit to each institution, ensuring privacy, willingness and data confidentiality as well as control of missing data.
Statistical analysis
Categorical sociodemographic and work variables (gender, marital status, medical-surgical specialty, type of work contract and leisure time activity, enthusiasm towards the job, psychological exhaustion, indolence, guilt, total BOS) were summarized as absolute and relative frequencies; for quantitative variables (age, number of children, years of experience, total working hours per month, weekend hours worked per month, night shifts per month) mean and standard deviation or median and inter-quartile range were used according to whether data distribution was symmetrical or not. Floor and ceiling effect was estimated for each domain and also in the total SBI score, when more than 15% of the participants obtained the lowest or the highest score, respectively 22.
Content validity. Lawshe's critical content validity ratio (CVR) was used for each item, as it is accepted above the lower critical level of 0.622 for 7 experts, according to Schipper (assessed by the psychiatrist, psychologist and social worker by e-mail), with a one-tail normal approximation and a: 0.05; once the critical CVR for all the items was calculated, the content validity index (CVI) was estimated for those that are accepted, with a total scale acceptability level ≥ 0.8 23-29.
Construct validity. Construct validity was assessed using the confirmatory factorial analysis (CFA) 15, with a model specified in advance 10, supported on the questionnaire validation studies with four domains that were used as separate latent, albeit correlated, factors. The Weighted Least Squares Mean and Variance adjusted estimation method was used 30; the model's overall goodness of fit was evaluated, with X2 (p > 0.05), estimating the ratio between its value and the degrees of freedom. Other indices estimated the overall error quantity in the model: Root Mean Squared Error of Approximation (RMSEA), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI) and the Weighted Root Mean Square Residual (WRMSR). The RMSEA index quantified data divergence, where a value of < 0.06 indicated good fit, while values up to 0.08, with 90% CI, represented acceptable fit. For CFI and TLI, values > 0.95 were considered as acceptable model fit; for WRMR, values < 0.9, with model fit acceptability; for variables, considering their categorical nature, a polychoric correlation was used 31,32.
Reliability. Reliability was evaluated for domains and for the total scale, assessing item-item, item-domain and item-scale internal validity using Cronbach's alpha coefficient and its 95% confidence interval; Cronbach's alpha coefficient between 0.7 and 0.9 indicates good reliability; low values refer to poor homogeneity of the scale, as it may be assessing different aspects or conditions due to poor item-to-item correlation; and values above 0.9 reflect an exceedingly homogenous scale, with potential redundancy or item duplication 19,20,33.
RESULTS
Sample description
The sample was collected in 14 hospitals in Antioquia and consisted of 246 medical-surgical specialists; 12 participants were excluded (5 with different nationality, 2 with a different specialty, and 5 who declined to participate), leaving 234 for the analysis; the sociodemographic characteristics of the participants are summarized in Table 1.
TABLE 1 Sociodemographic and work characteristics of medical-surgical specialists working in 14 healthcare institutions included for SBI validation and reliability in Medellin, Colombia, 2018.
| Variable name | Median | SD |
|---|---|---|
| Age (years) | 44.9 | 10.46 |
| Number of children | 1.17 | 1.14 |
| Median | Q1-Q3 | |
| Years of experience | 10 | 5.0-20 |
| Number of workplaces | 2 | 2-3 |
| Hours worked/month | 240 | 200-300 |
| Hours worked/week-ends | 36 | 18-55.5 |
| Hours worked/nighttime | 26 | 0.0-70 |
| Absolute frequency | % | |
| Gender | ||
| Female | 43 | 18.4 |
| Male | 191 | 81.6 |
| Marital status | ||
| Single | 54 | 23.1 |
| Married | 138 | 59 |
| Free union | 27 | 11.5 |
| Separated | 15 | 6.4 |
| Surgical specialty | ||
| Anesthetist | 84 | 35.9 |
| Surgeon | 38 | 16.2 |
| Pediatric surgeon | 7 | 3.0 |
| Obstetrician and gynecologist | 31 | 13.2 |
| Orthopedist | 38 | 16.2 |
| Neurosurgeon | 11 | 4.7 |
| Urologist | 17 | 7.4 |
| Otolaryngologist | 8 | 3.4 |
| Type of contract | ||
| Non-fixed term | 84 | 35.9 |
| Fixed term | 17 | 7.3 |
| Service contract | 66 | 28.2 |
| Union | 67 | 28.6 |
| Leisure time activity | ||
| Reading | 111 | 47.4 |
| Sports | 108 | 46.2 |
| Family | 183 | 78.2 |
| Movies and TV | 86 | 36.8 |
| Travel | 13 | 5.6 |
| Friends | 3 | 1.3 |
| Others | 47 | 20.1 |
| Healthcare institution | ||
| Institution 1 | 28 | 12 |
| Institution 2 | 7 | 3 |
| Institution 3 | 12 | 5.1 |
| Institution 4 | 22 | 9.4 |
| Institution 5 | 30 | 12.8 |
| Institution 6 | 5 | 2.1 |
| Institution 7 | 14 | 6 |
| Institution 8 | 8 | 3.4 |
| Institution 9 | 20 | 8.5 |
| Institution 10 | 12 | 5.1 |
| Institution 11 | 14 | 6 |
| Institution 12 | 49 | 20.9 |
| Institution 13 | 7 | 3 |
| Institution 14 | 6 | 2.6 |
+Shapiro Wilk normality test (p < 0.05). Me: Median; Q1: first quartile; Q3: third quartile; SD: Standard Deviation; %: Percentage.
SOURCE: Authors.
No floor or ceiling effect was found; only the Enthusiasm Towards the Job domain showed a ceiling effect in 25.2% of the respondents. There were no missing data in the assessment.
Content validity (CV)
CV was determined using the content validity index (CVI) with relevance, pertinence and sufficiency criteria applied to all the items. All items are accepted since they are above the lower critical level of 0.622 for seven experts (Table 2).
TABLE 2 Lawshe's critical CVR according to Schipper in the Enthusiasm Towards the Job, Psychological Exhaustion, Indolence and Guilt domains.
| Domain | Item | Relevance | Pertinenence | Sufficiency |
|---|---|---|---|---|
| Enthusiasm towards the job | 1 | 1.00 | 1.00 | 0.71 |
| 5 | 1.00 | 1.00 | ||
| 10 | 1.00 | 1.00 | ||
| 15 | 1.00 | 1.00 | ||
| 19 | 0.71 | 0.71 | ||
| Psychological exhaustion | 8 | 1.00 | 1.00 | 0.86 |
| 12 | 0.71 | 1.00 | ||
| 17 | 1.00 | 0.71 | ||
| 18 | 1.00 | 1.00 | ||
| Indolence | 2 | 1.00 | 1.00 | 0.71 |
| 3 | 1.00 | 0.71 | ||
| 6 | 0.71 | 0.71 | ||
| 7 | 1.00 | 1.00 | ||
| 11 | 0.71 | 0.71 | ||
| 14 | 0.71 | 1.00 | ||
| Guilt | 4 | 1.00 | 1.00 | 0.86 |
| 9 | 1.00 | 1.00 | ||
| 13 | 0.71 | 0.71 | ||
| 16 | 1.00 | 1.00 | ||
| 20 | 1.00 | 1.00 |
Lower critical level of 0.622 for 7 experts according to Schipper with a one-tailed normal approximation α 0.05.
SOURCE: Authors.
Once the critical CVR for all items was estimated, the CVI was estimated for the entire test, with an acceptability of 0.82.
Structural validity
The CFA results for construct validity were based on a reflective model where global goodness of fit indices were X2=384,578 (p < 0.001; degrees of freedom [df]: 165). The X2/gdf ratio was calculated at 2.3 and, considering that it is above 2, it indicates that there is no fit for the model with that test; the RMSEA = 0.075 [90% CI: 0.0660.085]. The remaining indices are shown in Figure 1, where an acceptable model fit is observed. Acceptable data fit was found for the CFA model in the first contrast for the selected sample using all of the fit indices. There was no need to modify the model or to delete or remove items. As shown in Figure 1, all of the values for the standardized coefficients are high, the lowest being found for item 4 of the Guilt domain (0.62) and the highest for item 18 of the Psychological Exhaustion domain.
Reliability
Cronbach's alpha was used to assess reliability, showing adequately homogenous values with no data redundancy or duplication; as illustrated in Table 3, Cronbach's alpha was above 0.7 for all domains, as was also true for all the 20 items of the SBI scale at 0.777 and 95% CI [0,733-0,816]. Therefore, the whole questionnaire as well as each of the domains were found to be reliable for measuring BOS.
TABLE 3 Reliability using Cronbach's alpha for the SBI domains.
| SBI - Dimensiones | Alfa de Cronbach* | IC 95 % |
|---|---|---|
| Ilusión por el Trabajo | 0.789 | [0.744 - 0.829] |
| Desgaste Psíquico | 0.897 | [0.874 - 0.917] |
| Indolencia | 0.750 | [0.697 - 0.797] |
| Culpa | 0.792 | [0.746 - 0.831] |
| Total SBI | 0.777 | [0.733 - 0.816] |
*Cronbach's alpha for the total SBI does not include Guilt.
SOURCE: Authors.
Exploratory results
An exploratory calculation was performed with the aim of determining the level of burnout among the participants; of the 234 respondents, it was found that 24 (10.3%) and 12 (5.1%) of the surgical specialists were at a high and critical level for BOS, respectively (Table 4).
TABLE 4 Distribution of SBI levels among all the medical-surgical specialists of the 14 healthcare institutions in Medellín, Colombia, 2018.
| SBI Levels | Absolute frequency | % |
|---|---|---|
| Enthusiasm towards the job | ||
| Critical | 0 | 0 |
| High | 92 | 39.3 |
| Medium | 112 | 47.9 |
| Low | 24 | 10.3 |
| Very low | 6 | 2.6 |
| Psychological exhaustion | ||
| Critical | 16 | 6.8 |
| High | 29 | 12.4 |
| Medium | 93 | 39.7 |
| Low | 50 | 21.4 |
| Very low | 46 | 19.7 |
| Indolence | ||
| Critical | 18 | 7.7 |
| High | 42 | 17.9 |
| Medium | 96 | 41.0 |
| Low | 50 | 21.4 |
| Very low | 28 | 12.0 |
| Total SBI in health | ||
| Critical | 12 | 5.1 |
| High | 24 | 10.3 |
| Medium | 103 | 44.0 |
| Low | 60 | 25.6 |
| Very low | 35 | 15.0 |
| Profiles | ||
| 1 (with Guilt) | 220 | 94.0 |
| 2 (without Guilt) | 14 | 6.0 |
Level distribution by percentiles: Very low: ≤ 10; Low: 11-33; Medium: 34-66; High: 67-89 and Critical: 90-99. Without Guilt: ≤ 89; with Guilt: 90-99.
SOURCE: Authors.
DISCUSSION
The objective of this study was to assess content and construct validity, as well as consistency-type reliability of the SBI in surgical specialists working in 14 hospitals in Antioquia, during 2018. In a sample of 234 respondents, results showed that the scale is valid and reliable.
In 1981, Maslach and Jackson developed the Maslach Burnout Inventory (MBI) with three domains: emotional exhaustion, depersonalization and a low sense of professional accomplishment 2,34; as for its limitations, it lacks a common cutoff criterion to determine the diagnosis, and the Depersonalization domain has a low reliability index; the Spanish language has psychometric weaknesses and low discrimination validity 35.
Results show an acceptability of 82% for the content validity index (CVI) for the overall test, meeting the acceptability criterion above the critical level of 0.622 for seven experts 26,28; in contrast, in the thesis by Peñalosa who used the SBI as a measurement tool and assessed the CVI among five reviewers, item 16 "I think I should apologize to someone for my behavior" in the relevance variable, the CVI is 0.7, failing to meet the content validity criterion 36.
Using the WLSMV method, the estimated fit indices for CFA confirm acceptably the hypothesis of the overall SBI fit to the observed data, mainly for RMSEA, CFI and TLI, which is consistent with the findings by Gil-Monte in 2011 for CFA with 9080 participants using the maximum likelihood method (ML), with adequate model fit and the following overall fit results: X2=3126.75 (p = 0.001; df: 164); RMSEA = 0.045; CFI = 0.947; NFI: 0.945; and GFI: 0.965 10.
As far as correlations with the four domains of the SBI are concerned, the expected result was obtained for the Enthusiasm Towards the Job domain, with negative scores when compared with the other domains, while the correlations among the remaining domains scored positive, with the highest correlation found between Psychological Exhaustion and Indolence, and Guilt and Indolence.
Reliability results using Cronbach's alpha coefficient for Enthusiasm Towards the Job, Psychological Exhaustion, Indolence and Guilt are reliable, as is also the case for overall Cronbach's alpha coefficient for all the items, where results show that the test is reliable for the study of BOS in each of the domains. The study by Peñalosa shows a Cronbach alpha value of 0.748 for the SBI as well as for each of the domains, as follows: Enthusiasm Towards the Job, 0.754; Psychological Exhaustion, 0.806; Indolence, 0.715; and Guilt, 0,814 36. In contrast, the study by Córdoba et al., in which the authors used the Maslach Burnout Inventory-Human Services Survey (MBI-HSS) for test adaptation and validation in Cali, Colombia, with 314 healthcare professionals, found the following internal consistency using Cronbach's alpha coefficient: 0.77 for the MBI-HSS, and 0.83 for emotional exhaustion, 0.52 for depersonalization, and 0.57 for personal accomplishment. Of these, the latter two show an inadequately low coefficient for BOS reliability using the MBI-HSS in the Cali-Colombia sample 37. Regarding scale reliability using Cronbach's alpha coefficient in the four domains as well as for the overall SBI result, adequate values above 0.7 were obtained. Therefore, it can be concluded that the scale is reliable for assessing burnout syndrome (BOS) and can be used to make an adequate diagnosis according to profile 1 (without Guilt) or profile 2 (with Guilt).
In their study with 110 Mexican physicians, Gil-Monte et al. carried out a factorial validity test of the SBI using the maximum likelihood (ML), with the following results, considered sufficient: X2/gl=1.37 and GFI: 0.83. The model was found to be adequate when considering the RMSEA indices: 0.061; NFI: 0.9; CFI: 0.92 and PNFI: 0.65; factorial loads were significant, and item 11 under the Indolence domain was the lowest at ʎ: 0,25. Reliability indices using Cronbach's alpha for Enthusiasm Toward the Job (0.72), Psychological Exhaustion (0.86), Indolence (0.75) and Guilt (0.79) show that the SBI is a valid and reliable tool to assess BOS in Mexican physicians 21; Gil Montes et al. showed similar results in a study with 720 nurses in Valencia, Spain, showing validity and reliability of BOS 38.
Strengths of this study include the following: several medical-surgical specialties in 14 healthcare institutions were considered; reviewer bias was controlled by means of training in the SBI of the psychologist in charge of collecting the data face-to-face; a pilot test was carried out in order to avoid questionnaire bias; to avoid response bias, privacy was ensured, with no audio or video recording; personal data confidentiality was secured. Although convenience sampling was used, the selected participants met the inclusion criteria and they all agreed to participate by signing an informed consent. Polychoric matrices were used for the factorial analysis, allowing the use of quantitative variables with ordinal response options (Likert-type) for the tool. A validated and reliable tool is now available for future BOS studies designed to provide information for diagnosis and prevention and intervention plans.
This study contributes to the knowledge about this phenomenon at an institutional, regional and national level, for future design of occupational wellness and mental health plans targeted at healthcare personnel working with patients, in order to enhance overall performance in their personal, family, social and work life.
Weaknesses were found in terms of selection bias, given that only surgical specialists were considered and selection was non-randomized, which means that results cannot be extrapolated. The study was conducted in the Medellín metropolitan area, hence the recommendation to conduct further research at a broader regional and national level. It is worth considering that participants with BOS might be more inclined to participate, leading to overestimation of the results.
Given that the participants were not involved in assessing relevance or thoroughness of the SBI content and only the judgement of the seven experts was taken into account, there was a limitation in terms of content validity.
Further studies including test-retest, convergent, divergent and discriminating construct validity and sensitivity to change are needed; fit tests show that the model is acceptable; an exploratory factorial analysis to determine whether the construct behaves differently in the assessed population would be an important component in a future study.
Prospects for research include a national study of the healthcare population in order to identify the population at risk, BOS prevalence and incidence at a regional and national level, with the aim of creating awareness among scientific societies, healthcare institutions and occupational risk management organizations (ARL).
The results found by this study for content validity are acceptable and applicable to all items and the four domains. Construct validity as determined by the CFA is acceptable, as is also the case with the internal consistency-type reliability results for BOS study in surgical specialists. Specialties in which BOS was found, in order, were: surgery, 22.2%; obstetrics and gynecology, 22.2%; orthopedics, 13.9%; anesthesiology, 11.1%; and urology 11.1%. The results of this study pave the way for future research on BOS in different healthcare institutions, thus expanding the study population working in direct contact with patients.
ETHICAL RESPONSIBILITIES
Ethics committee endorsement
This study on SBI validation and reliability was approved by the Ethics Committee of the Universidad de Antioquia Medical Research Institute, as stated in minutes No. 088 of May 24, 2018, and by the committees of each of the participating institutions.
Human and animal protection
The authors declare that no human or animal experiments were conducted for this research. The authors declare that all of the procedures used were in compliance with the ethical standards of the responsible human experimentation committee, the World Medical Association and the Declaration of Helsinki.
ACKNOWLEDGEMENTS
Authors' contributions
LMGB: Original project conception, study planning, data collection, data analysis, interpretation of the results and final drafting of the manuscript.
DCAA: Study planning, data analysis, interpretation of the results and review of the final drafting of the manuscript.
CMZR: Study planning, data collection, data analysis, interpretation of the results and final review of the manuscript.
HIGG: study planning, data analysis, interpretation of the results, review of the drafting and final approval of the manuscript.
