English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
Burnout is a term that is historically used to refer to a syndrome characterised by emotional exhaustion and cynicism (development of indifference and/or aloofness towards other people), which often occurs in individuals who "work with people" in some way or another. One fundamental aspect of burnout syndrome (BS) is the increased perception of emotional exhaustion, and these two characteristics of BS are related to a certain extent. A third aspect of BS is the tendency to assess oneself in a negative manner, particularly with regard to working with others. Workers feel unhappy with themselves and their occupational achievements.1
Since healthcare professionals and dental students in particular are included in this category of human care, the intensity of these symptoms may be increased and can affect different aspects of their personal and professional lives. Mafla et al.2 determined in 2014 that 7% of dental students in Colombia meet the criteria for BS. Thus, measuring this construct becomes an important aspect for ensuring adequate psychological wellbeing.
The measurement of this construct was originally justified for professionals working in human care institutions. The instrument comprised 25 items and was structured around three main factors (emotional exhaustion, personal fulfilment and depersonalisation), with a fourth optional factor (participation).1 The fact that BS was initially only identified in human services workers is largely due to an artificial incident resulting from the universal use of the Maslach Burnout Inventory (MBI). Namely, this questionnaire can only be applied to the aforementioned population group due to the fact that it includes dimensions that are defined in terms of interactions with the survey respondents.
Even though the MBI was previously the most universally accepted instrument, there are currently other alternatives such as the Oldenburg Burnout Inventory (OLBI),3 the Burnout Measure (BM)4 and the Copenhagen Burnout Inventory (CBI).5 The OLBI has two subscales (exhaustion and disengagement) and is based on the argument that the MBI has psychometric limitations due to the fact that the items are all framed in the same direction (positive or negative). The BM was exclusively designed for measuring exhaustion. The items are worded in a general fashion and can thus be applied to any population group. However, studying the psychometric properties of this measure has revealed underlying theoretical issues.6 On the other hand, the CBI allows for context-free BS assessment (specific population group).
Due to growing interest in measuring this phenomenon and the early onset of common mental health disorders in young people, and taking into account that health sciences students dedicate at least 40-60% of their time and schedule to professional training and human care, respectively, which can lead to emotional exhaustion and thus a burnout, a version was designed for screening BS in students (MBI-Student Survey [MBI-SS]). This version contains 15 items with affirmative statements, the wording of which was modified in some cases to adapt it to the student context, and is available in various languages, including Spanish.7,8
In 2013, Rostami et al. assessed the psychometric properties of the MBI-SS among 238 female students from a university in Isfahan. Internal consistency was assessed with the Cronbach's alpha coefficient, according to the subscales, and revealed values between a = 0.84 and a = 0.90; factor structure determined through exploratory factor analysis (EFA) showed three factors: emotional exhaustion, cynicism and academic efficacy. The authors concluded that this instrument presented adequate validity and internal consistency for measuring the construct.9 Nevertheless, the study was only conducted on women, and its validity was determined using EFA. It is therefore necessary to report confirmatory factor analysis (CFA) in order to make suitable inferences regarding the content validity of this phenomenon.
Various research papers report the use of the MBI-SS for assessing BS in dental students in different languages. However, in Colombia there are currently no studies that describe psychometric properties including CFA of the MBI-SS in Spanish. As such, the objective of this study is to determine the content validity and internal consistency of the MBI-SS among dental students in Cartagena, Colombia.
Material and methods
Screening scale validation study with no reference criteria. Given that there are only three Dentistry faculties in Cartagena, which is considered a captive, geographic and spatially well-defined population, it was decided to perform a census. In this sense, the students of all three universities completed the instrument anonymously after familiarising themselves with the research objectives, taking into account the ethical considerations described in Resolution 008430 for health research in Colombia.
The sample of this study belongs to another broader study undertaken in Colombia in 2011: The STRESSCODE Study,2 led by the Cooperative University of Colombia (Pasto campus), whose protocol was approved by the university's Ethics Committee. However, the dental research committees of the three Dentistry faculties taking part in this study approved the protocol at a later time. Likewise, the participants agreed to participate by signing a written informed consent form.
Population characteristics
All undergraduate dental students from the three Dentistry faculties in Cartagena (Colombia) were invited to participate. The inclusion criteria were: students who agreed to participate voluntarily and who were registered and academically active in the second semester of 2011 and the first semester of 2012 during the conduct of the STRESSCODE Study.
The STRESSCODE Study is a multicentre study performed between 2011 and 2012 on 5647 dental students in Colombia. By applying a battery of tests, it revealed the prevalence of stress, psychological distress, burnout, overall self-regulation and related factors. However due to technical and research-related considerations, this study only describes the validity and internal consistency of the MBI-SS in Cartagena students who met the aforementioned inclusion criteria.
Instrument
The MBI is a scale comprising 15 items expressed in the form of statements that investigate the presence of BS-related elements, such as "I feel emotionally drained by my studies" or "I have become more cynical about the potential usefulness of my studies", measured on three factors: emotional exhaustion, cynicism and reduced academic efficacy. To apply the scale, the total time required is 5-8 min.
The frequency with which these aspects appear was determined by assessing the score of their Likert-type responses. They vary on a scale of 0 (never) to 6 (daily) and is considered dimensional given that there is no cut-off point. As such, scores may vary from 0 (lowest intensity) to 90 (highest construct intensity).
The Spanish language version used in this study derives from the version used in the STRESSCODE Study, to which a prior transcultural adaptation process was applied, comprising an English-Spanish translation followed by a Spanish-English translation by qualified scientific personnel.2
Statistical analysis
Descriptive statistics were initially performed. Proportions were calculated for the qualitative variables, while the mean ± standard deviation or median [interquartile range] were calculated for quantitative variables, depending on the distribution symmetry. Internal consistency was estimated with the Cronbach's alpha coefficient overall, as well as in each of the subscales (domains).
In order to assess content validity, the factorability of the matrix was determined with Bartlett's test of sphericity and the Kaiser-Meyer-Olkin measure of sampling adequacy. Subsequently, EFA was performed to reveal the factor structure. The number of factors to be extracted was determined using the values themselves and a Cattell sedimentation graph. To define factor conformity, factor loads with a value of >0.40 were taken into account. Oblique rotation (promax) was also used, given that it was thought that the potential factors would be correlated with each other.
To proceed with CFA, multivariate normality was verified using Mardia's test. Although multivariate normality was not detected, CFA is robust for these types of scenario when there are large sample sizes (>200) and no multivariate normality.10 CFA was performed to confirm the factor structure previously determined in the EFA, and thus the content validity through Partial Least Squares with Mean and Adjusted Variance (PLSMAV). To assess the model fit, the following statistical measures were determined: x 2, degrees of freedom (df), root mean square error of approximation (RMSEA) and confidence interval of 90% (90% CI), comparative fit index (CFI) and Tucker-Lewis index (TLI). These measures were assessed (model fit) using the criteria proposed by Hu and Bentler: p >0.05, RMSEA < 0.06, CFI > 0.95 and TLI > 0.95.11
The descriptive statistical analysis and EFA were performed on the Stata package, v.13.2, for Windows, (StataCorp., College Drive Station, Texas, USA); CFA was performed using Mplus v.7.31 for Windows (Muthén & Muthén, Los Angeles, California, USA).
Results
The MBI-SS was applied to a total of 886 subjects; of these, 59.3% were female, 44.1% were undertaking semesters in the clinical academic year, the majority of whom belonged to stratum 3, and 54.39% came from a private university. The mean age was 20.42 ±2.8 years. Fig. 1 depicts the study entry flow diagram. The study response rate varied from 88% to 100% (mean: 92.5%), while the MBI-SS scores were between 0 and 90. The highest scores on the scale corresponded to means of5.11 and 4.93, which were found under the "I have learned many interesting things during the course of my studies" and "I feel stimulated when I achieve my study goals" items. The lowest were found under the "I doubt the significance of my studies" and "I have become more cynical about the potential usefulness of my studies" items, with a mean of 1.23. The scores obtained in each domain of the scale are shown in Table 1 By domain, the lowest score corresponded to cynicism.
Table 1 Descriptive statistics of each of the reactive components of the MBI-SS scale (Cartagena de Indias, February-August 2011, surveyed students).
Content validity
Bartlett's test of sphericity, applied in the EFA, showed x 2 = 6924.232, df=105 and p < 0.001; the Kaiser-Meyer-Olkin measure of sampling adequacy was: 0.8529. Factor structure indicates three factors: factor I was named "emotional exhaustion" and showed an eigenvalue of 3.96, which explained the 26.41% total variance; factor II, named "cynicism", reached an eigenvalue of 3.45, accounting for the 23% total variance; and factor III, named "reduced academic efficacy", had an eigenvalue of 1.08, thus explaining the 7.26% total variance. Table 2 how is the factor solution obtained.
Table 2 Exploratory factor analysis (EFA) results. Item and uniqueness loads (Cartagena de Indias, February-August 2011, surveyed students).
The CFA found the following fit indices: x 2 = 1137.38, df =87, RMSEA = 0.117 (90% CI, 0.111-0.123), CFI = 0.934 and TLI = 0.920. Nevertheless, after applying two modification indices, the following fit indices were obtained: x 2 = 926.036, df=85, RMSEA=0.106 (90% CI, 0.100-0.112), CFI=0.947 and TLI=0.934.
Discussion
Various studies have used the Spanish version of the MBI-SS to quantify BS, indicating a prevalence that ranges from 7% to 28%.2,12,13 However, few studies have explored its validity and internal consistency in Spanish and for populations in Colombia.2 In this sense, the results of this study provide the first ever approach to the content validity and internal consistency of this scale, in Spanish, among dental students from Cartagena, Colombia.
In validation studies, it is necessary to take population characteristics into account, especially when the test is applied to the general population; the age and gender of the participants could act as a barrier to generalising the results obtained,14 due to the fact that the risk of suffering from BS may vary according to these sociodemographic characteristics. This is one of the main reasons for developing the MBI-SS, given that the context to which the general version of the scale was traditionally applied was unsuited to students and implied measurement bias.
In this sense, the results of the study are only applicable to students registered at the three dentistry faculties of the city of Cartagena undertaking semesters in the clinical academic year, from stratum 3, for the most part, and private universities, which limits the application of the instrument to other health areas or to populations with characteristics that differ from those of the sample included here.
The MBI-SS is easy to apply due to the lower number of questions, and the scale used for construct quantitation facilitates quick and reliable measurement. These questions explore the major elements that may define the presence of BS, which is reflected by an adequate internal consistency according to Cronbach's alpha coefficient, calculated overall as well as for the three domains proposed, thus also agreeing with other studies that have validated the MBI-SS in other languages.10 It is important to take into account that reliability is a necessary step for initiating the validity estimation process, but a reliable instrument is not necessarily valid and, in turn, a valid instrument is not useful if it is not sufficiently reliable, thereby reflecting the need to estimate internal consistency in validation studies in order to express interrelation between construct items.15-17
The results of this study show that the MBI-SS on undergraduate dental students in Cartagena offers a duplicable factor structure compared to the version that was originally proposed1,11,18 and studies on the content validity of this scale in other languages, such as Portuguese.19 The three factors found explain more than half of the variance; this may assert that the construct is present in the items included in the scale. This process is important for gathering data with adequate validity and reliability, and must be performed before beginning any study. However, on evaluating the fit indices provided, the CFA revealed that the model presents poor fit. In this sense, it is necessary to continue investigating some items that in this population do not completely reflect the nature of the construct despite the fact that its factor structure was successfully duplicated in the exploratory factor analysis.
A research study in Isfahan to determine the validity of the Persian version of the scale showed a three-factor structure, but with two fewer items in the cynicism factor, which were included under the emotional exhaustion factor. However, these authors did not perform CFA, which is an essential approach to determine the validity of a scale.9,20 This discrepancy between the findings can also be explained by the fact that this study only included female students with different academic levels (masters and doctorates), which may signify greater sources of variance in construct measurement.
The justification behind the inclusion of modification indices (MI) in the CFA models is that, by identifying "areas of tension" in the model, statistical strategies may be applied to improve the content validity of the instrument, which is reflected in the improvement of the model fit indices. The modification made to the model is based on the similar wording of the "I feel emotionally drained by my studies" and "I feel exhausted at the end of a day at university" items; moreover, due to belonging to the same domain, these items may share the error covariance. Moreover, this phenomenon could also be observed in the "I have become more cynical about the potential usefulness of my studies" and "I doubt the significance of my studies" items. The usefulness of including the MI may thus be reflected in the statistically significant improvement obtained for the model on comparing the two models in CFA.10,20
In this sense, in the case of the MBI-SS, two MIs were applied, resulting in a statistically significant improvement in the model fit; however, the scale still presents a poor fit with respect to its content validity. These findings do not match those reported in other studies, which indicate a good fit on employing CFA techniques.9 The differences lie fundamentally in the criteria used to assess the model fit obtained, which note a CFI value of >0.90 and a RMSEA of <0.10.21,22 Nevertheless, in this research study, stricter assessment criteria were used that were universally consistent with statistical scientific literature on CFA and structural equation modelling, which consider model fit "acceptable" if the RMSEA is <0.06, x 2 with p>0.05 and CFI and TLI >0.95. Although the model had poor fit, the results of the x2 test are expected to be significant due to the fact that its use in CFA techniques is sensitive to sample size; on increasing the sample size, this will tend to be significant. In light of this, other measures of fit, such as the CFI, RMSEA and TLI, must be taken into account for assessing the model.6
Measuring BS is important as it permits the early detection of potential cases, thereby attenuating its serious consequences on physical and mental health. In this sense, historically, few instruments have been proposed for measuring this construct, so the MBI is the most used model; more than 90% of the world's research on burnout has been performed with this instrument.1 Although various versions have been proposed, including the MBI-SS, with the purpose of improving the detection of the construct in different population groups, numerous authors have noted the marked shortcomings of the instrument's design, which may compromise its results when applied to populations other than the one for which it was originally developed.2 Exhaustion is the central aspect of this syndrome and the MBI, on considering the other two factors, erroneously characterises the concept of burnout given that these dimensions represent a coping strategy and consequence of the syndrome, respectively. This may partially explain the poor fit observed on assessing content validity. Although these hypotheses stem from research studies with the original instrument for professionals, it must be taken into account that the student version is an adaptation.
This created the need to obtain new instruments for measuring BS, such as the Copenhagen Burnout Inventory3 and the one-dimensional student burnout scale.23 The former is intended for measuring the construct among professionals, while the latter is for students (originally designed in Spanish). The one-dimensional student burnout scale still has no scientific evidence on its content validity, but EFA results show that there are two diverging dimensions with respect to the MBI-SS: behavioural indicators and attitude indicators.
These findings highlight the importance of performing more in-depth studies on the content validity of the MBI-SS and of exploring other instruments for this population that may reflect adequate content validity. Future efforts will have to be made to improve this aspect of its psychometric performance, and to thus improve the quality of the measurements performed with the instrument and, as a result, the possibility of generalising the results derived from such research.
