English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Stress is associated with a high public health and societal impact (Xiong et al., 2020). Over time, it has been conceptualised mainly based on three complementary approaches: (1) psychological, based on people’s subjective appraisals (Lazarus & Folkman, 1984); (2) biological, related to physiological responses to stress (McEwen, 2007); and (3) environmental, associated to life events (Monroe & Simons, 1991). Based on a psychological transactional approach, stress has been defined as the reaction that occurs when people perceive a discrepancy between their resources and/or their capacity to respond to an event, stimulus, or stressor (Lazarus & Folkman, 1984).
For many years, psychological stress has been linked to negative health status, due to an increased allostatic load through psychophysiological reactivity, epigenetic mechanisms, and poor mental health outcomes (Feng et al., 2023). There is evidence that chronic psychological stress is associated with 75-90% of health conditions (Liu et al., 2017). The early detection and prevention of psychological stress symptoms, thus avoiding their chronification, is a challenge of increasing interest for clinicians and researchers worldwide (Xiong et al., 2020).
A progressive increase in psychological stress has been documented (Enticott et al., 2022). Already during the COVID-19 pandemic, an acute increase in psychological stress levels was documented in 57 countries, particularly in females, older adults, and people with low educational levels (Adamson et al., 2020). In the post-pandemic era, an increase in the prevalence of mental health disorders has been reported (Newnham et al., 2022). These events have especially impacted populations with high social and economic inequality, such as university staff (Stringer, 2023).
A timely assessment contributes to a reduction of psychological stress and prevents the development of emotional disorders, such as anxiety and depression (Feng et al., 2023; Xiong et al., 2020). It is important to have reliable, valid, and brief screening tools for measuring psychological stress (Goldberg et al., 2017). Particularly, ultra-brief screening tools could help reduce misdiagnosis, optimise healthcare system resources, and improve clinical outcomes (Sanabria-Mazo et al., 2021a; 2023).
The Perceived Stress Scale (PSS) is a tool for assessing psychological stress. The psychometric properties of the PSS-14 and PSS-10 versions have been extensively explored in clinical and non-clinical populations (Lee et al., 2015; Maroufizadeh et al., 2018; Schneider et al., 2020), demonstrating their validity and reliability. It has been translated into Spanish (Remor, 2006), with evaluations of cross-cultural factorial invariance (Perera et al., 2017) and adaptations with samples of the population with a university education, specifically in the Spanish-speaking context in Colombia (Campo-Arias et al., 2014), Mexico (González-Ramírez et al., 2013), and Peru (Boluarte-Carbajal et al., 2023).
The PSS-4, an ultra-brief version of four items of the original PSS-14, has also been tested in England (Warttig et al., 2013), the United Kingdom, France, Spain (Vallejo et al., 2018), Italy (Mondo et al., 2021), China (She et al., 2021), and Ecuador (Ruisoto et al., 2020). Although some of these studies have reported adequate fit indices in the two-factor model (hopelessness and coping), some evidence indicates that this ultra-brief version does not report a completely adjusted model (Ingram et al., 2016). For this reason, further research is needed to investigate the psychometric properties of the PSS-4 in different contexts and populations (Yılmaz-Koğar & Koğar, 2023).
This study evaluated, for the first time, the psychometric properties of the PSS-4 in a Colombian sample. The specific objectives and hypotheses explored were:
1. First, to examine the goodness-of-fit of three models: (1) a one-factor model (i.e., psychological stress) with the four items loading on one latent factor, (2) a one-factor model (i.e., psychological stress) with method effects (correlated error terms on the negative-phrased items), and (3) a two-factor model (i.e., hopelessness and coping) with two items in each factor. It was speculated that the one-factor model (psychological stress) with method effects would yield the most optimal solution in terms of goodness-of-fit and parsimony considerations (hypothesis 1).
2. Second, to investigate invariance (i.e., configural, metric, scalar, and strict) of the best-fitting model across socio-demographic characteristics (i.e., gender, income level, age range, and work status). This is the first study to provide evidence of invariance in PSS-4. As in previous validations of PSS-10 (Barbosa-Leiker et al., 2013; Lee, 2022; Liu et al., 2020; Reis et al., 2019), it was expected that the scores were invariant across gender (hypothesis 2).
3. Third, to explore the reliability of the PSS-4 through different reliability indices (i.e., Cronbach’s a, McDonald’s w, and Guttman’s l2). Being consistent with findings reported in previous studies (Ruisoto et al., 2020; Vallejo et al., 2018; Warttig et al., 2013), the PSS-4 was expected to reliably measure psychological stress beyond the reliability index examined (hypothesis 3).
4. Fourth, to analyse the construct validity (convergent and discriminant). In congruence with other studies (Mondo et al., 2021; Ruisoto et al., 2020; She et al., 2021; Vallejo et al., 2018; Warttig et al., 2013), the PSS-4 scores are expected to be positively correlated with anxiety and depression and negatively correlated with post-traumatic growth and resilience (hypothesis 4).
5. Fifth, to evaluate the predictive validity of the PSS-4 on a set of psychological variables (resilience, post-traumatic growth, anxiety, and depression). As in previous studies (Feng et al., 2023; Finstad et al., 2021; Gómez-Acosta et al., 2023), PSS-4 scores were expected to significantly predict resilience, post-traumatic growth scores, anxiety, and depression (hypothesis 5).
6. Sixth, to test the relationship between the PPS-4 scores and the socio-demographic characteristics of this sample. Based on previous psychometric studies (Ruisoto et al., 2020; Vallejo et al., 2018; Warttig et al., 2013) and from a known-groups validity approach, it was expected that females, older individuals, those with lower incomes, or the unemployed would exhibit higher PSS-4 scores (hypothesis 6).
Method
Study design
This is a cross-sectional online psychometric study. Data analyses of the PSS-4 were performed using the dataset from the PSY-COVID study in Colombia. More detailed information about this project is provided elsewhere (Sanabria-Mazo et al., 2021a; 2021b).
Participants
This study included adult participants (≥ 18 years) with a university education residing in Colombia. A total of 1,911 adults (Mage = 38.40; SDage = 11.82; range = 19-82 years old) participated in the study. As shown on Table 1, most participants were female (79%), adults between 30-44 years old (41%), with medium income levels (65%), and formal work (56%).
Procedure
An anonymous survey was carried out using Google Forms® through non-probabilistic sampling. This survey was disseminated through social networks, media, and institutional contacts. The data collection for responses was available from December 1st (2021) to April 30th (2022). This research was approved by the Animal and Human Experimentation Ethics Committee of the Autonomous University of Barcelona (CEEAH-5197) and was conducted according to the principles reported in the Helsinki Declaration.
Instruments
Sociodemographic Questionnaire. It was used to identify gender, age, income level, and work status.
Perceived Stress Scale (PSS-4). It assesses perceived stress (Cohen et al., 1983). There are three versions of the PSS (PSS-4, PSS-10, and PSS-14), although in this study only the ultra-brief version is explored. This version contains 4 items with a 4-point Likert response format, where 0 corresponds to “never” and 3 to “almost every day”. The total score of the PSS-4 ranged from 0 to 12, with higher scores indicating higher perceived stress.
Patient Health Questionnaire-4 (PHQ-4). It assesses depressive and anxiety symptoms (Löwe et al., 2010). This ultra-brief version contains 4 items with a 4-point Likert response format, where 0 corresponds to “not at all” and 3 to “nearly every day”. The scores of PHQ-2 (depression) and GAD-2 (anxiety) range from 0 to 6, with higher scores indicating higher depressive and anxiety symptoms (Caro-Fuentes & Sanabria-Mazo, 2023). The Colombian version (Sanabria-Mazo et al., 2023) of PHQ-4 showed adequate internal consistency for depression (Cronbach’s a = .79) and anxiety (a = .83). In this sample, the PHQ-4 showed adequate internal consistency for depression (a = .85) and anxiety (a = .84).
Post-traumatic Growth Inventory (PTGI-5). It assesses post-traumatic growth (Tedeschi & Calhoun, 1996). Specifically, the 5 items (i.e., relationship with others, new possibilities, personal strength, spiritual change, and appreciation for life) with the highest factor loadings were evaluated in a principal component analysis (PCA). The 5 items are rated with a 4-point Likert response format, where 0 corresponds to “not at all” and 3 to “nearly every day”. The total score of the PTGI-5 ranges from 0 to 15, with higher scores indicating higher post-traumatic growth. The Colombian version of PTGI-5 (Gómez-Acosta et al., 2023) showed adequate internal consistency (a = .79). In this sample, the PTGI-5 showed adequate internal consistency (a = .83).
Connor-Davidson Resilience Scale (CD-RISC-2). It assesses resilience (Connor & Davidson, 2003). This short version contains 2 items with a 4-point Likert response format, where 0 corresponds to “not at all” and 3 to “nearly every day”. The total score of CD-RISC-2 ranges from 0 to 6, with higher scores indicating higher resilience. In this sample, the CD-RISC-2 showed adequate internal consistency (a = .77).
Statistical analysis
A descriptive analysis of sociodemographic variables was conducted using frequencies (n) and percentages (%), for categorical variables, and means (M) and standard deviations (SDs), for continuous variables. Characteristics of the PSS-4 were explored, including item means and standard deviations, skewness and kurtosis, and corrected item-total correlations. These correlations were analysed using the Spearman-Brown correction. No participants were excluded due to missing data.
The dimensionality of the PSS-4 was tested through PCA and confirmatory factor analysis (CFA), using maximum likelihood robust (MLR). The following factor models were tested: (1) a one-factor model (psychological stress) with the four items loading on one latent factor, (2) a one-factor model (psychological stress) with method effects (correlated error terms on the negative-phrased items), and (3) a two-factor model (hopelessness and coping) with two items in each factor. The Tucker-Lewis’s Index (TLI), Normed Fit Index (NFI), and Comparative Fit Index (CFI) were used to evaluate goodness-off, with >. 90 confidence intervals and the Root Mean Square Error of Approximation (RSMEA) < .08. The invariance (configural, metric, scalar, and strict) was tested by gender, age, income level, and work status in comparable subsamples with random assignment. To determine measurement invariance, multigroup CFA was conducted, observing a change of ΔCFI that is less than or equal to .01. Additionally, factor loadings equal to or greater than .30 were accepted for factor analysis.
The reliability of the PSS-4 was assessed through McDonald’s w, Cronbach’s a, and Guttman’s l2. Construct validity analysis (convergent and discriminant) was performed using Spearman’s rho statistic, considering the nonparametric distribution of the data. PSS-4 scores were correlated with other mental health indicators, such as resilience (CD-RISC-2) and post-traumatic growth (PTGI-5), as well as anxiety (GAD-2) and depression (PHQ-2). For assessing predictive validity, a hierarchical multiple regression analysis was performed to examine whether perceived stress was a significant predictor of resilience (CD-RISC-2), post-traumatic growth (PTGI-5), anxiety (GAD-2), and depression (PHQ- 2). Sociodemographic variables (gender, age, and income level) were entered in the first step and PSS-4 scores were entered in the second step.
Finally, a known-groups validity approach was used to estimate associations between PHQ-4 scores and socio-demographic characteristics (gender, age, income level, and work status). Univariate group comparisons were performed with the PSS-4 scores as dependent variables through t-tests and one-way analysis of variance (ANOVA). Statistical analyses were performed with SPSS with AMOS29®, JAMOVI 2.3®, and JASP 0.17.1.®.
Results
Item analysis
Table 2 shows descriptive analyses of PSS-4 items and the total scale. The mean (SD) score of PSS-4 was 3.77 (2.61) and corrected item-total correlation coefficients were all greater than .44 (p < .01), suggesting good homogeneity.
Table 2 Characteristics of the items and subscales of the PSS-4
Note. n = 1991. PSS-4 = Perceived Stress Scale. M = mean; 95% CI = 95% confidence interval; SD = standard deviation; S = Skewness; K = Kurtosis. rtot = corrected item-total correlations. Items scores can range from 0 (“not at all”) to 3 (“nearly every day”).
Dimensionality
The total sample was randomly divided into two subsamples to examine the PSS-4 factor structure. A PCA was computed with the first subsample and CFA with the second subsample. The sample size for both the PCA and the CFA was adequate since the recommendation of a minimum of 10 participants per item could be met. Kaiser-Meyer-Olkin analyses indicated that the first subsample was significantly sufficient for factor analysis. Consequently, a PCA (with Promax rotation) was performed, which showed that the items were grouped into a one-factor model that explained 56% of the total variance. Nevertheless, when conducting the CFA, the PSS-4 was found to have inadequate indicators in the one-factor model. Then the one-factor model was recomputed with method effects (correlated residual between the two negatively phrased items).
The fit indices for the correlated two-factor model were significantly better (p < .001) than the one-factor model with the four items loading on one latent factor; however, they were not significantly better than those obtained for the one-factor model with method effects (correlated error terms on the negative-phrased items). The comparison of the fit indices is displayed on Table 3. Considering these results, the one-factor model was retained for the subsequent analysis.
Regarding factor loadings of the three tested factor models, the one-factor model ranged between .64 and .77, the one-factor model with method effects between .32 to .85, and the two-factor model between .59 to .93. Responding to hypothesis 1, these findings confirm that the one-factor model with method effects is the most optimal model according to fit indices and parsimony considerations (see Figure 1). For this reason, this model was used to conduct the invariance analysis.
Configural invariance
As shown on Table 4, comparable subsamples with random assignment were used to evaluate configural invariance by socio-demographic characteristics. No structural differences were found in the best-fitting model according to gender and age. However, the PSS-4 scores were significantly varied in all models (configural, metric, scalar, and strict) according to income level and work status. These results confirm hypothesis 2.
Reliability
The reliability was adequate for the PSS-4 total scale (a = .73, w = .72, l2 = .74), considering its length (only 4 items). The reliability indices of the PSS-4 confirm hypothesis 3.
Construct validity
Significant positive correlations of the PSS-4 were identified with depression (r = .59) and anxiety (r = .55), as well as significant negative correlations with post-traumatic growth (r = -.37) and resilience (r = -.47). These indicators rule out collinearity between the measures explored (see Table 5). All these associations were significant at p < .01. These findings are consistent with hypothesis 4.
Predictive validity
As shown on Table 6, hierarchical multiple regression analysis showed that elevated PSS-4 scores significantly (p < .001) predicted depression (R2 = .39, Beta = 0.38) and anxiety scores (R2 = .34, Beta = 0.37). PSS-4 low scores significantly (p < .001) predicted resilience (R2 = .24, Beta = -0.21) and post-traumatic growth (R2 = .15, Beta = -0.47). Although the sociodemographic variables contribute to the prediction in the first step of the model, their contribution to the explained variance was not significant. These results confirm hypothesis 5.
Known-groups validity
A comparative analysis of the PSS-4 was carried out by socio-demographic characteristics. As shown on Table 7, statistically significant differences were found according to age, income level, and work status, but not according to gender. Significantly higher scores on the PSS-4 were observed among young people, as well as respondents with lower incomes and those who are unemployed. Contrary to hypothesis 5, young unemployed people scored higher on the PSS-4 than older employed people. People with lower income levels scored higher on the PSS-4, which is consistent with hypothesis 6.
Discussion
This study evaluated the psychometric properties of the PSS-4 in a Colombian sample. The first objective was to examine the factorial structure. Although some previous studies have reported a better fit to the two-factor model (hopelessness and coping) in the PSS-4 (Mondo et al., 2021; Ruisoto et al., 2020; She et al., 2021), in this sample the one-factor model (psychological stress) with method effects (correlated error terms on the negative-phrased items) was the most appropriate according to fit indices and parsimony considerations. This finding is consistent with the original proposal of Cohen et al. (1983), who conceived the PSS as a one-factor model. Previously, Cohen et al. (1983) noted that the PSS items tended to be grouped according to their positive or negative sense, which could reveal a response bias rather than a multidimensional structure in factor analyses. The psychometric problem of “method effects” (or “item wording”), reported in this study, has been widely addressed in the scientific literature (van Sonderen et al., 2013).
The second objective was to investigate the invariance (configural, metric, scalar, and strict) of the best-fitting model across socio-demographic characteristics. The PSS- 4 was invariant according to gender and age, but not according to income level and work status. Invariance was detected by gender and age, being consistent with previous validations of PSS-10 (Barbosa‐Leiker et al., 2013; Lee, 2022; Liu et al., 2020; Reis et al., 2019). The reported differences could occur because this study’s sample presented some differences in age, income level, and work status, which could indirectly affect the factorial invariance analysis (Chen, 2007). The one-factor model with method effects was invariant by gender and age in this sample, supporting hypothesis 2.
The third and fourth objectives were to determine the reliability and construct validity of the PSS-4, respectively. The internal consistency was acceptable (greater than .70), with better results than those reported by Kim (2016) for the Korean version, by She et al. (2021) for the Chinese version, by Mondo et al. (2021) for the Italian version, and by Ruisoto et al. (2020) in Ecuador for PSS-4, but lower than those achieved in Colombia for the PSS-14 and PSS-10 (Campo-Arias et al., 2014) in samples of participants with a university education. It’s well known that Cronbach’s alpha is influenced by the number of items in the tested scale; for shorter tests, it has naturally reduced values. Consistent with available evidence, the convergent validity between psychological stress scores with anxiety and depression (Perera et al., 2017), as well as the discriminant validity with post-traumatic growth (Gómez-Acosta et al., 2023) and resilience (Ruisoto et al., 2020) was in the expected direction. These findings are consistent with hypotheses 3 and 4.
The fifth objective was to evaluate the predictive validity of the PSS-4 on a set of psychological variables (resilience, post-traumatic growth, anxiety, and depression). The PSS-4 has an adequate capacity to predict possible depressive and anxiety symptoms (Feng et al., 2023), as well as acceptable levels of prediction of protective factors such as resilience (Finstad et al., 2021) and post-traumatic growth (Gómez-Acosta et al., 2023), confirming hypothesis 5. Accordingly, the ultra-brief version of the PSS-4 could be used as an input to obtain possible indicators of mental health impairment and to develop primary care actions (Stringer, 2023).
Finally, the sixth objective was to test the relationship between PPS-4 scores and the sociodemographic characteristics of this sample. This study provides further evidence of gender, age, income level, and work status as risk factors for psychological stress. Consistent with hypothesis 6, it was observed that people with higher educational levels presented lower scores in psychological stress (Ruisoto et al., 2020; Vallejo et al., 2018; Warttig et al., 2013). Contrary to what was hypothesised, it was observed that young unemployed individuals reported higher scores than older employed individuals. These findings could be explained considering the large amount of empirical evidence reflecting the negative effect generated by the COVID-19 pandemic on mental health (Newnham et al., 2022).
Limitations
The following limitations should be kept in mind when interpreting these results. Firstly, the analyses were performed on a non-representative sample, which prevents the generalisation of these results to the population with a university education. Secondly, the sampling (aimed exclusively at adults with a university education) limited the representation of people with other sociodemographic characteristics. Thirdly, the temporal stability was not calculated, which could help to explore the performance of this instrument over time. Fourthly, the impossibility of comparing the psychological stress scores obtained with the self-report with biomarkers under controlled conditions so that the scale sets the optimal cut-off point. Finally, the online data collection may negatively affect the representation of population groups with difficulties connecting to the Internet (self-selection bias). To improve the reliability and validity of the test, these limitations (mainly financial and lo-gistical, as well as related to the moment of administration of the forms) must be corrected in subsequent studies.
Conclusions
The PSS-4 is a reliable and valid ultra-brief self-administered instrument for measuring psychological stress in the Colombian population with a university education. The CFA showed that the one-factor model (psychological stress) with method effects (correlated error terms on the negative-phrased items) was the most appropriate according to fit indices and parsimony considerations, being invariant across gender and age, but not across income level and work status. The internal consistency was adequate for the PSS-4. The convergent and discriminant validity, as well as the predictive validity, explored was consistent with the theoretically expected direction. Significantly higher scores on the PSS-4 were observed among young people, as well as among people with lower incomes and the unemployed. The PSS-4 can be useful as a screening tool for psychological stress in clinical and non-clinical populations, especially as a complement to the assessment and monitoring processes, considering its adequate psychometric properties1 2 3 4 5.
