Introduction

Adolescence is a stage of development in which biological, physical and psychological changes take place.Also, it is the period of youth's identity formation, when a safe environment and social life are important (^{Santrock, 2014}). The relationship between peers makes easier the feeling of belonging to a group and the good adjustment in adult life. However, the establishment of unsatisfactory relationships contributes negatively to the teenager's future interpersonal relationships (^{Sprinthall & Collins, 2008}).

Violence and aggression among peers have been the focus of attention nowadays and are a problem discussed in the media, in schools and the whole society. This phenomenon is defined as 'bullying' and can be manifested in many ways (^{Calbo, Busnello, Rigoli, Shaefer, & Kristensen, 2009}; ^{Silva & Cabral, 2014}). Typically, it is characterized by intentional and repeated physical and social actions that are perpetrated by one or more people against an individual who cannot easily defend himself (^{Dellazzanna, Sattler, & Freitas, 2010}; ^{Stelko-Pereira & Williams, 2012}; ^{Wassen & Nickerson, 2017}). In Brazil, the 2015 National School Health Survey (PENSE) showed that the prevalence of bullying victims at schools was 7.4%, with a greater chance of happening in boys, aged 13, in public school students, in adolescents who reported loneliness (those who have no friends), who had insomnia, who suffered physical aggression from family members, who had a history of missing class without notifying their parents, and who used tobacco (^{Malta et al., 2019}).

In general, bullying involves physical behaviors such as hitting, pushing, punching, stealing snacks, using weapons to attack; verbal behaviors with threats and curses, and nicknames; and relational behaviors, as the spread of gossip and lies that end up damaging the relationship with peers. There is also indirect bullying, which involves exclusion, indifference and extortion actions. Roles in the bullying scenario are also classified: aggressor, victim, aggressor/ victim and witness (^{Bandeira & Hutz, 2012}; ^{Dellazzanna et al., 2010}; Hui, Tsang, & Law, 2011; ^{Stelko-Pereira & Williams, 2012}). The aggressor is the child or teenager who practices bullying on a weaker child or adolescent (victim) who, generally, does not have many physical and emotional conditions to defend himself. The witness is the one who observes bullying situations. Also, there are differences in the way bullying is perpetrated by gender: physical bullying is more common among boys and relational bullying among girls (^{Bandeira & Hutz, 2012}).

Although studies on bullying are recent, there are already signs of the immediate or late consequences of direct and indirect (witness) bullying in children and adolescents' lives. Bullying is known as a common problem in the life of school-age teenagers and is related to care and health because it can develop some long-term pathologies (^{Oliveira, Silva, Yoshinaga, & Silva, 2015}). Some emotional damage was identified in bullying victims when compared to other adolescents who did not experience this intimidation, such as low self-esteem, depressive symptoms (^{Sino et al., 2014}) and a greater degree of anxiety (^{Malecki et al., 2015}). In the most severe cases, the effects are so damaging that they result in suicide attempts (^{Silva & Cabral, 2014}).

The bullying consequences can already be noticed in elementary school. Negative feelings are prevalent, such as lack of motivation to go to school or to change school, fear, sadness, and the eagerness to attack the aggressor, which can cause a considerable impact on the psychological, physical, social and learning areas (^{Santos, Perkoski, & Kienen, 2015}). Additionally, in many teenagers bullying triggers depression and anxiety disorders, a higher risk of developing suicidal ideation and even suicide attempts (^{Moore et al., 2017}). The most prevalent effects in common victims are depression and anxiety disorders. There is a high probability correlation for suicidal ideation and even suicidal attempts. The effects are so harmful that they end up lasting even after bullying has stopped (^{Arseneault, 2017}). The literature also indicates that there is a relationship between bullying and the development of symptoms of Post Traumatic Stress Disorder (PTSD) (^{Albuquerque, Albuquerque, Williams, & Affonseca, 2013}).

Considered a public health problem, bullying should be discussed together, both by school and families, emphasizing the importance of respect and tolerance to create a healthy living environment (^{Alves, 2016}; ^{Menegotto, Pasini, & Levandowski, 2013}; ^{Silva & Costa, 2016}). Above all, it is necessary to understand the bullying phenomenon and its harmful effects. Therefore, it is everyone's duty, especially parents (^{Hale, Fox, & Murray, 2017}) and educational institutions, to cherish the well-being of children and teenagers (^{Segundo et al., 2016}).

Thus, the present study aimed to investigate the correlation between the victimization of bullying, symptoms of depression, anxiety, stress, and suicidal ideation in teenagers. Also, to identify whether there is a difference in the variables investigated by gender and age.

Method

Instruments

Sociodemographic Questionnaire: developed by the researchers to collect sociodemographic data such as gender, marital status, age, education, race, family, psychological or psychiatric treatment, health problem, among others.

California Bullying Victimization Scale (ECVB): adapted by ^{Soares, Gouveia, Gouveia, Fonseca, and Pimentel (2015)}, consisting of seven questions, which indicate the frequency of bullying behaviors experienced in the school context, using a Likert scale from 0 (never) to 4 (several times). Later, the instrument assesses whether the behaviors were intentional and capable of causing suffering, through the answer options "yes" and "no". At the end, the scale assesses the imbalance of power between the victim and the aggressor using 10 adjectives (example: handsome, popular, among others), asking the respondent to compare himself with the main person who was bullied, using the options: "less than me", "like me", and "more than me". The study that assessed the instrument's construct validity and internal consistency obtained a Cronbach's alpha α = 0.72. The same investigation presented a confirmatory factor analysis with a single factor structure. In general, the validity study showed a reliability level of 0.78 (Soares et al., 2015).

Table 1 Sociodemographic data of the sample 

Depression, Anxiety and Stress Scale, adolescent version (EDAE-A): adapted and validated for Brazilian adolescents from the Depression, Anxiety and Stress Scale - Short Form (DASS-21) version used for Brazilian adults (^{Machado & Bandeira, 2013}). The scale has 21 items that assess symptoms of depression, anxiety, and stress. The participant indicates the degree to which he/she experienced the symptom in the last week, being 0 (It didn't happen to me this week) to 3 (It happened to me most of the time this week). The study that adapted the scale for adolescents tested three models that were available in the literature to verify the dimensionality of the EDAE-A. The model that best fitted was that of the original scale, that is, the three factors representing the dimensions of depression, anxiety and stress [χ2 = 366.16 (186), p <0.001, CFI = 0.96, TLI = 0.96, RMSEA = 0.047 (0.040 - 0.054)]. The internal consistency of the scale showed, for each factor, the following Cronbach alphas: 0.86 stress, 0.83 anxiety and 0.90 depression (^{Patias, Machado, Bandeira, & Dell'Aglio, 2016}).

Beck Scale for Suicide Ideation (BSSI): composed of 21 items, scored from 0 to 3, which detects the presence of suicidal ideation and assesses the extent of motivation and planning of suicidal behavior. The last two items are not included in the final score, but provide qualitative information regarding suicide attempts and intention to die. The scale does not have a cut-off point, so the score of the 19 items is added, and thus, the higher the final count, the greater the risk of suicidal ideation. Psychometric studies revealed a Cronbach's alpha of 0.96 (^{Cunha, 2001}). The BSSI reliability estimate, based on Cronbach's alpha coefficient, in a sample of adolescents (non-clinical), was 0.94 (^{Werlang, Borges, & Fensterseifer, 2004}).

Results

The frequency analysis of the victimization behaviors of intentional bullying and having caused suffering indicated that most teenagers affirmed having been excluded from the group (41%), having been victims of rumors or gossip behind their back (30%), and having been victims of harrassment or nicknaming by classmates (17%), as shown in Table 2.

The results about the characteristics of peers who performed the behaviors indicated above were: regarding the adjective 'smart', 31.6% answered similar to me; 'physically strong', 32.5% answered similar to me; 'handsome', 30.8% looked like me, and 'nice' 39.3% said looked like me. Further, the highest score for similar to me was in the items 'partner' (41%), 'extrovert' (38.5%), and 'intelligent' (31.6%). The answer less than me was checked for the features 'attractive' (32.5%) and 'thin' (34.2%).

When comparing the statements by gender, girls reached a higher score in those assertions that characterize relational bullying: "Were you the victim of rumors or gossip behind your back by your colleague (s)?" (U = 948,500, p <0.01), "Were you excluded from the group or ignored by your colleague(s)?" (U = 1094.00; p = 0.04) and "Were you a victim of sexual comments or corresponding gestures?" (U = 1137.00, p = 0.03).

Table 2 Comportamiento de victimización de bullying intencional y que hizo daño a la víctima 

The BDI results showed that 57 adolescents (48.71%, M=1.75, SD=2.34) have had suicidal ideation. The scores for statements 1 to 19 were added up to assess the severity of suicidal ideation. The variation in scores was from 3 to 28, with a mean of 12 (SD=6.64). The comparison of severity in suicidal ideation by gender showed that girls reached more severity than boys (U = 1030.00, p = 0.03).

Correlation analyzes were performed between the constructs investigated (symptoms of depression, anxiety, and stress, suicidal ideation, and victimization of bullying) and the participants' age. The Spearman test indicated a single correlation between age and victimization of bullying. The negative correlation suggests that with increasing age there is a decrease in victimization, or the younger the teenager, the greater the victimization (rho = -0.19; p = 0.04). However, this correlation must be interpreted carefully considering the magnitude (weak correlation) and the sample size.

The Mann-Whitney U test was performed to check for differences in the variables investigated among boys and girls. The only difference found was between gender and stress symptoms, with girls showing more symptoms than boys (U = 1035,500; p = 0.02).

The Spearman test was used to verify the correlation between symptoms of depression, anxiety, and stress, bullying victimization and suicidal ideation. The analysis indicated a positive and moderate correlation between suicidal ideation and symptoms of depression, anxiety, stress, and between suicidal ideation and bullying victimization, as shown in Table 3.

Discussion

The present study aimed to investigate possible correlations between bullying victimization, symptoms of depression, anxiety, stress, and suicidal ideation in teenagers. Also, to check if there is a difference between the variables investigated by gender and age. The ECVB scores, which assessed bullying victimization, range from 0 to 28 points, and in this study they ranged from 0 to 16 points (M = 3.88; SD = 3.76). A similar study used this same instrument and found that 40.6% of the sample claimed to be a victim of bullying (^{Lima, 2013}). Another survey that used the Social Exclusion and School Violence Questionnaire found an average of 18.34 points (SD = 3.86) for victimization (^{Simões, Ferreira, Braga, & Vicente, 2015}).

A Canadian study evaluated the medical records of patients under 18 with mental health complaints in the emergency department of two hospitals and found that 77% of them had been bullied at some point in their lives (^{Alavi et al., 2017}). An American survey of patients in the pediatric emergency with an indication for psychiatric hospitalization found that 13% reported being frequent victims of bullying (2-3 times a month or more) (^{Leader, Singh, Ghaffar, & Silva, 2018}). Another survey, with students from schools in Bolivia, Costa Rica, Honduras, Peru, and Uruguay, found that 37.8% of the participants had been victims of any type of bullying in the last 30 days (^{Romo & Kelvin, 2016}).

A Brazilian survey with teenagers aged 11 to 16 years used the Kidscape questionnaire and showed that almost half of the sample (48.22%) declared having been the target of bullying, mostly verbal (46.56%) (^{Garbin, Gatto, & Garbin 2016}). In a similar study, also conducted in Brazil, 76.75% of students reported having been victims of this type of aggression (^{Paixão et al., 2014}). Another investigation, carried out in Southern Brazil, found that 67.9% of students declared to be victims of bullying (^{Fernandes & Yunes, 2017}). Together, the studies conclude that many adolescents are being victims of bullying. This conclusion is worrying since victimization has consequences for youths.

The results of the present study found that with increasing age there is a decrease in victimization. This datum corroborates the study by ^{Malta et al. (2014)} who found a higher prevalence of bullying in students under the age of 15. Another survey showed that 13-year-old students reported having suffered more bullying compared to other age groups and that this attitude decreased after the age of 14 (^{Malta et al., 2019}).

The exposure of adolescents to acts of bullying is increasingly precocious; however, there is a tendency for this phenomenon to decrease as age increases (^{Moreno et al., 2012}). An explanatory hypothesis for this finding is that older students learn to defend themselves (^{Melim & Pereira, 2013}). Also, younger adolescents may report greater victimization than older ones, since the scale is a self-report measure.

Table 3 Spearman correlations between symptoms of depression, anxiety, stress, bullying victimization and suicidal ideation 

Note. ** Correlation is significant at the 0.01 level (two-tailed).

Regarding the behavior of intentional bullying victimization that caused suffering, most of the participants indicated that they were excluded from the group and were victims of rumors or gossip behind their backs. The findings corroborate the results of another research that used ECVB and found that the most frequently reported behaviors were: having been harassed or ignored (60.9%), having been victim of rumors and gossips (42.8%), and having been excluded from the group (42%) at some point in school life (^{Cavalcanti, Coutinho, Pinto, Silva, & Do Bú, 2018}). In the study by ^{Leader et al. (2018)}, victims also referred to the exclusion and spread of rumors as the most frequent ways of bullying.

According to the literature, exclusion by colleagues, jokes and damaging rumors are the major reasons for psychological distress in victims of bullying (^{Agência Brasil, 2017}). One study found that groups that reject victims are made up of colleagues and friends (33%) who leave them out of sports teams or school works. The short- and long-term consequences for the victims are the limitations of living spaces, the difficulty to socialize, low school performance and low self-esteem (^{Santos & Santos, 2013}). The group plays an important role in teenagers' life since it is an essential part of their identity development, which foresees to distancing from parents and getting closer to peers, in order to assume different rules and values (^{DeSousa, Rodriguez, & De Antoni, 2014}).

Studies reveal that verbal bullying is more common than physical (^{Alavi et al., 2017}; ^{Bandeira, Moraes, & Hutz, 2012}, ^{Garbin et al., 2016}; Marcolino et al., 2018; ^{Vieira, Torales, Vargas, & Oliveira, 2016}). In the present study, some teenagers revealed that they felt distressed about the "annoyance of nicknaming by their colleagues". Verbal aggression is a type of covert violence that in many cases can be understood as a joke. Thus, it is more difficult to identify and intervene in this type of conduct. However, verbal bullying should not be seen or minimized as a lower risk behavior, for it causes intense suffering to the victim (^{Cavalcanti et al., 2018}).

The majority of adolescent victims of bullying in this study marked the alternative answer "similar to me" to indicate that they had similar features to the aggressors, which corroborates the research by ^{Bandeira, Moraes, and Hutz (2012)}, in which 54.7% of the students identified themselves with the aggressors. Usually, victimization goes hand in hand with low self-esteem in teenagers (^{Brito & Oliveira, 2013}), resulting in feelings of inferiority concerning the aggressor (^{Santos & Santos, 2013}). The answer "less than me" was scored only on the 'slim' and 'attractive' features. This result can be justified by the fact that the instrument is a self-report measure, which can inhibit the true expression of the adolescent's feelings. Also, it is possible to think whether the victim could also be an aggressor.

The findings regarding bullying and gender were also evidenced by other studies, which found a higher prevalence of physical aggression in boys, and other more subtle forms of aggression in girls, such as verbal offenses, insults, and gossips (^{Martins, 2009}; ^{Moreno et al., 2012}). Another survey found the following percentages of victimization by bullying: verbal: girls 92% and boys 84%; psychological: girls 70% and boys 53%; and physical: girls 46% and boys 44% (^{Brito & Oliveira, 2013}). Also, another study found a significant difference between genders, showing that boys use more shoves, kicks and punches, and girls more lies and gossip (^{Bandeira et al., 2012}).

This difference can be understood from a cultural and recreational point of view. Girls tend to acquire social skills of empathy and recognition of voices and faces since they are very young, and their games usually involve dolls and houses, associated with social and verbal interactions. Boys, in turn, tend to play more with toy cars, balls, and fights, emphasizing more physical and mechanical activities, which is culturally accepted and associated with the male gender (^{Seixas, 2009}).

The difference found in stress symptoms between genders is corroborated by studies that indicate more stress symptoms in girls than in boys (^{Landstedt & Gadin, 2012}; ^{Liu & Lu, 2012}; ^{Machado, Veiga, & Alves, 2011}; ^{Marques, Gasparotto, & Coelho, 2015}). An investigation with 106 adolescents from a Brazilian public school assessed stress and school violence and concluded that girls had a higher average in stress reactions when compared to boys (p <0.01) (^{Sousa & Stelko-Pereira, 2016}).

An important aspect is a relationship between stress in high school and the performance of academic activities in the female gender (^{Marques, Gasparotto, & Coelho, 2015}). In this perspective, a study investigated stress symptoms in teenagers using the Adolescent Stress Scale (ESA) and found a prevalence of 10.3% in boys and 11.4% in girls. Additionally, the study identified that the quality of the adolescent's relationship with parents (when they appreciate and become interested in their children's activities) is an essential component to avoid these symptoms in childhood (^{Schermann et al., 2014}).

The results regarding stress symptoms and gender are in line with the scale validation study for Brazilian adolescents (^{Patias et al., 2016}). On the other hand, the results concerning symptoms of anxiety and depression are different. This variation may indicate that boys are currently experiencing symptoms that in previous research have been identified only or mainly in girls. Also, the fact that no difference was found in the symptoms of anxiety and depression between genders may be related to the small sample size (n=117) and the difference between groups, by gender (29% boys).

The BSI results in this study showed that almost half of the sample had suicidal ideation. The data corroborates findings of other studies that used the same scale and found prevalences between 31.9% (^{Borges, Werlang, & Copatti, 2008}) and 36% (^{Borges & Werlang, 2006}). A study in Peru showed that 20% of teenagers were at risk of suicide, 15.2% had a history of attempted suicide, 35.3% had a passive desire to die and 13.2% had planned suicide at some point in life (^{Sandoval Ato, Vilela Estrada, Mejia, & Caballero Alvarado, 2018}). The literature reveals that the prevalence of suicidal ideation in adolescents evaluated in a pediatric emergency unit is higher, reaching 68.9% (^{Alavi et al., 2017}).

Suicide is a major cause of death in the age group between 15 and 19 years old (^{Moreira & Bastos, 2015}). The high rate of suicide in adolescents is justified on the psychological pressure suffered by them at this phase. Many of them believe that taking their life is the only way to relieve suffering (^{Barbosa, Parente, Bezerra, & Maranhão 2016}).

Loneliness stands out as the main risk factor for suicide in adolescence, followed by the absence of parents, the influence of friends, the use of alcohol and drugs (^{Ulbrich, Oselame, Oliveira, & Neves, 2017}). Other elements such as family conflicts, problems related to school, bullying, impulsivity, and depressive symptoms are also associated with adolescent suicide (^{Sousa et al., 2017}). However, the correlation between bullying and suicidal ideas, mainly because of the intensity in which young people experience their life in this stage of development, can lead to self-harm outcomes. Therefore, prevention against different forms of bullying must be done to reduce this rate (^{Holt et al., 2015}).

This research's results also found that both victimization and suicidal ideation have a moderate and positive correlation with symptoms of depression, anxiety and stress. In addition, suicidal ideation is correlated with bullying victimization. According to ^{Klomek et al. (2013)}, bullying in high school is among the risk factors for future psychiatric problems and indicates a high tendency to depression and suicide. A survey found a strong correlation between victimization and adolescent psychiatric hospitalization (^{Leader et al., 2018}). The results by ^{Cavalcanti et al. (2018)} confirmed a positive correlation between victimization and depressive symptoms (p <0.01) and the study by ^{Pabian and Vandebosh (2016)} found significant positive correlations between victimization and social anxiety (p <0.001). Another study found that being a victim of bullying is associated with symptoms of depression and anxiety (p <0.001) (^{Stapinski, Araya, Heron, Montgomery, & Stallard, 2015}).

Depressive symptoms can be one of the consequences of the violence experienced by young people, and can be manifested through nervousness, negative thoughts, crying, and ideational and attempted suicide (Silva, 2010). Depressive symptoms can also be a warning signal that the young person is the target or the author of bullying (^{Forlim, Stelko-Pereira, & Williams, 2014}). Adolescent bullying can trigger depressive and anxiety disorders, and is associated with suicide attempts or greater chance of suicidal ideation (^{Moore et al., 2017}).

The positive, significant and moderate correlation found between suicidal ideation and bullying victimization is in line with other studies that show a significant correlation between bullying and suicide risk or suicidal ideation (p <0.001) (^{Alavi et al., 2017}; ^{Leader et al., 2018}; ^{Sandoval Ato et al., 2018}). Another study concluded that victimization is associated with an increased risk of suicidal ideation with planning and at least one attempted suicide (p <0.0001) (^{Romo & Kelvin, 2016}). These findings are relevant and highlight the need to prevent bullying due to the magnitude of the consequences for everyone involved.

This research aimed to investigate the correlation between bullying victimization and symptoms of depression, anxiety, stress and suicidal ideation in teenagers. The study reveals worrying data regarding bullying victimization, prevalence of symptoms of depression, anxiety, and stress, especially in girls, and high rates of suicidal ideation in the sample. These data contradict the belief of common sense and of many professionals who consider bullying as a "child's play".

Thus, the results of the study demonstrate the importance of having family and school intervention not only for victims of bullying, but for the school community in general, as witnesses and aggressors also suffer from the phenomenon. However, this study focused on victimization and indicated that being a victim of bullying is a serious matter and should not be overlooked due to the significant impact it brings to young people's lives, such as psychological damage, both emotional and behavioral.

As a limitation of this study, it is possible to highlight the fact that the scales used are self-report measures, which can make difficult the true expression of the adolescent's emotions. Also, only bullying victimization was investigated and it is important to include adolescent aggressors and spectators as part of future research on the subject.

One aspect that drew attention was the fact that not all parents consented to the participation of their children, even if the adolescents wanted to be part of the research. Although it was not intended, in the present study it was possible to reveal the suicidal ideation of many adolescents who were referred to school services. For future studies, the assessment of symptoms of depression, anxiety, stress and suicidal ideation in teenagers who practice bullying and in those who witness it is suggested. The investigation on school interventions in the face of these situations is another important aspect to be tackled. Still, it is important for future studies to expand the sample size and also have a better gender distribution in the groups.