Doi: http://dx.doi.org/10.12804/apl33.01.2015.10

Social Skills Rating System - Brazilian Version: New Exploratory and Confirmatory Factorial Analyses

Versión Brasileña del Sistema de Evaluación de Habilidades Sociales: Nuevo análisis factorial exploratorio y confirmatorio

Social Skills Rating System - Versão brasileira: Novas análises fatoriais exploratórias e confirmatórias

Lucas C. Freitas, Zilda A. P. Del Prette*

* Department of Psychology, Universidade Federal de São Carlos, São Carlos, São Paulo, Brasil.

This study was funded by The State of São Paulo Research Foundation (FAPESP) Correspondence concerning this article should be addressed to Lucas Cordeiro Freitas, Rua Senador Rui Palmeira, 260, apartment 707, Ponta Verde. 57035-250. Maceió, AL, Brazil. Email: lcordeirofreitas@yahoo.com.br

To cite this article: Freitas, C. L., & Del Prette, A. P. Z. (2015). Social Skills Rating System- Brazilian version: New Exploratory and Confirmatory Factorial Analyses. Avances en Psicología Latinoamericana, 33(1), 135-156. doi: http://dx.doi.org/10.12804/apl33.01.2015.10.

Received: April 3, 2014. Accepted: September 17, 2014

Abstract

This study aimed to verify the Factorial Structure of the Brazilian version of the Social Skills Rating System (SSRS-BR), with an extended sample of participants, and their subsequent Confirmatory Factor Analyses in another subset of data. The analyses were based on a total sample of 942 evaluations of children between six and thirteen years of age, 817 evaluations of teachers and 562 evaluations of parents, residents of four Brazilian states. The exploratory factor analysis performed on half of the data, indicated a five-factor structure for the social skills scale for parents, and a four factor structure for the scales of the teachers and students. For the behavior problems scales, three factors were found in the instrument for parents and two factors in the instrument for the teacher. Confirmatory factor analysis showed satisfactory fit indices for the three instruments, after removing some items and performing some re-specifications.

Keywords: Social skills, behavior problems, academic competence, evaluation scale, factor analysis.

Resumen

Este estudio tuvo como objetivo verificar la estructura factorial de la versión brasileña del Social Skills Rating System (SSRS -BR), con una muestra ampliada de participantes y su posterior análisis factorial confirmatorio en otro subconjunto de datos. Los análisis se basan en la muestra total de 942 evaluaciones de los niños entre seis y trece años de edad, 817 evaluaciones de maestros y 562 evaluaciones de padres, residentes de cuatro estados brasileños. El análisis factorial exploratorio realizado en la mitad de los datos indicó una estructura de cinco factores para la escala de habilidades sociales para los padres y una estructura de cuatro factores para las escalas de los maestros y estudiantes. Para las escalas de problemas de conducta, se encontraron tres factores en el instrumento para los padres y dos factores en el instrumento para el maestro. El análisis factorial confirmatorio demostró índices de ajuste satisfactorio para los tres instrumentos, después de la eliminación de algunos ítems y la realización de algunas especificaciones.

Palabras clave: Habilidades sociales, problemas de conducta, competencia académica, escala de evaluación, análisis factorial.

Resumo

Este estudoteve como objetivo verificar a estruturafatorial da versão brasileira do Social Skills Rating System (SSRS -BR), com uma amostra ampliada de participantes, e sua posterior análise fatorial confirmatório em outro subconjunto de dados. As análises se baseiamna amostra total de 942 avaliações das crianças entre seis e treze anos de idade, 817 avaliações de professores e 562 avaliações de pais, residentes de quatro estados brasileiros. A análisefatorialexploratória realizada nametade dos dados indicouumaestrutura de quatrofatores para as escalas dos maestros e estudantes. Para as escalas de problemas de conduta, encontraram-se trêsfatores no instrumento para os pais e doisfatores no instrumento para o professor. A análise fatorial confirmatória demonstrou índices de ajuste satisfatório para os três instrumentos, depois da eliminação de algunsitens e a realização de algumas especificações.

Palavras-chave: Habilidades sociais, problemas de conduta, competência académica, escala de avaliação, análisefatorial.

Social skills can be considered to be an important variable for childhood development, especially due to its positive association with several indicators of adaptive functioning, such as: satisfactory relationships with peers and adults (Del Prette & Del Prette, 2005), good academic performance (Caprara, Barbaranelli, Pastorelli, Bandura & Zimbardo, 2000; Del Prette, Del Prette, Almeida, Gresham, & Vance, 2012), positive social status in the group (Molina & Del Prette, 2007), and a lower frequency of internalizing (Rubin, Coplan & Bowker, 2009) and externalizing behavior problems (Sorlie, Hagen & Ogden, 2008).

According to Del Prette and Del Prette (2005), social skills are defined as the set of social behaviors of the individual repertoire that favor his/her social competence. Social competence is understood as “the capacity to articulate thoughts, feelings and actions in relation to personal aims and the demands of the situation and culture, generating positive consequences for the individual and for his/her relationships with other people” (Del Prette & Del Prette, 2005, p. 33).

The Social Skills Rating System (Gresham & Elliott, 1990) can be highlighted among the best instruments to evaluate children' social skills. It allows the social skills' evaluation by three informants (parents, teachers and the children themselves) producing indicators of social skills, behavior problems, and academic competence. The Social Skills Rating System (SSRS) has already been translated and/or subjected to psychometric studies in different countries, such as the United States, Germany, Iran, Portugal, Norway, the Netherlands, Slovakia, and Puerto Rico. According to the revisions by Crowe, Beauchamp, Catroppa and Anderson (2011), Humphrey et al. (2011), and Matson and Wilkins (2009), the SSRS is one of the most studied measures in the social skills literature and the one most frequently used with school age children, presenting very satisfactory properties of validity and reliability.

A Brazilian version of this instrument, SSRS-BR, was previouly validated through Component Principal Analysis (Bandeira, Del Prette, Del Prette & Magalhães, 2009) and tested with children with typical development and with intellectual disabilities, presenting adequate psychometric properties of construct, criterion, concurrent and experimental validity (Del Prette, Freitas, Bandeira & Del Prette, 2013; Freitas & Del Prette, 2010a, 2010b). The validation study (Bandeira et al., 2009) was based on a sample of 416 children with typical development from elementary public and private schools (2^nd to 5^th years), from four Brazilian states (MG, PR, RJ and SP). Atotal of 312 parents and 86 teachers also participated as informants.

The Principal Components Exploratory Analysis of the first Brazilian version of SSRS-BR scales showed differences in the number of components compared with the original scale, probably related to sample size and to cultural variables (Bandeira et al., 2009). The internal consistency analysis indicated high Cronbach's alpha values for the overall social skills (Student α = .78; Parent α = .86; Teacher α = .94), behavior problems (Parent α = .83; Teacher α = .91) and academic competence (Teacher α = .98) scales. Regarding the temporal stability analysis, positive and statistically significant test-retest correlations were found for the overall scores of the social skills (Student r = .78; Parent r = .69; Teacher r = .71), behavior problems (Parent r = .75; Teacher r = .80) and academic competence (Teacher r = .73) scales.

The studies of Freitas and Del Prette (2010a, 2010b) also provided construct and criterion validation of SSRS-BR for children with intellectual disabilities. The validation of the SSRS-BR enabled a great amount of studies on social skills with different groups of Brazilian children, including those with intellectual disabilities (Rosin-Pinola, Del Prette & Del Prette, 2007), visual impairments (Ferreira, Del Prette & Lopes, 2009), hearing impairments (Souza, 2008), autism (Santos, 2012), learning difficulties and behavior problems (Barreto, Freitas, & Del Prette, 2011), among other populations (Freitas & Del Prette 2013). The instrument has also been used in studies aimed to evaluate the effectiveness of social skills interventions conducted with parents of children with ADHD (Rocha, 2009), and with visual impairments (Freitas, 2005), as well as interventions involving direct training with children in the school context (Lopes & Del Prette 2013).

Although these satisfactory results, some other studies have failed in confirming the subscales structure of SSRS-BR and used it by making re-specifications in the model. This has been attributed by the authors to a gap in the original validation study (Bandeira et al., 2009). It was performed by means of Principal Components Exploratory Analysis. Despite having produced satisfactory results regarding the percentage of explained variance of the data scales (ranging from 40% to 62% for the different subscales) and items with factor loadings considered adequate (greater than 0.40), some authors point out that Principal Components Analysis may not be the most recommended method for the study of construct validity of psychological instruments (Pérez & Medrano, 2010; Reise, Waller, & Comrey, 2000; Tabachnick & Fidell, 2007).

According to Tabachnick and Fidell (2007), Principal Component Analysis is a method of extraction that permits a large number of variables to be reduced to a smaller number of components but with not any compromise with latent variables or factors. This method explains the largest possible amount of variance in the observed data (total variance), including common variance (or commonality), specific variance, and error variance (Perez & Medrano, 2010). Therefore, Reise, Waller, and Comrey (2000) argue that, in some cases, Principal Component Analysis can overestimate the weight of the factor loadings and of the correlations, inflating the values of the components matrix and producing biased data.

As an alternative extraction method to the Principal Component Analysis, in the Exploratory Factor Analysis or Principal Axes Analysis, only the variance that each observed variable shares with the others is available in the analysis, and the specific variance and error variance is excluded (Tabachnick & Fidell, 2007). According to Reise, Waller, and Comrey (2000), the factors can be considered as the hypothetical “causes” of the correlations between the variables and not only as a summary thereof, as occurs in Principal Component Analysis.

Considering the importance of SSRS-BR for the continuity of studies on children' social skills in Brazil as well as the advantages of Exploratory Factor Analysis over Principal Component Analysis, it is argued that the continuation of studies with the SSRS-BR scales should be based on a structure produced by Exploratory Factor Analysis method. This effort could generate improvements in the structure initially obtained, and could contribute to the refinement of its construct validity, with better psychometric indicators and better representativity of the evaluated construct. Furthermore, it should be noted that the identification of the factorial structure of the scales, through Exploratory Factor Analysis, is an important prerequisite for more refined analyzes, such as Confirmatory FactorAnalysis (Pilati & Laros, 2007).

In this context, the present study aimed to conduct further psychometric studies of construct validity of the SSRS-BR scales. In more specific terms, the goal of this study was to produce a Factorial Structure of the SSRS-BR scales and to verify its replication with a new subset of data, by means of Confirmatory Factor Analysis.

Method

Participants

The initial study sample consisted of 1012 elementary school students, of the 2^nd to 5^th years, of five cities in four Brazilian states (São Paulo, Minas Gerais, Paraná, and Río de Janeiro), from public and private schools. The participants were of both sexes, aged between seven and 14 years, and were representative of different socioeconomic strata, according to the Critério Brasil Questionnaire (ABEP, 2003). The children'parents and the teachers also participated as informants. The sociodemographic characteristics of each sub-sample used in the various analyses are described in the Results section.

Instrument

The SSRS was used, with all the items of the original instrument for three versions (parents, students and teachers), as translated for the previous study (Bandeira et al., 2009). The Student version is composed only of the Social Skills Scale, with 35 items measured on a Likert type scale, ranging from 0 to 2 (0 = never, 1 = sometimes, and 2 = very often). The Teacher version has three scales: (a) Social Skills Scale, with 30 items measured in terms of the frequency and importance of the social skills of the children, the frequency with response alternatives equal to the student' form of the student, for and the importance, with alternatives from 0 to 2 (unimportant) to 2 (very important); (b) Behavior Problems Scale, with 18 items that evaluate the frequency of behavior problems, ranging from 0 (never) to 2 (very often). (c) Academic Competence Scale, with nine items with five alternative responses, aiming to classify students with respect to their class, from 1 (with the lowest 10 %) to 5 (with the highest 10 %). The Parent version has two scales: (a) Social Skills Scale, with 37 items evaluated in terms of the frequency and importance, such as in teacher form; (b) Behavior Problems Scale, with 17 items evaluate as in the teacher' form.

Procedure

The project was approved by the Ethics Committee for Research with Humans of UFSCar (Protocol No. 002/04 CEP/UFSCar). The parents of the children received all the relevant information about the study and signed the Terms of Free Prior Informed Consent.

The instrument was completed by parents, teachers and students in groups, within the schools, in a standardized way, after a session of rapport and the clarification of possible doubts. The respondents were asked to read the instructions aloud together with the applicator, thereby verifying potential difficulties of understanding. Individual assistance was given in cases of difficulty, with the applicator reading each item and noting the answer given by the participant on the form. With those parents who did not attend the school, the application was performed in their homes. All the instruments were completed in the presence of the applicator and previously trained research assistants.

Data treatment

The database of total data was created, which consisted of 1012 evaluations by children, 991 evaluations by teachers and 638 evaluations by parents, from four Brazilian states. Next, the verification and exclusion of outliers was performed, through univariate (standardized z score) and multivariate analyzes (Mahalanobis distance test). After this process, the final database was composed of 942 evaluations by children, 817 evaluations by teachers and 562 evaluations by parents.

To carry out the Exploratory and Confirmatory Factor Analyses, the database was randomly divided into two equal halves, maintaining a balance of the age and gender variables, through contingency tables analysis (Crosstabs). In the first half of the data, the Exploratory Analysis of Principal Axes was performed in order to verify the factorial structure of the scales. After the factorial structure for each scale was defined, Confirmatory Factor Analyses were performed on the second half of the data. Exploratory and Confirmatory Factorial Analyses were carried out separately for the social skills, behavior problems, and academic competence scales of the versions for children, parents and teachers.

In the Statistical Package for the Social Sciences (SPSS) version 20.0 program, the Exploratory Factor Analyses were performed using the Principal Axis Factoring (PAF) method, with Promax rotation. For each factor, only those items greater than .30 saturation were selected and the items with saturation in more than one factor being excluded. The criterion for deciding the number of factors to be retained on a scale was defined based on eigenvalues greater than 1.0, the graphical analysis of the Scree-Plot and the presence of at least three items with loadings above the established minimum. Several structures were tested and the ones that best matched the following criteria were selected: percentage of variance explained, number of factors obtained, factor loading of the items, and interpretability of the factorial structure (Laros & Puente-Palacios, 2004). The reliability of the scales was evaluated through the internal consistency of the factors, using Cronbach's alpha index.

The factorial structures obtained with the PAF were submitted to Confirmatory Factor Analyses, using the IBM Statistics Amos 20.0 program. The estimation of the parameters was performed, based on the correlation matrix and adopting the Maximum Likelihood method. For the evaluation of the global fit of the hypothesized models, the goodness of fit criteria most often referenced in the literature (Antunes & Fontaine, 2005; Marôco, 2010) were used: ratio χ²/g.l. < 5, CFI > .90 (Comparative Fit Index), GFI > .90 (Goodness of Fit Index), RMR < .05 (Root Mean Square Residual) and RMSEA < .08 (Root Mean Square Error of Approximation ). As an index of quality of the local fit, the composite reliability of the items representative of each factor was verified (Marôco, 2010). The re-specifications of the models were made from the modification indices produced by the Amos program and based on theoretical considerations.

Results

The results of the exploratory factor analysis for each version (student, parent and teacher) of the SSRS-BR through the PAF method are described below. Because this analysis generated a new factorial structure, the instrument will henceforth be referred to using the acronym SSRS-BR₂.

Exploratory Factor Analyses

SSRS-BR₂ Teacher Version. The sample consisted of the teachers of 424 children, of whom 222 (52.4 %) were boys and 202 (47.6 %) were girls, aged six to 13 years (M = 8.58 years, SD = 1.46 years), from families with socioeconomic levels from A1 to D (A1 = 9.8 %; A2 = 14.5%; B1 = 13.2%; B2 = 20.8%; C = 29%; D = 12.6%).

Structure for Social Skills. The best factorial structure, shown in Table 1, had 25 items distributed in four factors with eigenvalues between 1.17 and 9.29, explaining 61.46 % of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .92) and Bartlett's test was significant (X² = 6091.90; df = 300; p = .001). Items 4, 7, 8, 13, and 15, were excluded from the factorial structure. The factors extracted were as follows: F1: Responsibility, F2: Self-control, F3: Assertiveness/ Social Confidence, and F4: Cooperation/Affectivity.

Structure for Behavior Problems. The best factorial structure, shown in table 2, had 17 items divided into three factors with eigenvalues between 1.01 and 6.69, explaining 62.02 % of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .91) and Bartlett's test was significant (X² = 3752.07; df = 136; p = .001). Item 47 was excluded from the factorial structure. The factors extracted were as follows: F1: Externalizing Problems, F2: Hyperactivity, and F3: Internalizing Problems.

Structure for Academic Competence. The best factorial structure, shown in Table 3, had 9 items in a single factor, with an eigenvalue of 8.04, explaining 89.28 % of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .93) and Bartlett's test was significant (X² = 7648.25; df = 36; p = .001). No items were excluded from the factorial structure.

SSRS- BR₂ Parent Version. The sample con sisted of parents of 290 children, of whom 162 (55.9%) were boys and 128 (44.1%) were girls, aged six to 13 years (M = 8.62 years, SD = 1.28 years), from socioeconomic levels between A1 and D (A1 = 8.4%; A2 = 19.5%; B1 = 11.8%; B2 = 16.8%; C = 34%; D = 9.5%).

Structure for Social Skills. The best factorial structure, shown in Table 4, had 25 items distributed in five factors with eigenvalues between 1.31 and 5.67, explaining 49.61% of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .84) and Bartlett's test was significant (X² = 1860.72; df = 300; p = .001). The items 6, 8, 9, 10, 12, 13, 16, 20, 23, 30, 32, 35, and 38 were excluded from the factorial structure. The factors extracted were as follows: F1: Responsibility, F2: Self-control, F3: Affectivity/Cooperation, F4: Social Confidence, and F5: Civility.

Structure for Behavior Problems. The best factorial structure, shown in Table 2, had 16 items, distributed in two factors with eigenvalues of 1.75 and 5.04, explaining 42.45% of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .87) and Bartlett's test was significant (X² = 1279.77; df = 120; p = .001). Item 54 was excluded from the factorial structure. The factors extracted were as follows: F1: Externalizing Problems and F2: Internalizing Problems.

SSRS-BR₂ Student Version. The sample consisted of 469 children, of whom 256 (54.6 %) were boys and 213 (45.4 %) were girls, aged six to 13 years (M = 8.62 years, SD = 1.51 years), from all socioeconomic levels (A1 = 8.3 %; A2 = 13.5 %; B1 = 14.1%; B2 = 16.7%; C = 36.2%; D = 10.9%; E = 3.0 %).

Structure for Social Skills. The best factorial structure, shown in Table 5, had 21 items distributed in four factors with eigenvalues between 1.29 and 3.43, explaining 36.96 % of the data variance. The results showed that the measure of sampling adequacy was satisfactory (K.M.O. = .77) and Bartlett's test was significant (X² = 1091.69; df = 210; p = .001). Items 1, 2, 5, 9, 12, 15, 19, 23, 24, 28, 29, 30 and 34 were excluded from the factorial structure. The factors extracted were as follows: F1: Empathy/Affectivity, F2: Responsibility, F3: Self-control/Civility, and F4: Assertiveness.

Reliability

The reliability of the scales, measured through Cronbach's alpha coefficient, indicated high values, according Bisquerra, Sarruera and Martinez (2004), for the overall scales of social skills (Student α = .73; Parent α = .85; Teacher α = .92), behavior problems (Parent α = .84; Teacher α = .89) and academic competence (Teacher α = .98). For the subscales, the alpha values were lower, although considered reasonable, for social skills (Student α = .41 to .63; Parent α = .58 to .75; Teacher α = .82 to .91) and behavior problems (Parent α = .66 to .85; Teacher α = .79 to .89).

After the elimination of some items in the Confirmatory Factor Analysis (see next section), new Cronbach alpha coefficients were calculated for the overall scales of social skills and behavior problems of the three instruments. The scales maintained virtually the same good indices obtained previously, with three slight variations of .01, for social skills (Student α = .73; Parent α = .84; Teacher α = .91) and behavior problems (Parent α = .85; Teacher α = .89).

Confirmatory Factor Analyses

SSRS-BR₂TeacherVersion. The sample for this analysis consisted of the teachers of 393 children, of whom 201 (51.3%) were boys and 191 (48.7%) were girls, aged six to 13 years (M = 8.61 years, SD = 1.64 years), from all socioeconomic levels (A1 = 2.8%; A2 = 11.8%; B1 = 17.1%; B2 = 20.2%; C = 37.6%; D = 9.8%; E = 7.0%).

Structure for Social Skills. The goodness of fit indices of the first structure were below those considered satisfactory for some of the previously presented references (χ2/g.l. = 3.909, CFI = .859, GFI = .811, RMR = .029, RMSEA = .086). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. Initially, it was decided to remove items 19, 29 and 14, which presented numerous correlations with measurement errors of other items, within the same factor and also between factors. Subsequently, seven cova riances were specified between errors of items with very similar semantic content, within the same factor. The correlations were specified between the errors of items 27 and 21, 16 and 9, 5 and 25, 25 and 11, 5 and 11, 26 and 30, 03 and 17. With each re-specification of the model, the goodness of fit indices were verified. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l. = 2.257, CFI = .949, GFI = .906, RMR = .025, RMSEA = .057. The final model with standardized factorial weightings can be seen in figure 1.

Structure for Behavior Problems. The goodness of fit indices of the first structure were below those considered satisfactory for some of the reference criteria previously presented (χ2/g.l. = 4.818, CFI = .883, GFI = .865, RMR = .044, RMSEA = .099). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. Initially, it was decided to remove items 35, 45 and 39, which presented numerous correlations with measurement errors of other items within the same factor and also between factors. Subsequently, two covariances were specified between errors of items with very similar semantic content, within the same factor. The correlations were specified between the errors of items 41 and 31, 31 and 33. With each re-specification of the model, the goodness of fit indices were verified. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l. = 3.359, CFI = .947, GFI = .925, RMR = .036, RMSEA = .078. The final model with standardized factorial weightings can be seen in figure 2.

Structure for Academic Competence. The goodness of fit indices of the first structure were below those considered satisfactory for some of the reference criteria previously presented (χ2/g.l. = 35.693, CFI = .872, GFI = .707, RMR = .032, RMSEA = .297). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. In this model, four covariances were specified between errors of items with very similar semantic content. The correlations were specified between the errors of items 52 and 50, 51 and 53, 54 and 55, 54 and 57. With each re-specification of the model, the goodness of fit indices were verified. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l.= 3.113, CFI = .993, GFI = .962, RMR = .015, RMSEA = .073. The final model with standardized factorial weightings can be observed in figure 3.

SSRS-BR₂ Parent Version. The sample for this analysis consisted of the parents of 272 children, of whom 142 (52.2 %) were girls and 130 (47.8 %) were boys, aged six to 13 years (M = 8.47 years; SD = 1.35 years), from all socioeconomic levels (A1 = 10.4%; A2 = 13.1%; B1 = 12.0%; B2 = 21.9%; C = 30.3%; D = 11.6%; E = 8.0%).

Structure for Social Skills. The goodness of fit indices of the first structure were below those considered satisfactory for some of the reference criteria previously presented (χ2/g.l. = 1.997, CFI = .824, GFI = .862, RMR = .027, RMSEA = .061). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. Initially, it was decided to remove items 2 and 34, which presented numerous correlations with measurement errors of other items, within the same factor and also between factors. Subsequently, four covariances were specified between errors of items with very similar semantic content, within the same factor. The correlations were specified between the errors of items 26 and 14, 27 and 11, 33 and 1, 7 and 24. With each re-specification of the model, the goodness of fit indices were verified. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l. = 1.608, CFI = .902, GFI = .899, RMR = .024, RMSEA = .047. The final model with standardized factorial weightings can be observed in figure 4.

Structure for Behavior Problems. The goodness of fit indices of the first structure were below those considered satisfactory for some of the reference criteria previously presented (χ2/g.l. = 2.329, CFI = .868, GFI = .900, RMR = .028, RMSEA = .070). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. Initially, it was decided to remove item 52, which presented numerous correlations with measurement errors of other items, within the same factor and also between factors. Subsequently, one covariance was specified between the errors of items 46 and 39, which presented very similar semantic content, within the same factor. With each re-specification of the model, the goodness of fit indices were verified. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l. = 1.901, CFI = .920, GFI = .924, RMR = .025, RMSEA = .058. The final model with standardized factorial weightings can be observed in figure 5.

SSRS-BR₂ Student Version. The sample for this analysis consisted of 473 children, of whom 242 (51.2 %) were boys and 231 (48.8 %) were girls, aged six to 13 years (M = 8.60 years; SD = 1.40 years), from all socioeconomic levels (A1 = 6.7%; A2 = 15.3%; B1 = 12.7%; B2 = 21%; C = 33.7% D = 10.3%; E = 3.0%).

Structure for Social Skills. The goodness of fit indices of the first structure were below those considered satisfactory for some of the reference criteria previously presented (χ2/g.l. = 1.593, CFI = .855, GFI = .946, RMR = .021, RMSEA = .035). Based on the modification indices produced by the Amos program and on theoretical considerations, re-specifications were made in the model. It was decided to remove item 17, as it present numerous correlations with measurement errors of other items, within the same factor and also between factors. Satisfactory indices were obtained for all the adopted criteria only after the last specification performed: χ2/g.l. = 1.392, CFI = .903, GFI = .954, RMR = .019, RMSEA = .029. The final model with standardized factorial weightings can be observed in figure 6.

Internal Consistency

The internal consistency of the confirmatory factorial structures was assessed through composite reliability, as suggested by Hair, Black, Babin, Anderson and Tatham (2010). Composite reliability estimates the internal consistency of the items that comprise each factor or construct, indicating the degree to which these items are consistent manifestations of the latent factor. In general, it is considered that a value ≥ 0.70 is an indicator of adequate reliability of the construct. For the teacher scale, high values of composite reliability were found for the factorial structures of social skills (F1 = .91, F2 = .85, F3 = .80, F4 = .86), behavior problems (F1 = .90, F2 = .85, F3 = .77) and academic competence (.98). For the parents scale, the values of composite reliability found were reasonable for the factorial structure of social skills (F1 = .69, F2 = .73, F3 = .62, F4 = .65, F5 = .63) and behavior problems (F1 = .84, F2 = .64). For the student scale, the composite reliability values were lower than expected for the factorial structure of social skills (F1 = .52, F2 = .58, F3 = .49, F4 = .34).

Discussion

This study allowed the extraction of factorial structures for the set of scales of the SSRS-BR, through the PAF method of exploratory analysis. Compared to the first Brazilian validation study of the scales (Bandeira et al., 2009, based on Principal Components), the PAF method represents an advance and refinement of the factorial structure of the scales, which are now being referred to as SSRS-BR₂. Due to the higher statistical requirement of the PAF method with respect to the factor loading, various items were excluded from the social skills and behavior problems scales for the students (13 items), parents (14 items), and teachers (6 items), due to presenting low saturation coefficients. Only in the academic competence scale were no items excluded for this reason. Furthermore, unlike the North American validation study (Gresham & Elliott, 1990) and the previous Brazilian study (Bandeira et al., 2009), it was chosen to exclude items with saturation in more than one factor, in order to generate more theoretically and statistically consistent factorial structures.

The best factorial structures obtained in this study consisted of a smaller number of factors in the social skills scale, compared with the study of Bandeira et al. (2009). There was a reduction of one factor in each of the versions: for parents, teachers and students. In the behavior problems scale, there was a decrease of one factor in the parent version and the addition of one factor in the teacher version. In the academic competence scale, there were no changes in terms of the factorial composition. It is noteworthy that in other cross-cultural validation studies of the SSRS-BR scales, different factorial compositions from the original were also found, for example, in the recent studies with the scale for students in Portugal (Mota, Matos, & Lemos, 2011) and with the scale for parents of preschool children in Germany (Hess et al., 2013).

The situational-cultural character of social skills should be understood as an indispensable base for the construction, validation and interpretation of psychological measures in this field. Therefore, the evaluation of social skills and of some constructs related to them largely depends on variables of the sociocultural context in which social behavior is being emitted. Del Prette et al. (2004) argue that, even within a specific culture, there may be regional subcultures that differently influence the social performance and the interpersonal relationships of the individuals who participate in them. In this sense, the impact of these differences on the criteria used to evaluate social skills can not be disregarded.

The new factorial structures obtained for the SSRS-BR₂ appear to be more parsimonious and reduced than the previous one, with higher indices of explained variance (of at least .40), factor loading (above .30) and reliability (above .70 for all the overall scales). Regarding the student version of the social skills scale, the percentage of explained variance was below the expected minimum (36.96%). This more modest result reproduces the pattern obtained in the previous validation studies (Gresham & Elliott, 1990; Bandeira et al. 2009) and it is probably related to the children' biases in social skills self-evaluation , as often reported in the literature (Barreto, Freitas, & Del Prette, 2011; Freitas & Del Prette, 2010c; Gresham, Elliott, Cook, Vance, & Kettler, 2010).

The factorial structures extracted through PAF were confirmed in the confirmatory factor analysis for the three versions of the SSRS-BR₂, after conducting re-specifications in the models, suggested by the modification indices. Where high correlations were observed between errors of items from different factors, it was chosen to remove the respective items. According to Marôco (2010), the correlations between errors of items belonging to different factors may indicate the existence of factors of a higher hierarchical order which were not considered in the model. In the case of the items of the same factor, in which correlated errors were found, a covariance was added to the model, since this was consistent from a theoretical point of view.

In the teacher version, three items were eliminated from the social skills scale and seven co-variances added between errors of items belonging to the same factor. In the behavior problems scale, three items were eliminated and two covariances specified between errors of items. In the academic competence scale no item was eliminated and four correlations between errors were added to the model.

In the parent version, two items were eliminated from the social skills scale and four co-variances added between errors of items. In the behavior problems scale, an item was removed and two correlations between errors were added. In the student version, only one item was removed from the model. The re-specifications performed allowed a good overall fit of the models considered, according to the indices χ²/g.l., CFI, GFI, RMR, and RMSEA.

The existence of many correlations between errors of items belonging to the same factor seems to indicate a specificity of the instruments for the evaluation of social skills. In fact, some items have very similar formulations, with alterations related only to the context or the speaker considered, for example: Item 11 - Congratulates family members on accomplishments and Item 27 - Gives compliments to friends or other children in the family. The maintenance of items with similar formulations is justified, from the theoretical point of view, by considering the situational-cultural specificity of social skills (Del Prette & Del Prette, 2010/2012). Accordingly, small differences in items, related to the context and the interlocutor, may be relevant in the identification of social skill deficits associated with particular situations.

With respect to the local adjustment of the models, the composite reliability was calculated, which estimates the internal consistency of the items representative of each factor. In this question, the teacher version presented high composite reliability for all the factors of the social skills, behavior problems and academic competence scales, with values above .70. The parent version showed more modest composite reliability values for the social skills and behavior problems factors, some of which were between .60 and .70. The student version had the lowest reliability values for the social skills factors, all less than .60. Ideally, the composite reliability values should be above .70, however, Hair et al. (2010) argue that in exploratory studies a value above .50 can be considered adequate. It is recognized that in the present study, the student version presented low composite reliability values. Self-assessment of social skills and social competence by children is usually less accurate compared to the evaluation by other informants (Gresham et al., 2010; Renk & Phares, 2004), and this trend is certainly reflected in the data analyzed here.

Final Considerations

This study represents a refinement of the psychometric properties of the scales of the SSRS-BR₂ according to three main aspects: (a) in obtaining factor solutions through the PAF method, which can be more appropriately named factor structures than those obtained by principal component method (b) in adjusting the factor structures found in a new dataset, through confirmatory factor analysis, after the introduction of some re-specifications and (c) in obtaining more parsimonious scales in terms of the number of factors, number of items, and absence of repeated items in more than one factor, facilitating the application and the analysis of the results.

Added to other evidence of validity and reliability previously verified (Bandeira et al., 2009; Freitas & Del Prette, 2010a, 2010b), the data from this study extend the good SSRS-BR psychometric indices, making it available for evaluating social skills and related constructs in different populations of Brazilian children.

References

Antunes, C., & Fontaine, A. M. (2005). Percepção de apoio social na adolescência: Análise fatorial confirmatória da escala Social Support Appraisals. Paidéia, 15(32), 355-366. doi: http://dx.doi.org/10.1590/S0103-863X2005000300005.         [ Links ]

Bandeira, M., Del Prette, Z. A. P., Del Prette, A., & Magalhães, T. (2009). Validação das escalas de habilidades sociais, comportamentos problemáticos e competência acadêmica (SSRS-BR) no ensino fundamental. Psicologia: Teoria e Pesquisa, 25(2), 271-282. doi: http://dx.doi.org/10.1590/S0102-37722009000200016.         [ Links ]

Barreto, S. O., Freitas, L. C., & Del Prette, Z. A. P. (2011). Habilidades sociais na comorbidade entre dificuldades de aprendizagem e problemas de comportamento: Uma avaliação multimodal. Psico, PUC-RS, 42(4), 503-510.         [ Links ]

Bisquerra, R., Sarriera, J. C., & Martinez, F. (2004). Introdução à estatística: Enfoque informático com o pacote estatístico SPSS. Porto Alegre: Artmed.         [ Links ]

Brazilian Association of Research Companies - ABEP (2003). Critério de classificação econômica Brasil. Retrieved from http://www.abep.org.         [ Links ]

Caprara, V. G., Barbaranelli, C., Pastorelli, C., Bandura, A., & Zimbardo, P. G. (2000). Prosocial foundations of children's academic achievement. Psychological Science, 11(4), 302-306. doi: 10.1111/1467-9280.00260.         [ Links ]

Crowe, L. M., Beauchamp, M. H., Catroppa, C., & Anderson, V. (2011). Social function assessment tools for children and adolescents: A systematic review 1989-2011. Journal of Neurotrauma, 29, 1277-1291. doi: 10.1016/j.cpr.2011.03.008.         [ Links ]

Del Prette, Z. A. P., & Del Prette, A. (2005) Psicologia das habilidades sociais na infância: Teoria e prática. Petrópolis: Vozes.         [ Links ]

Del Prette, Z. A. P., Del Prette, A., Barreto, M. C., Bandeira, M., Rios-Saldaña, M.R., Ulian, A.L.A.O. Villa, M.B. (2004). Habilidades sociais de estudantes de psicologia: Um estudo multicêntrico. Psicologia: Reflexão e Crítica, 17(3), 341-350. doi: http://dx.doi.org/10.1590/S0102-79722004000300007.         [ Links ]

Del Prette, Z. A. P., & Del Prette, A. (2010/2012). Social skills and behavior analysis: Historical proximity and new issues. Perspectivas em Análise do Comportamento, 1(2), 104-115 (English version of a Portuguese paper, originally published in 2010).         [ Links ]

Del Prette, Z. A. P., Del Prette, A., Oliveira, L. A., Gresham, F. M., & Vance, M. J. (2012). Role of social performance in predicting learning problems: Prediction of risk using logistic regression analysis. School Psychology International Journal, 2, 1-16. doi: 10.1177/0020715211430373.         [ Links ]

Del Prette, Z. A. P., Freitas, L. C., Bandeira, M., & Del Prette, A. (2013). Inventário de habilidades sociais, problemas de comportamento e com petência acadêmica para crianças (SSRS-BR₂): Manual de aplicação, apuração e interpretação. Manuscript submitted for publication.         [ Links ]

Ferreira, B. C., Del Prette, Z. A. P., & Lopes, D.C. (2009). Habilidades empáticas de crianças videntes e cegas e a possível influência de variáveis sociodemográficas. Interação em Psicologia, 13(1), 49-58.         [ Links ]

Freitas, L. C., & Del Prette, Z. A. P. (2010a). Validade de critério do Sistema de Avaliação de Habilidades Sociais (SSRS-BR). Psicologia: Reflexão e Crítica, 23(3), 430-439. doi: http://dx.doi.org/10.1590/S0102-79722010000300003.         [ Links ]

Freitas, L. C., & Del Prette, Z. A. P. (2010b). Validade de construto do Sistema de Avaliação de Habilidades Sociais para crianças brasileiras com deficiência intelectual. Interamerican Journal of Psychology, 44(2), 312-320.         [ Links ]

Freitas, L. C., & Del Prette, Z. A. P. (2010c). Comparando autoavaliação e avaliação de professores sobre as habilidades sociais de crianças com deficiência mental. Interpersona, 4(2), 183 193. doi: http://dx.doi.org/10.5964/ijpr.v4i2.48.         [ Links ]

Freitas, L. C., & Del Prette, Z. A. P. (2013). Habilidades sociais de crianças com diferentes necessidades educacionais especiais: Avaliação e implicações para intervenção. Avances en Psicología Latinoamericana, 31(2), 344-362.         [ Links ]

Freitas, M. G. (2005). Desenvolvimento e avaliação de um programa de habilidades sociais com mães de crianças deficientes visuais. (Doctoral dissertation, Universidade Federal de São Carlos, São Carlos, Brazil). Retrieved from http://www.bdtd.ufscar.br/htdocs/tedeSimplificado//tde_busca/arquivo.php?codArquivo=767.         [ Links ]

Gresham, F. M., & Elliott, S. N. (1990). Social Skills Rating System: Manual. Circle Pines, MN: American Guidance Service.         [ Links ]

Gresham, F. M., Elliott, S. N., Cook, C. R., Vance, M. J., & Kettler, R. J. (2010). Cross-informant agreement for social and problem behavior ratings: An investigation of the Social Skills Improvement System Rating Scales. Psychological Assessment, 22(1), 157-166. doi: 10.1037/a0018124.         [ Links ]

Hair, J. F. Jr., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate data analysis (7th Edition). Upper Saddle River: Pearson Education.         [ Links ]

Hess, M., Scheithauer, H., Kleiber, D., Wille, N., Erhart, M., & Ravens-Sieberer, U. (2013). The parent version of the preschool Social Skills Rating System: Psychometric analysis and adaptation with a German preschool sample. Journal of Psychoeducational Assessment, ahead of print. doi: 10.1177/0734282913483977.         [ Links ]

Humphrey, N., Kalambouka, A., Wigelsworth, M., Lendrum, A., Deighton, J., & Wolpert, M. (2011). Measures of social and emotional skills for children and young people: A systematic review. Educational and Psychological Measurement, 71(4), 617-637. doi: 10.1177/0013164410382896.         [ Links ]

Laros, J. A., & Puente-Palacios, K. E. (2004). Validação cruzada de uma escala de clima organizacional. Estudos de Psicologia, 9(1), 113-119. doi: http://dx.doi.org/10.1590/S1413294X2004000100013.         [ Links ]

Lopes, D. C., Del Prette, Z. A. P., & Del Prette, A. (2013). Recursos multimídia na promoção de habilidades sociais de crianças com baixo rendimento acadêmico. Psicologia: Reflexão e Crítica, 26(3), 451-458. doi: http://dx.doi.org/10.1590/S0102-79722013000300004.         [ Links ]

Marôco, J. (2010). Análise de Equações Estruturais: Fundamentos Teóricos, Software e Aplicações. Lisboa: Report Number.         [ Links ]

Matson, J. L., & Wilkins, J. (2009). Psychometric testing methods for children's social skills. Research in Developmental Disabilities, 30(2), 249-274. doi: http://dx.doi.org/10.1016/j.ridd.2008.04.002.         [ Links ]

Molina, R.C.M.M., & Del Prette, A. (2007). Mudança no status sociométrico negativo de alunos com dificuldades de aprendizagem. Psicologia Escolar e Educacional, 11(2), 299-310. doi: http://dx.doi.org/10.1590/S141385572007000200009.         [ Links ]

Mota, C. P., Matos, P. M., & Lemos, M. S. (2011). Psychometric properties of the social skills questionnaire: Portuguese adaptation of the student form (grades 7 to 12). The Spanish Journal of Psychology, 14(1), 486-499. doi: http://dx.doi.org/10.5209/rev_SJOP.2011.v14.n1.44.         [ Links ]

Pérez, E. R., & Medrano, L. (2010). Análisis factorial exploratorio: Bases conceptuales y metodológicas. Revista Argentina de Ciencias del Comportamiento, 2(1), 58-66.         [ Links ]

Pilati, R., & Laros, J. A. (2007). Modelos de equações estruturais em psicologia: Conceitos e aplicações. Psicologia: Teoria e Pesquisa, 23(2), 205-216. doi: http://dx.doi.org/10.1590/S010237722007000200011.         [ Links ]

Reise, S. P., Waller, N. G., & Comrey, A. L. (2000). Factor analysis and scale revision. Psychological Assessment, 12(3), 287-297. doi: 10.1037//1040-3590.12.3.287.         [ Links ]

Renk, K., & Phares, V. (2004). Cross-informant ratings of social competence in children and adolescents. Clinical Psychology Review, 24(2), 239-254. doi: http://dx.doi.org/10.1016/j.cpr.2004.01.004.         [ Links ]

Rocha, M. M. (2009). Programa de habilidades sociais educativas com pais: efeitos sobre o desempenho social e acadêmico de filhos com TDAH. (Doctoral dissertation, Universidade Federal de São Carlos, São Carlos, Brazil). Retrieved from http://www.bdtd.ufscar.br/htdocs/tedeSimplificado//tde_busca/arquivo.php?codArquivo=2283.         [ Links ]

Rosin-Pinola, A. R., Del Prette, Z. A. P., & Del Prette, A. (2007). Habilidades sociais e problemas de comportamento de alunos com deficiência mental, alto e baixo desempenho acadêmico. Revista Brasileira de Educação Especial, 13(2), 239-256. doi: http://dx.doi.org/10.1590/S141365382007000200007.         [ Links ]

Rubin, K. H., Coplan, R. J., & Bowker, J. C. (2009). Social withdrawal in childhood. Annual Review of Psychology, 60, 141-171. doi: 10.1146/annurev.psych.60.110707.163642.         [ Links ]

Santos, L. H. Z. (2012). Caracterização e análise das habilidades sociais e problemas de comportamento de crianças com autismo. (Master's thesis, Universidade Federal de São Carlos, São Carlos, Brazil). Retrieved from http://www.bdtd.ufscar.br/htdocs/tedeSimplificado/tde_busca/arquivo.php?codArquivo=5413.         [ Links ]

Sorlie, M. A., Hagen, K. A., & Ogden, T. (2008). Social competence and antisocial behavior: Continuity and distinctiveness across early adolescence. Journal of Research on Adolescence, 18(1), 121-144. doi: 10.1111/j.1532-7795.2008.00553.x.

Souza, T. M. P. (2008). Inclusão de crianças com deficiência auditiva na escola regular: Uma análise sob a perspectiva das habilidades sociais. (Master's thesis, Universidade Federal de São Carlos, São Carlos, Brazil). Retrieved from http://www.bdtd.ufscar.br/htdocs/tedeSimplificado//tde_busca/arquivo.php?codArquivo=3262.

Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics (5ª Edição). New York: Allyn and Bacon.         [ Links ]

Urbina, S. (2007). Fundamentos da testagem psicológica. Porto Alegre: Artmed.         [ Links ]