The performance of students with and without disability on the 11^th grade university admission test

Rendimiento en la prueba de admisión a la universidad (Saber 11) de personas en y sin condición de discapacidad

Andrea Padilla-Muñoz¹, Viviana Rodríguez², Sergio Castro², Yamile Reveiz² and Carlos Gómez-Restrepo²

¹ Universidad del Rosario, Bogotá, Colombia. andrea.padilla@urosario.edu.co
² Pontificia Universidad Javeriana, Bogotá, Colombia. viviana.rodriguez@javeriana.edu.co; sergiomariocastro@gmail.com; yamile16@hotmail.com; cgomez_restrepo@yahoo.com

Received 13^th October 2012 / Send for Modification 12th November 2012 / Accepted 7th December 2012

ABSTRACT

Objective Comparing standardized university admission scores between students with and without disability within the same school and also comparing the performance of students with hearing, visual and physical disability.

Methods This was a retrospective cohort study which compared the test results of students with and without disabilities within the same school. A matched case-control study nested in a cohort was assembled. Standardized differences were calculated for comparing such scores.

Results Students without disabilities performed significantly better than disabled ones(i.e. disabled students scored lower than their classmates).

Conclusion Further research is needed to determine whether the Colombian educational system, adjustments made to tests for this population or students' individual skills were the factors underlying the difference regarding the test results.

Key Words: Disabled person, college admission test, test-taking skills (source: MeSH, NLM).

RESUMEN

Objetivo Comparar los resultados en la prueba Saber 11 (prueba estandarizada para la admisión en la mayoría de universidades de Colombia) entre los estudiantes con y sin discapacidad dentro de la misma institución educativa. Asimismo, comparar el rendimiento de los alumnos con problemas de audición, discapacidad visual y física.

Métodos Se realizó un estudio de cohorte retrospectiva en la que se dividió a los estudiantes según su condición de discapacidad. Se anidó un caso-cohorte emparejando cada caso de discapacidad con su grupo de compañeros de colegio. Se calcularon las diferencias estandarizadas para comparar los grupos.

Resultados Los estudiantes sin discapacidades se desempeñaron significativamente mejor que los estudiantes con discapacidad, es decir, los estudiantes con discapacidades obtuvieron puntajes más bajos que sus compañeros de clase.

Conclusión Se necesita más investigación para investigar si el sistema educativo colombiano, los ajustes en las pruebas para esta población o las capacidades individuales de los alumnos son los factores subyacentes de la diferencia en los resultados de las pruebas.

Palabras clave: Personas con discapacidad, prueba de admisión académica, habilidades para tomar exámenes (fuente: DeCS, BIREME).

For the past two decades, the Colombian government has been interested in improving its students' educational performance through standardized testing (1). Such testing is used for evaluating the quality of the education being provided and selecting higher education applicants. However, it remains unknown whether the Colombian university admission test is suitable for assessing disabled people (DP) and whether their performance is similar to that obtained by people without disabilities.

Disabled students often obtain lower scores on standardized tests (2), perhaps because many require special accommodation in the curriculum as well special teaching techniques (3-6).Teachers feel unprepared for handling and teaching DP (7,8) which could have a huge impact on these students' performance.

Given that DP are often victims of discrimination (9,10), social movement and policy have supported their social integration, such as the International classification of functioning(ICF)disability and health (11) and the United Nations' Convention for People with Disabilities (12) which states that it is necessary to, "promote, protect and ensure the full and equal enjoyment equality of all human rights and fundamental freedoms by all people with disabilities and to promote respect for their inherent dignity".

Accommodation means an adaptation introduced into an evaluation setting allowing students having physical, sensorial and cognitive impediments to demonstrate their knowledge, without altering test complexity. Accommo-dation changes the context in which information is presented or the manner in which a student responds to ensure that DP have the opportunity to achieve their greatest potential (13). Adaptation most commonly used around the world would include an extended time setting, read aloud accommodation, modifying a classroom setting (e.g. providing a separate room for conducting the test) and changing the answer format (14-16).

The most popular adaptation studied to date has been extended-time accommodation. Providing DP with additional time for completing any test has proved to be very effective, and the most important; DP performance has improved substantially compared to students not receiving this accommodation (17,18). Another adaptation which has turned out to be very beneficial has been the read aloud accommodation and several studies have demonstrated that DP perform significantly better when given this aid, thus supporting its use (19).

Other accommodations, such as Braille or sign language, allow students having sensory impairment to participate in educational activities and perform on standardized tests with very good results (14, 20-22).

An observational study has found that tests having accommodation improve disabled students' performance, increase their confidence and lift their self-esteem. It has been noted that since the scholastic aptitude examination (SAT) involved accommodation for students having physical, hearing, visual and cognitive impairment, there was an 82 %increasein enrolment for the exam between 1980 and 1983 (23). Similarly, Ragosta et al., (1991) found that disabled students' SAT score was a suitable predictor of performance and persistence within higher education institutions (24).

Some authors have argued that students achieving better grades are those who have a better grasp of the concepts being evaluated by a particular test; such studies have concluded that such cases make the most of the benefit of an adaptation, achieving similar or even better results than people without disability (25). Other studies have compared the performance of students receiving accommodation on a standardized test to those who did not; they concluded that differences in results were not due to adaptation but rather each student's intellectual competence (26,27).

Given the above, the US Rehabilitation Act clearly stated that any test taken by a disabled person must be administered in such a way that the results demonstrate their skills, and not their limitations (28). Also The American Law of the Disabled and the Americans with Disabilities Act have dictated that all DP must receive accommodation in evaluation settings, regardless of their condition, race and ethnicity (29).

The 2009 Colombian university admission test (Saber 11) only provided accommodation for people with sensorial disability. This did not follow the World Health Organization 2001 Standard Rules on the Equalization of Opportunities for Persons with Disabilities stating that the state must guarantee DP integration into the official educational system (30).

Regardless of such recommendations, there is no clear evidence regarding whether disabled people should take a standardized test, especially those who are severely compromised. It is not clear whether tests having accommodation can really measure disabled students' true level of knowledge. There is not enough evidence to suggest that any accommodation does really increase a student's performance (31).

No studies have been conducted in this field in Colombia. A study was thus needed which could evaluate the performance of student-aged DP living in Colombia. The number of disabled students in the USA completing high school has increased by 17 % due to US policy and laws; those admitted to universities has increased by 32 % and there has been a 15% increase in those obtaining jobs (32). Having objective data regarding the differential results on these tests could help making informed decisions to improve the quality of education in Colombia.

METHODS

A secondary analysis was made of Colombian university admission test scores (Saber 11) for 11th grade students in 2009. Information was obtained from ICFES data bases using an FTP provided by this entity which is responsible for evaluating education in Colombia. Data was analyzed using STATA v11. The results for students with and without disability were described using means and standard deviations (SD) in each area evaluated. The scores of students with and without disability were initially compared by calculating mean differences between groups and robust regression for adjusting such differences regarding the nature of an institution (private or public school), the area where DP live (urban-rural) and gender.

A case-cohort analysis was conducted where DP performance was compared to that of students without disability within the same school. Educational institutions teaching young DP and people without disabilities (PWOD) were selected and students were then divided into cases (DP) and controls (PWOD) for each school. The difference between DP score and that of her/his peer group without disabilities was calculated. The difference was divided by control group score SD. This led to calculating distances in terms of SD for the scores obtained by DP compared to the overall cohort score as well the results regarding a specific disability.

Analysis took into account the entire student population taking the test for the first time in 2009; no hypothesis testing was thus used.

RESULTS

Table 1 shows the differences between subjects with and without disability for each area evaluated by Saber 11. It was observed that students without disability obtained higher scores on all areas compared to those with disability (1.9 to 4.7 points higher). Differences were adjusted by the nature of the institution, the area where they lived(urban or rural) and gender showing that, even controlling for these variables, people without disability (PWOD) had higher average scores than DP.

Table 1 also shows standardized differences regarding test scores. These differences showed the average distance in SD between DP score and that for their matched cohort. Disabled students obtained lower values than those for her/his cohort for all areas tested, having average values between 0.1 and 0.5 SD lower.

Table 2 classifies standardized difference according to the type of disability. The data shows that blind students were less distant from their cohort; no overall trends were observed for the other DP. It should be noted that the deaf who received help from an interpreter performed better on the language test; however, the deaf who did not receive help from an interpreter obtained better scores on the physics and language performance tests.

Mean scores for DP in most areas were lower in private schools in analysis (Table 3) based on the nature of schools (i.e. public or private). Mean standardized differences also show that DP were much further away from their cohort of fellow students in private schools. When comparing scores by socioeconomic status, DP from a lower economic status obtained a higher mean score that those belonging to the upper class. The latter group's scores were more distanced from their peers' scores when compared to the group from the lower socio-economic class (Table 4).

DISCUSSION

It was noted that disabled students' scores were lower than those for students from the same cohort (Table 1). This may have been due to discrimination against DP, a flawed educational system or the absence or presence of inappropriate adaptation (2). It might thus be assumed that disabled students were less competent than non-disabled students, they were just as competent as the rest of the population but the educational system is not properly suited for teaching DP, accommodation is to blame for disabled students' poor performance and not their intellectual capacity or the educational system (or all of the above).

The present study could not conclude which of the fore mentioned options would be the most plausible; further studies are thus needed for determining this.

Obtaining low scores would mean that some DP would not gain access to higher education. This represents an important social and economic burden for the nation (33). The results could suggest that the educational system is not preparing DP for facing present day world demands.

Disabled students' test scores were lower than those for their cohort for all areas tested, values being 0.1 to 0.5 SD below. A meta-analysis by Cooper et al studied changes in student performance after summer holidays (34). It was observed that English and maths test performance dropped by one tenth of an SD in the course of 3 months after summer vacations (i.e.-0.1 SD would stand for three months' knowledge loss). Extrapolating this data to disabled students obtaining-0.2,-0.3, -0.4 and-0.51 SD might infer that they would be 6, 9, 12 and 15 months behind in school.

Attending school is not enough; it is thus crucial to know whether students are learning what they need to learn and when they need it (35). Poor quality education (causing DP to fall behind in school) means that educational policy must be developed to deal more directly and effectively with quality issues (36). There is a lot of controversy concerning the best policy for achieving this goal. It has been recommended that teachers' salaries be increased, more schools built, more materials purchased, etc. (37). However, Latin-American countries already spend around 5 % of their GDP on education, therefore not leaving much room for increased expenditure; there is no guarantee that such measures would have any effect whatsoever on the quality of education (38).

The educational system may not be entirely guilty for poor performance on tests; accommodation may be flawed. Table 2 shows that deaf students who received help from an interpreter performed better on the language test; however, deaf students who did not receive help from an interpreter obtained higher scores on the physics test.

The above observations do not indicate that accommodation is out-dated and useless. Although there is not enough evidence to assess the impact of accommodation on standardized tests, such tests are still needed to facilitate increase DP performance. Assessment should be aimed at ascertaining whether grouped accommodation has the same effect on the results as individual accommodation does (10). Not everyone having hearing impairment has the same degree of severity. A deaf person may need different types of adaptation than other deaf people.

It has been noted worldwide that good test performance is closely related to a particular country's socioeconomic status, the person and the school (39). Educational resources, parental participation and school autonomy are associated with good test performance by students with and without disability. It is known that public schools have fewer resources to invest in infrastructure, textbooks, computers and scientific laboratories compared to private schools (39). Our study revealed that the test scores of disabled students attending public schools and lower social status were higher than the results of disabled students attending private schools and having higher social status(Table 3 and 4). However, students without disability attending private schools and having higher social status outperformed students without disabilities attending public schools and having lower social status. This made the distance between cohorts of fellow students greater in private schools and regarding higher social status than in public schools and lower social status. It is not yet clear why low social status DP attending public schools scored higher than higher social status DP attending private schools. Such differences may be secondary to discrimination against disabled students (9).

Despite Colombian legislation aimed at ending discrimination, DP sometimes experience unfair treatment affecting their quality of their per-sonal and professional life. Although significant progress has been made, many still encounter significant obstacles. However, discrimination must still be fought, maybe through organizations representing DP raising awareness that a society which practices discrimination is a doomed society.

REFERENCES

1. Morrison C. High-stakes tests and students with disabilities. Boston Coll. Law Rev. 2000 Sep 1;41(5):1139.         [ Links ]

2. Katsiyannis A, Zhang D, Ryan JB, Jones J. High-stakes testing and students with disabilities challenges and promises. J. Disabil. Policy Stud. 2007 Dec 1;18(3):160-7.         [ Links ]

3. Hanbury M. Educating pupils with autistic spectrum disorders: a practical guide. Thousand Oaks, CA: SAGE Publications Inc; 2005.         [ Links ]

4. Myles B. Children and youth with Asperger Syndrome. Thousand Oaks, CA: Corwin Press; 2005.         [ Links ]

5. U.S. Government Accountability Office. Report to the Chairman and Ranking Minority Member, Subcommittee on Human Rights and Wellness, Committee on Government Reform, House of Representatives: Special education, children with autism. 2005. Report No.: GAO-05-220.         [ Links ]

6. Wagner S. Inclusive programming for the middle school students with autism/Asperger's syndrome: topics and issues for consideration by teachers and parents. Arlington, TX: Future Horizons, Inc.; 2002.         [ Links ]

7. Moores-Abdool W. Included Students with Autism and Access to General Curriculum: What Is Being Provided? Issues Teach. Educ. 2010;19(2):153-69.         [ Links ]

8. Padilla A. Estudio de la opinión de rectores sobre educación de PCD. Pontificia Universidad Javeriana; 2012.         [ Links ]

9. Hahn H. The political implications of disability definitions and data. J. Disabil. Policy Stud. 1993 Jul 1;4(2):41-52.         [ Links ]

10. Bickenbach JE, Chatterji S, Badley E., Üstün T. Models of disablement, universalism and the international classification of impairments, disabilities and handicaps. Soc. Sci. Med. 1999 May;48 (9):1173-87.         [ Links ]

11. World Health Assembly. WHA54.21 [Internet]. Available from: http://www.who.int/classifications/icf/wha-sp.pdf. Cited 2001 May 22.         [ Links ]

12. Mensaje del Secretario General. Relativo al día internacional de las personas con discapacidad [Internet]. Nac. Unidas Enable. 2008. Available from: http://www.un.org/spanish/disabilities/default.asp?id=1460. Cited 2009 Apr 13.         [ Links ]

13. La integración educativa de los niños y las niñas con discapacidad: Una evaluación en Bogotá desde la perspectiva del derecho a la educación [Internet]. Defensoría del Pueblo de Colombia; 2004. Available from: http://www.defensoria.org.co/red/anexos/publicaciones/DESC_derecho_a_la_educacion_especiales.pdf. Cited 2012 Apr 30.         [ Links ]

14. Tindal G, Fuchs L. A Summary of Research on Test Accommodations: What we know so far. Lexinton: University of Kentucky, Mid-South Regional Resource Center; 1999.         [ Links ]

15. Bennett RE, Ragosta M. Technical characteristics of postsecondary admissions tests for handicapped examinees: a review of research. J. Spec. Educ. 1985;19(3):255-67.         [ Links ]

16. Tindal G, Fuchs L. A summary of research on test changes: an empirical basis for defining Accommodations [Internet]. Lexinton: University of Kentucky, Mid-South Regional Resource Center; 1999 Jul. Available from: http://www.specialed.us/discoveridea/topdocs/msrrc/Tindal%26Fuchs.PDF. Cited 2012 Apr 13.         [ Links ]

17. Lindstrom JH, Gregg N. The role of extended time on the SAT[R] for students with learning disabilities and/or attention-deficit/hyperactivity disorder. Learn. Disabil. Res. Pr. 2007 May; 22(2):85-95.         [ Links ]

18. Zuriff GE. Extra examination time for students with learning disabilities: an examination of the maximum potential thesis. Appl. Meas. Educ. 2000;13(1):99-117.         [ Links ]

19. Bolt SE, Ysseldyke JE. Comparing DIF across math and reading/language arts tests for students receiving a read-aloud accommodation. Appl. Meas. Educ. 2006;19(4):329-55.         [ Links ]

20. Rudner LM. Using standard tests with the hearing impaired: the problem of item bias. Volta Rev. [Internet]. 1978. Available from: http://www.eric.ed.gov/ERICWebPortal/detail?accno=EJ181561. Cited 2013 Jul 11.         [ Links ]

21. Cawthon SW. Accommodations use for state wide standardized assessments: prevalence and recommendations for students who are deaf or hard of hearing. J. Deaf Stud. Deaf Educ. [Internet]. 2007 Jan 1. Available from: http://jdsde.oxfordjournals.org/content/early/2007/06/21/deafed.enm029. Cited 2013 Jul 11.         [ Links ]

22. Pitoniak MJ, Royer JM. Testing accommodations for examinees with disabilities: a review of psychometric, legal, and social policy issues. Rev. Educ. Res. 2001 Mar 1;71(1):53-104.         [ Links ]

23. Ragosta M. Students with disabilities: four years of data from special test administrations of the scholastic aptitude test, 1980-83. 1987 Jan 1. Available from: https://research.collegeboard.org/publications/content/2012/05/students-disabilities-four-years-data-special-test-administrations. Cited 2013 Jul 11.         [ Links ]

24. Ragosta M. Performance and persistence: a validity study of the sat for students with disabilities. college board report No. 91-3. [Internet]. College Board Publications, Box 886, New York, NY 10101-0886 ($12.00).; 1991. Available from: http://research.collegeboard.org/publications/content/2012/05/performance-and-persistence-validity-study-sat-students-disabilities. Cited 2012 Jun 20.         [ Links ]

25. Fuchs LS, Fuchs D, Eaton SB, Karns KM. Supplementing teacher judgments of mathematics test accommodations with objective data sources. Sch. Psychol. Rev. 2000;29(1):65-85.         [ Links ]

26. Cohen AS, Gregg N, Deng M. The role of extended time and item content on a high-stakes mathematics test.learn. Disabil. Res. Pr. 2005;20(4):225-33.         [ Links ]

27. Bennett RE, Rock DA, Kaplan BA. Level, reliability, and speediness of SAT scores for nine handicapped groups. Spec. Serv. Sch. 1989;4(3-4):37-54.         [ Links ]

28. Section 504 Rehabilitation Act of 1973 [Internet]. 29 Usc.Sect.29 U.S.C. § 701 et seq., Pub. L. 93-112 Sep 26, 1973. Available from: http://www2.ed.gov/policy/speced/reg/narrative.html. Cited 2011 Sep 20.         [ Links ]

29. Americans with Disabilities Act of 1990 - Wikipedia, the free encyclopedia [Internet]. Available from: https://en.wikipedia.org/wiki/Americans_with_Disabilities_Act_of_1990. Cited 2013 Jul 11.         [ Links ]

30. Naciones Unidas. Normas uniformes sobre la igualdad de oportunidades para las personas con discapacidad [Internet]. Resolution 48/96 48a Reunión de diciembre de 1993. Available from: http://www.un.org/spanish/disabilities/default.asp?id=498. Cited 2011 Jun 25.         [ Links ]

31. Sireci SG. Validity issues in accommodating naep reading tests [Internet]. National Assessment Governing Board. 800 North Capital Street NW Suite 825, Washington, DC 20002. Tel: 202-357-6938; Fax: 202-357-6945; Web site: http://www.nagb.org; 2004 Jan. Available from: http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED500434. Cited 2012 May 09.         [ Links ]

32. National Center for Special Education Research (NCSER) Projects and Programs: National Longitudinal Transition Study-2 (NLTS2). National Longitudinal Transition Study-2 [Internet]. Available from: http://www.nlts2.org/. Cited 2011 Jul 21.         [ Links ]

33. Bakken T, Kortering L. The constitutional and statutory obligations of schools to prevent students with disabilities from dropping out. Remedial Spec. Educ. 1999;20(6):360-6.         [ Links ]

34. Cooper H, Nye B, Charlton K, Lindsay J, Greathouse S. The effects of summer vacation on achievement test scores: a narrative and meta-analytic review. Rev. Educ. Res. 1996 Sep 1;66(3):227-68.         [ Links ]

35. OECD. Knowledge and Skills for Life [Internet]. Paris: Organisation for Economic Co-operation and Development; 2001. Available from: http://www.oecd-ilibrary.org/content/book/9789264195905-en. Cited 2013 Jul 12.         [ Links ]

36. SITEAL. Sistema de información de tendencias educativas en América Latina [Internet]. Available from: http://www.siteal.iipe-oei.org/. Cited 2010 Dec 16.         [ Links ]

37. Comini C, Soares JF. Comparação do desempenhoem matemática dos alunosdasescolas públicas e privadas: A evidência do SAEB 99. Belo Horizonte: mimeo, INEP: Relatório pedagógico do ENEM 2002. Brasília: Ministério da Educacão; 2003.         [ Links ]

38. Duncan G, Brooks-Gunn J. Consequences of growing up poor. New York, Plymouth: Russell Sage Foundation, Plymbridge; 1999.         [ Links ]

39. ICFES. Colombia en PISA 2009 [Internet]. 2010. Available from: http://54.208.2.57/datos/Colombia%20en%20PISA%202009%20Sintesis%20de%20resultados.pdf. Cited 2011 Oct 20.         [ Links ]