English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
A good command of English language is fundamental for the professional development and the growth of the country. 1 Some health sciences areas in which learning English is necessary include academic research and development 2, handling instruction manuals for medical devices and instruments, using computer programs and interacting with English-speaking patients and their caregivers. 3
Similarly, in a globalized world, medical services have begun to be outsourced. Thus, it is possible to find that in areas such as radiology or interpretation of specialized medical examinations, foreign medical professionals, coming from countries where these examinations are practiced, are hired if they live in countries that can provide the service in different time zones. For example, if an x-ray is taken during the day in the USA, a doctor in India (during his daytime working hours) can review it at night. This makes assistance more efficient and allows lower costs for users.
In Colombia, that health personnel is able to master a second language is even more important considering that the government has adopted a strategy to position the country as a destination for health and wellness tourism and hopes to reach a share between 20% and 30% of the medical tourism market by 2032. 4 This type of tourism targets at those people who enter a country looking for specialized medical, dental or surgical care, which can be preventive, cosmetic, curative or wellness treatments. 4
To achieve this goal, one of the proposed objectives is to increase the English language proficiency levels in health professionals. 5 This objective was established after identifying that the level of English in health professionals was one of the deficiencies of the sector according to the consultancy firm McKinsey & Company, which conducted a baseline study of this public policy embodied in the Conpes 3678 of 2010. 6
Specifically, it is crucial for physicians to master English for the health tourism sector and thus be able to have contact with patients (who mostly come from English-speaking countries or use English as a lingua franca), prescribe medications and medical orders, and understand assertively their needs and symptoms, among others. 7 Moreover, these reasons are even more important considering that the lack of understanding between doctors and patients for not sharing a common language can prevent the former from correctly identifying symptoms, which increases the probability of diagnosis errors and incorrect treatment. 8
In order to assess the challenge for achieving this goal and to take advantage of the health tourism boom that the country is experiencing, it is necessary to establish the level of English in Colombian doctors. 9 Determining this information is possible and there are tools to calculate the proficiency level of medical graduates. Therefore, the first objective of this article is to present an analysis of the results of the English component of the Examen de Estado de Calidad de la Educación Superior - Saber Pro (Higher Education Quality State Exam) taken by potential graduates of different medical programs offered by universities between 2011 and 2015.
In addition, prior to the adoption of the strategy to position the Colombian health sector as an option for health and well-being tourism, in 2005, the Ministry of National Education (MEN by its acronym in Spanish) created the National Bilingualism Program Colombia 20042019 within the framework of an educational revolution. 10 With this program, the MEN established the Program for Strengthening the Development of Foreign Language Skills (PFDCLE by its acronym in Spanish) and set as a goal that 20% of graduates should be classified at intermediate or higher level in English by 2014. 1 It is worth noting that Colombia is not the only country that has a bilingualism policy in Latin America: Chile proposed the National English Strategy 2014-2030 11 with the program English Opens Doors (PIAP by its acronym in Spanish); Peru created the policy English, The Door to the World (2015-2021) 12, and Uruguay developed the Ceibal Plan. In these three countries, the policy focuses on improving performance in English language in students at initial educational levels (primary and secondary education).
The second objective of this article is to assess whether the goal (20% of graduates classified at intermediate or higher level according to the Saber Pro test) proposed by MEN for 2014 was met or not by medical programs in Colombia.
This document is organized in four sections. The first presents the justification and objectives, the second describes the database and the methodology used to establish whether different medical programs met or not the goal established by the MEN for 2014, the third presents the results and the fourth offers some final comments.
Materials and methods
In order to determine the proportion of future medical graduates that reached an intermediate level of English language proficiency or higher, the database developed by the Colombian Institute for the Evaluation of Education (ICFES by its acronym in Spanish) for the Saber Pro tests conducted between 2011 and 2015 was used. This is a standardized and mandatory test to obtain the degree in all higher education programs. The exam is given once a year, in two sessions for two days and may last between 4 hours and 40 minutes and 8 hours depending on the program, because not all students have to take the second section. In the morning, generic skills (including English) are evaluated and in the afternoon the specific skills of the study program are evaluated. The English section provides results comparable over time, as they are aligned with international reference frameworks, and assesses reading and language use skills; this is a written test with multiple-choice questions.
The ICFES database contains the performance level achieved by each student in the English component. This level coincides with the adjustments made by the Colombian government of the conceptual framework and measurement of language proficiency of the Common European Framework of Reference for Languages for the country (Table 1).
Table 1 Equivalences between the reference levels of the Common European Framework of Reference for Languages, the reference levels of the country and of the classification of the Saber Pro test.
CEFR: Common European Framework of Reference for Languages.
Source: Own elaboration based on Alonso et al.10
Table 2 describes the total number of students evaluated and the percentage of those who were classified as B+ in each of the years studied. The percentage of students of medical programs is around 3% of the total potential of university graduates. The number of medical students evaluated was 3 722 in 2011. The students of medical programs evaluated had a relatively steady growth between 2011 and 2013, then, it decreased in 2014 and increased again in 2015. The average number of medical graduates from 2011 to 2015 is 4 457 per year.
Table 2 Percentage of students in intermediate or higher level (B+) according to the Saber Pro test per type of program 2011-2015.
n: number of students evaluated.
Source: Own elaboration based on the data obtained in the study.
The results of the English test are available for all professional and technical programs in the country and they allow comparisons between programs of different types. Table 2 presents the population of medical students and, to contextualize the results of the medical programs, presents the results of administrative science programs, software engineering and other future graduates.
As described in Table 3, the universities that registered their medical students in the Saber Pro test went from 40 in 2011 to 44 in 2015. The average number of students per program increased from 93 in 2011 to 111 in 2015. The past year, the Universidad Metropolitana de Barranquilla registered 342 medical students in the Saber Pro test, while the Universidad de Nariño registered 22 students. The increase, of about 30% in the minimum and maximum number of students per university, partially explains the 20% growth in the average number of medical students per program.
Table 3. Descriptive statistics of the Saber Pro test according to the medical program. 2011-2015.
Source: Own elaboration based on the data obtained in the study.
A descriptive approximation was carried out using this database. First, the percentage of students from the 44 medical programs classified in each of the five categories of English language proficiency level was estimated according to Table 1. In addition, in order to infer about the achievement or not of the policy goal, parametric (Z test) and nonparametric tests were carried out (χ2 test) to determine if the proportions found are statistically higher than the 20% goal; both tests were performed for each program with the medical students that presented the test in 2014.
The statistics of the parametric test (and the corresponding correction) that was used was:
Where 
is the observed proportion of medical students classified in B+. This statistics allows refuting the null hypothesis that the observed proportion of students classified in B+ is ≤0.2 (20%). The alternative hypothesis is that said proportion is >0.2.
The trial statistics of the first equation follows a standard normal distribution, if a single comparison is made; so, to reject the null hypothesis, different corrections were used taking into account the number of comparisons made. In addition, the nonparametric test that was used involved the following trial statistics:
Where Oi. is the observed value of individuals that are classified in B+ and Ei is the expected number of individuals in B+ if the 20% goal was met. Said trial statistics allows verifying the same hypothesis obtained with the parametric test described above and follows a χ2 distribution with 1 degree of freedom, if only one test is made.
For this exercise, it was necessary to perform these tests on all the programs that registered students in the Saber Pro test. In other words, it seeks to determine simultaneously if each of the 42 programs that registered students in the test in 2014 met the goal or not. To avoid the bias generated when drawing a joint conclusion from joining individual test results, it is necessary to make an adjustment that takes into account the number of comparisons. This adjustment is made on the p-value and not on the statistics; for this purpose, the methods of Holm 13, Hochberg 14, Hommel 15, Benjamini & Hochberg 16 and Yekutieli & Benjamini were used. 17
Microdata and statistical tests processing was carried out using the statistical software R 18. The confidence level used for the conclusions was 95%.
Results
Before analyzing the results per program, it is important to note that once the results have been evaluated by category for Administration and related programs, Software and Medicine, the proportion of medical students classified in B+ was found to be higher than in the other programs (Table 2). In 2011, while Administration and related programs classified 7.8% of their students and software 10.8% in B+, Medicine had 20.3%. Likewise, for the year 2014,10.7% of the students of Administration and related programs, 15.4% of software students and 28.6% of medical students were classified as B+. For 2015, 31.8% of medical students, 11.9% of students of Administration and related programs, and 19.1 % of software students were classified as B+. It should be noted that medical students had a higher number of students classified in B+ during the study period than the other areas; furthermore, this number increased by 40.6% from 2011 to2014andby 11.8% in the past two years.
Figure 1 represents with a dot the proportion of medical students classified in B+ and with colors, public institutions (in black) and private institutions (in gray). This graph allows identifying six interesting results:
Source: Own elaboration based on the data obtained in the study.
Figure 1 Percentage of medical students in intermediate or higher level (B+) according to the Saber Pro test. 2011 -2015.
1 ) Most Colombian universities exceeded the goal established by the PFDCLE for 2014, specifically, 23 programs (it is important to note that the corresponding statistical tests will be presented later on).
2) The number of universities that have a proportion >20% of students classified in B+ increased during the study period: it went from 17 programs in 2011 to 27 in 2015.
3) Programs offered by public universities do not reach relatively high percentages of students in B+ level, but a large part of them is located above the goal, especially in 2014.
4) There is no medical program in Colombia for which all of its students are classified as B+ in English language proficiency.
5 ) The number of programs with over 70% of students classified in B+ is relatively low; no public university was found in this group.
6) The dispersion in the proportion is very high: there is a relatively large number of universities that do not have 10% of their students classified in B+; the dispersion in private universities is higher, while for public universities is lower.
To go into the detail of each of the programs and the fulfillment of the goal established for 2014, it was necessary to determine if the proportions calculated were statistically greater than the goal (Tables 4 and 5). It was found that the null hypothesis could be rejected for 16 medical programs only by applying either the parametric or nonparametric test; that is, the proportion of medical students who achieved B+ was >20%. Said programs are offered by the Universidad de los Andes, the Universidad Icesi, the Universidad Javeriana -Bogotá Campus, and the Universidad Nacional de Colombia.
Table 4 Results of nonparametric test regarding the difference of proportions in students with B+ classification according to the medical program. 2014.
* The universities are listed alphabetically.
Source: Own elaboration based on the data obtained in the study.
Table 5 Results of parametric test regarding the difference of proportions in students with B+ classification according to the medical program. 2014.
* The universities are listed alphabetically.
Source: Own elaboration based on the data obtained in the study.
If the fact that multiple comparisons were made is not considered and if p-values were adjusted, two more universities (Universidad Militar Nueva Granada and Universidad del Tolima) would have met the goal in 2014. It is worth noting that although the ρ values for most universities were different in magnitude among the criteria used -Holm 13, Hochberg 14, Hommel 15, Benjamini & Hochberg 16 and Yekutieli & Benjamini 17-, the decision was always directed in the same direction: rejecting the null hypothesis. However, for the Universidad Mlitar Nueva Granada, when the parametric test was used only under the Yekutieli & Benjamini criterion, the null hypothesis was rejected, which did not occur in other cases. Given that under the nonparametric test the null hypothesis was rejected when using any of the criteria, it is concluded that, for this year, the university did not meet the goal.
The results of applying the statistical tests on the results of the English component of the Saber Pro 2011-2013 and 2015 were not included due to space limitations, but are available upon request.
Table 6 shows that of the 44 programs analyzed, only 9 met the goal in each of the years of the study period. Likewise, it was found that five universities improved: Universidad Autónoma de Bucaramanga, Universidad El Bosque and Universidad Tecnológica de Pereira met the goal for 2014, while Universidad de La Sabana and Universidad Pontificia Bolivariana have met the goal since 2012. On the contrary, Universidad Militar Nueva Granada did not have a clear trend, since it did not meet the goal in 2011, 2012 and 2014. The medical programs of Universidad Icesi and Universidad Javeriana, Cali Campus, met the goal of the government since the first cohort of graduates who presented the Saber Pro test, that is, since 2014 and 2015, respectively.
Table 6 Summary of target compliance (20% of graduates in B+ per medical program). 2011-2015.
✓: The university met the goal, which is determined by rejecting the null hypothesis in favor of the alternative with 95% confidence; x: It was not possible to determine if the university met the goal since the null hypothesis could not be rejected in favor of the alternative to include a definition; NA: The university did not register medical students for that year, either because no student classified in B+ existed or had not yet graduated the first class of students of the program.
* The universities are listed alphabetically.
Source: Own elaboration based on the data obtained in the study.
Discussion
The goal (20% of graduates in B+) of the PFDCLE for 2014 countrywide was reached by 28.6% of the medical students evaluated. However, it should be taken into account that only 18 (41.9%) of the 43 medical programs met the goal that year, a percentage that increased to 45.5% in 2015.
To the best knowledge of the authors, this is the first study that uses the Saber Pro tests to study the English proficiency level of future graduates of medical programs and that assesses whether or not the government goal established for 2014 was fulfilled regarding the proportion of students in intermediate or higher level in English language proficiency. Previous studies that have used Saber Pro tests have done it to study different aspects of Colombian medical graduates, for example, the need to evaluate, through clinical simulation scenarios, skills that are impossible to evaluate by means of a written exam 19, the evaluation of specific skills in health professionals 20 or the importance of training competent professionals that respond to the needs of the country. 21
The only similar study 22 uses the results of the Saber Pro 20112014 tests to assess the English proficiency level of the graduates of the Information Technology sector in Colombia and emphasizes on systems engineering students. The results show that the programs of the software sector did not meet the goal established by the PFDCLE for 2014 in the country 22. In other words, medical students had a better performance than systems engineering students in English.
Conclusions
Results show that the medical programs did meet the bilingualism goal established by the government for 2014. However, it should be taken into account that only 18 (41.9%) of the 43 medical programs met this goal, that is, less than half of the programs met the goal individually.
On the other hand, these results do not fully reflect the state of English language training received by medical graduates because the Saber Pro test only evaluates reading comprehension and language use skills, leaving out other linguistic skills of equal importance such as listening comprehension, and oral and written expression. These competences could be evaluated if other language proficiency tests (such as MET, IELTS, TOEFL or APTIS) were used to assess listening, reading, grammar, writing and oral communication skills. In spite of this, the English section of Saber Pro is the best and only tool available to researchers today to make a national diagnosis of this skill.
Future studies should evaluate if the English proficiency levels achieved by the students are the result of the added value that they receive during their time in the university or if they are product of primary and secondary education. Similarly, it is recommended that future works extend the analysis to other health areas such as Nursing and to other sectors such as Hotel Management and Tourism, which are also part of the Health Tourism sector. Finally, there is still a long way to go in terms of bilingualism if we take into account that this skill is essential for Colombia to achieve a global position in the health tourism sector, which is and will be key to the long-term competitiveness of the Colombian economy.
