English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
The nutritional transition, both in Brazil as in other countries, has intensified with demographic variations, urbanization, industrialization, and globalization. This made evident in contemporary society the maintenance of some nutritional deficiencies: Anemia (hidden hunger) and emergence of pathologies, such as obesity and overweight due to stressful and intense routines with intake of industrialized foods and fast foods. Additionally, the persistence of malnutrition (real hunger) in some socially vulnerable population areas, making it more complex to confront the paradox among malnutrition and obesity, scarcity and excesses 1. Thus, it constitutes Food and Nutrition Education as a way of achieving the promotion of health through preventing and combatting these nutritional problems 2.
The development of an instrument that, as a teaching-learning strategy, permits reflecting on the theme and discussing emerging information, whether in the classroom or in the family and community setting, is useful to encourage changes in relation to eating habits 3. Within this context, the role of the nursing professional as a health educator stands out, whether in school, the community or in their daily lives in health services, responsible for integrating their knowledge with popular knowledge to construct and reconstruct knowledge, given that this is the most appropriate route for disease prevention and health promotion of the population 4.
For a school to be considered an adequate facility to apply health education programs, it must, besides developing food and nutrition education practices, provide nutritionally adequate food, both at lunch and in the places where food is commercialized, to encourage students to consume healthy foods and to materialize the teaching-learning process 5.
The school environment has been chosen by public policies on food and nutrition as a place to promote healthy eating, recognized as the priority locus to develop healthy habits and choices. However, much concern exists regarding the theoretical-methodological bases that govern these educational actions, with gaps registered between the educational actions undertaken, which still remain closer to the traditional biomedical model, and the actions and strategies that still focus on recovery of health, rather than the health promotion approach, and of the Food and Nutrition Security 6.
Therefore, food and nutrition education constitutes a field of action to achieve food and nutrition security and for health promotion. Highlighted as fundamental strategy, the School Health Program, whose principal focus consists in contributing to students’ education through actions related to prevention, care, and health promotion, includes in this context the promotion of healthy feeding. Among the executable results of Food and Nutrition Education are: Contribution to the prevention and control of chronic non-communicable diseases and nutritional deficiencies; appreciation of different expressions of food culture; consolidation of regional eating habits; diminished food waste; promotion of sustainable consumption and healthy eating 7.
The aforementioned health education resolutions are also among the initiatives established as nurses’ attributions and include the interdisciplinary work of three major areas: Healthy environment, participation of health services, and health education. Hence, the school environment (School Health Program or School Health Promotion Program) and the communities (Family Health Program) are the nurses’ privileged spaces for teaching and implementing actions to improve health conditions, and among them, food and nutrition education 8.
When considering the school as the place destined to construct knowledge, it is necessary to develop materials to facilitate the teaching-learning process, and their validation is important for greater credibility of the instrument and, consequently, of the intervention results 9.
This study sought to construct and validate an instrument titled “Pre- and Post-Intervention test”, destined to verify knowledge on healthy eating in schoolchildren, by using judges (specialists selected) to validate content and appearance and the target audience for semantic validation (usability).
Methodology
This was a methodological study to validate an instrument destined to verify knowledge on healthy eating in schoolchildren from 4th and 5th grade of primary school education, ranging in age between nine and ten years (target population), conducted by using judges. The study is part of a PhD thesis of the Graduate Program in Nursing of the Health Sciences Center at Universidade Federal de Pernambuco (UFPE).
The judges were chosen based on the scientific production declared in the Lattes Platform, from the following professionals: Nutritionists, nurses, pediatricians, pedagogues/educators, and designers. This search was done by subject, using the following keywords: “Nutritional Education AND Healthy Eating AND Student” and “Healthy Eating AND Student AND Educational Technology”. In this selection, the following criteria were considered, adapted from the Fhering model: Being a professional with proven experience on the theme addressed “Healthy Eating” and/or have knowledge regarding the preparation of ludic materials 10.
Then, a “Letter-Invitation to participate as a judge” was sent to the selected professionals via e-mail, in which the aim of the research was briefly introduced. If there was agreement, the following materials were sent through Google Drive: Informed Free Consent Term (IFCT); questions related to the professional profile, which aim to prove the minimum five-point score of the pre-established criteria to select the judges, and questions related to the validation of the instrument to verify the knowledge of schoolchildren about Healthy Eating “Pre- and Post-intervention test”. The instrument to validate the “Pre- and Post-intervention test” has 12 items (one to evaluate content, three for language, four for appearance, and four for suitability for the target population) and takes into account the age and schooling of the participants. After each question, a space was left for judges to include their considerations and suggestions when they deemed it necessary.
The items were scored by using a Likert scale 11-13, with the following options: (i) No Disagreement: Totally agree OR agree OR do not agree and do not disagree; (ii) Disagreement: Disagree OR totally disagree. For analysis, the following scores were adopted: Totally agree (2 points), agree (1 point), do not agree and do not disagree (0 points), disagree (-1 point) and totally disagree (-2 points).
A 30-day period was established, from the receipt of the material, to return the “Pre- and Post-intervention test” instrument with the answers deemed relevant. After this deadline, a new e-mail was sent requesting that the evaluation be completed within 10 days. If no reply was received the judge was substituted. During that stage, 30 professionals were contacted until obtaining 22 judges (minimum sample amount).
This amount, necessary for the validation process, was determined through the following formula: n = Zα2.P.(1-P)/d2. Where: Zα = tabulated value corresponding to the confidence index (95 %); P = proportion of agreement among the judges (85 %); d = difference of acceptable proportion among the judges, i.e., the maximum permissible error (15 %) 11,14. It is recommended that a validation process have six to twenty judges, and that, depending on the total considered, have at least three specialists from each area to be selected 11.
Based on the aforementioned, the sample of 22 judges should be comprised of the following professionals: Five nutritionists, five nurses, four pediatricians, four pedagogues/educators, and four designers. However, given that none of the eight designers contacted answered the invitation, a new search was conducted and this time among professionals selected from other categories. Searching for those with experience in educational technology and/or in the area of game elaboration, six nutritionists, six nurses, four pediatricians, and six pedagogues/educators comprised the final sample.
As stated, the judges validated the content and appearance of the “Pre- and Post-intervention test” instrument. Considering the purpose and applicability of this instrument for teaching-learning, the target audience was used for semantic validation (pilot test). The present instrument was elaborated from the information contained in food guides by the Ministry of Health 15,16.
The Scale-Level Content Validity Index (S-CVI) was verified, corresponding to the arithmetic mean of the proportion of items that received the evaluation of “no disagreement” by the judges. This validation was considered approved with S-CVI ≥ 0.90 11,13.
The judges’ congruence of opinion was verified, item-by-item, using the I-CVI in which the value of the I-CVI for each item is the proportion of judges that scored it in positive or neutral manner (no disagreement). Then, the proportion mean (or level) of “no disagreement” was calculated by the total number of judges and items evaluated. This validation was considered approved with I-CVI ≥ 0.80 11,13.
The Binominal test, through the p value of the proportion (the H0 was rejected if p ≤ 0.80), was used to select the items that should be revised/modified (items validated at significance level ≤ 0.05) 11,13.
Suggestions for changes by the judges were duly examined. After making the pertinent modifications, the instrument was submitted to the target audience for the semantic validation (in total, 12 students). At that time, in addition to testing the applicability of the instrument regarding appearance, language, and content comprehension, the time spent to carry out the proposed intervention was also verified 11,13.
The pilot test took place in May 2017, in a school of the municipal network of Recife (Pernambuco, Brazil), randomly selected. The 12 students participating in this validation (six from 4th grade and six from 5th grade of primary school education, initial grades) were also selected randomly after being separated through the following inclusion and exclusion criteria: Inclusion - 1) being regularly matriculated in the school, 2) being between nine and ten years of age, 3) being literate; exclusion - 1) having cognition problems, 2) having hearing, visual, or communication deficiency. An information letter on the research and the IFCT was delivered to students selected, for their parents and/or legal guardians to permit their participation in the research. During the pilot test, all occurrences and observations by students, related to lack of understanding of the items and suggestions for modifications, were duly documented in the “field diary”, for them to be evaluated and included in the final version of the instrument when deemed coherent. Figure 1 summarizes the stages of the semantic validation (pilot test of the instrument about Healthy Eating “Pre- and Post-intervention test”).
Source: Own elaboration.
Figure 1 Flowchart of semantic validation (pilot test) of an instrument “Pre- and Post-intervention test” to verify knowledge on healthy eating in schoolchildren from 4th and 5th grade of primary school, ranging in age between nine and ten years. Recife-Pernambuco, 2018.
This study (CAAE: 60824216.8.0000.5208) was approved by the Research Ethics Committee at UFPE (Opinion 1.814.698 in 09/11/2016) and conducted according to resolution 466/2012 of the National Health Council 17.
Results
The majority of the judges were females (91 %), with graduate education (96 %), with 64 % PhDs and 32 % Masters, and 82 % worked in a higher education institution. Regarding the teaching experience, a time interval was found from 5 to 46 years, with an average of 22 years. With respect to experience in the elaboration of educational technology and/or in elaboration of games, 64 % said they had experience, of these, the shortest time was six years and the longest was 23 years, with a mean of 13 years of experience.
Table 1 presents the proportion of “no disagreement” by judges with the respective S-CVI, which results, for the instrument as a whole, in S-CVI = 0.93. Upon observing the result by judge, it is noted that in only three of the 22 judges (Judge 4, 11, and 13) the score of “no disagreement” was below the validity coefficient considered approved (S-CVI ≥ 0.90).
Table 1 Validation of an instrument to evaluate knowledge on healthy eating (Pre- and Post-intervention test) in students from the 4th and 5th grades of elementary school, aged between nine and ten years, according to the opinion of the 22 judges. Proportion of non-disagreement among judges and their respective Scale Level Content Validity Index (S-CVI). Recife-Pernambuco-Brazil, 2018.
Note: S-CVI/AVE = Scale-level Contente Validity Index/Averange Calculation Method. Validity Coefficient = S-CVI ≥ 0,90. (S-CVI = sum of points S-CVI/AVE divided by the number of judges).
Source: Own elaboration.
Table 2 presents the judges’ responses by instrument item, which allows observing that: In seven of the twelve items (1.1; 2.1; 2.2; 3.1; 3.3; 3.4 and 4.2) over 80 % of the judges evaluated positively (totally agree OR agree); in another two items (3.2 and 4.4) over 70 % of the judges evaluated in the same manner; and in the three remaining items (2.3; 4.1 and 4.3) the positive evaluation was made by over 68 % of the judges. Observing by component of the instrument, the arithmetic means of the proportion of judges who evaluated positively were content 100 %, language 77 %, appearance 86 %, and suitability 75 %.
Table 2 The judges’ responses by item of an instrument to evaluate knowledge on healthy eating (Pre- and Post-intervention test) in students from the 4th and 5th grades of elementary school, aged between nine and ten years. Recife-Pernambuco-Brazi 2018.
Source: Own elaboration.
Table 3 shows that the “Pre- and Post-intervention test” instrument was approved with I-CVI ≥ 0.80 in the 12 items evaluated, with a mean proportion of “no disagreement” among judges of 0.93. By using the Binominal test, it was observed that six of the 12 items (1.1; 3.1; 3.3; 3.4; 4.2 and 4.4) could remain in the original version (items validated with p value of the proportion at significance level ≤ 0.05). The rest of the items (the three from the language component; item 3.2 of the appearance component; and items 4.1 and 4.3 of the suitability component) were revised and modified.
Table 3 Validation of an instrument to evaluate knowledge on healthy eating (Pre- and Post-intervention test) in students from the 4th and 5th grades of elementary school, aged between nine and ten years. Proportion of non-disagreement among judges and their respective Item-Level Content Validity Index (I-CVI). Recife-Pernambuco-Brazil, 2018.
Source: Own elaboration.
Note: Validity Coefficient I-CVI ≥ 0,80. (*) Binominal Test, through the p value of the proportion (the H0 was rejected if p ≤ 0,8). (**) Items validated at significance level ≤ 0,05. Mean = sum of non-discordance points (= 246) divided by the number of judges (= 22), and this result divided by the number of items evaluated (= 12). Punctuation adopted (Likert scale): Total agreement = 2 points; Partial agreement = 1 point; Neither agree Nor disagree = Zero point; Partial disagreement = minus 1 point; Total disagreement = minus 2 points.
Based on the observations of the judges, the following changes were made in the “Pre- and Post-intervention test” instrument: Composed initially by 11 questions (eight in the first part and three in the second) and with 36 illustrations in the last part, had the second part deleted. The number of illustrations was maintained; however, some foods were replaced by regional foods and the name of the food was placed below the respective image.
Because the highest percentage of “disagreement” among the judges occurred in relation to the suitability of the instrument to the age range, rather than submitting the instrument again to the judges, it was decided to validate the new version with the target audience (pilot test of the “Pre- and Post-intervention test”), approved in its entirety by all the participating students. Table 4 exposes that version of the instrument.
Discussion
To validate is to make something legitimate, true, through a process that aims to provide an instrument capable of interfering positively in the quality of the desired results (9). Healthy eating habits are formed and consolidated during childhood, and can be maintained throughout life, therein, the importance of introducing this theme in school 18. The theme of nutrition looking at healthy eating has diverse publications from the Brazilian Ministry of Health 7,15,16,19. However, offering instruments with a ludic-didactic approach improves the parameters of nutritional education, as was proven in a study with elementary school children, which demonstrated significantly higher results in the intervention school, with evidence that the figures illustrated by the games influenced students in the teaching-learning process 20.
This “Pre- and Post-intervention test” instrument was elaborated to evaluate the effect of educational approaches on healthy eating and scored in the validation process by the judges 0.93 in both S-CVI (Table 1) and I-CVI (Table 3); with the following arithmetic means of the proportion of judges who evaluated positively (Table 2): 100 % in the content, 77 % in the language component, 86 % in the appearance component, and 75 % in the suitability component of the items to the target population. A similar result, with S-CVI 0.95 and I-CVI 0.98 is reported in a methodological study conducted with 12 judges, in the validation as to the appearance and content of a serial album for the promotion of food security by using regional foods 21. Another study on healthy eating, validated by 11 judges and five children from primary school, also reached the minimum value proposed in the literature for most items evaluated by the judges (0.80), and acceptance and agreement of the target population (0.99) 22.
Regarding appearance, in the present study some foods were replaced by others from the region. Food from the region was also used in the validation study of the serial album for the promotion of food security 21. This fact points to the need to always prioritize regional foods in instruments designed to assess the effect of educational approaches on healthy eating.
Adequacy of the items to the age group, number of questions of the instrument to achieve the objectives, and the clarity of the questions to facilitate the answers are aspects that were also the focus of other validation studies, such as a questionnaire to evaluate the diet of young women, regarding folate-specific feeding frequency 23, and an instrument to evaluate health promotion in the school environment 24.
Education and health professionals have currently discussed the feeding of children and adolescents, given that inadequate eating habits have contributed to the increase of diseases, like obesity, diabetes, hypertension, diverse food allergies, among other health problems. Authors highlight the need to give due importance to Food and Nutrition Education in the training of children and young people, starting from early childhood education, going through the initial and final years of elementary school through high school 25.
Elaboration and availability of active teaching-learning methodologies, duly validated, to evaluate the effects of an intervention, like that shown in this study, favor the development of skills of nursing students involved in education and in the promotion of children’s health, which enables them to take an effective and coherent approach with the role of future health professionals 8.
Educators in the process of knowledge mediation should use active methodologies that, in addition to favoring the teaching-learning process (by awakening in students a reflective and questioning spirit, based on the use of critical and logical reasoning), offer students an environment that stimulates social interactions and active and autonomous action, so that, thus, they can build their own learning process 26.
Conclusions
This study reinforces the importance of developing and validating instruments before their use to grant them greater credibility and reliability. Another fundamental point is that, by avoiding misinterpretation during their application, ensures that the objectives of the research are appropriate to the target audience.
Thus, the “Pre- and Post-intervention test” instrument to evaluate the knowledge of schoolchildren about healthy eating, duly validated regarding content, language, appearance, and suitability to the age range and level of schooling of the students, is a tool that can be used by educators (during their education activities of the “School Health Program”) and primary care professionals (principally from “Family Health Program” and “Community Health Agents Program”), during health education activities looking to encourage healthy eating.
The “Pre- and Post-intervention test” was used to diagnose learning disabilities about Healthy Eating, already worked with students in the classroom in the School Health Program, to highlight knowledge gaps and, after the intervention on Healthy Eating, to assess the effects in students’ knowledge.
