Introduction

In the world scenario, there is a growing number of validation studies in the health field aimed at the production of technological resources with the purpose of supporting care in different contexts, both in the training and qualification of professionals, as well as in management and patient care.¹ From this perspective, there are studies aimed at nursing which make it possible to build tools that support the science of this area of knowledge and, consequently, the promotion of a safe, qualified praxis based on scientific evidence.^1,2 It is noteworthy that the validation procedures seek to verify the suitability, quality, legitimacy and credibility of an instrument based on the opinion of experts in the area of the resource theme and/or coming from the view of the target public, to which the validated object is intended.^1,3

Furthermore, numerous materials can be submitted to validation stages, such as research instruments, care protocols, educational booklets, standard operating procedures, virtual learning objects, algorithms and others.³ These materials may require one or more types of validation that must be specific and/or adapted, adequate, rigorous and linked to the material elaborated; among the types of validation, the content, cross-cultural, appearance, use and usability validation stand out.^1,4

Content validation assesses whether the content produced is relevant, correct and adequate for what is proposed.⁴ Transcultural validation determines the pertinence of translating an instrument to another reality regarding language and culture.⁵ Appearance validation analyses whether the technology built has suitability in relation to graphic and cultural resources and whether it is understandable in terms of the language adopted.⁶

Use validation seeks to trace the degree of user satisfaction with the tool built. As for usability, the validation investigates the effectiveness, efficiency and relevance of the use of the technology in relation to the intended objective in a more broad manner.^6,7 Therefore, each type of validation needs an appropriate analysis of the data, which is based on a rigorous theoretical-methodological framework, since from this analysis it will be determined whether the technology in question is validated and suitable for use or requires adjustments so that it can be effectively used in practice.^1,3

Thus, the data analysis stage is important for validation studies, as it is considered essential to establish the relevance and quality of the constructed material. However, the reliability of the analysis process must be in line with the material and type of validation performed.³ Thus, it is essential to investigate the types of data analysis elucidated in the validation studies available in the scientific literature in order to identify and compile the various analysis methods and verify their association with the validation process used. Thus, there is the following guiding question “What types of analysis are used in validation studies in nursing?” and the objective was to identify and map the types of analysis of validation studies in nursing.

Methods

Study design. This is a Scoping Review with a research protocol registered in the Open Science Framework (DOI:10.17605/OSF.IO/YH9UZ) based on the recommendations of the Joanna Briggs Institute (JBI) Reviewer's Manual,⁸ according to the proposed stages in the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation⁹ according to the theoretical framework of Arksey and O'Malley.¹⁰ This type of study seeks to identify and map the main concepts that support an area of knowledge through the survey of scientific productions and relevant studies in order to list the existing gaps in the literature. For this, the following stages are adopted: 1) delimitation of the research question; 2) identification of relevant studies; 3) selection of studies; 4) data mapping and; 5) grouping, summarization and presentation of the results.¹⁰ With regard to the elaboration of the research question and descriptors listed, the Population, Concept and Context (PCC) strategy was used with the use of descriptors in health sciences (DECS ) and their correspondents in English indexed in the Medical Subject Headings (MESH), where P (population) - validation study, C (concept) - data analysis and C (context) - nursing research. Thus, the following question was established: What types of analysis are used in validation studies in nursing?

Study location and data collection period. A primary investigation was carried out by crossing Data analysis AND Validation study AND nurse research in the National Library of Medicine (PUBMED) and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases to identify and survey the most common keywords frequently in surveys. Thus, the search strategy was obtained with the help of the Boolean operators AND and OR: Validation Study OR (Validation OR Instrument Validation) AND Data Analysis OR (Analysis) AND Nurse Research OR (Nursing Sciences). It should be noted that the search in each database was adapted to its specific search engines, but the compatibility of descriptor combinations was maintained. The capture of studies was performed in July 2020. The following platforms were used to search for articles: U.S. National Library of Medicine (PUBMED), Cumulative Index to Nursing and Allied Health Literature (CINAHL), SCOPUS, COCHRANE, Web of Science, PSYCHINFO, Latin American and Caribbean Literature in Health Sciences (LILACS). In view of the theses and dissertations that make up the gray literature, the following were explored: CAPES Thesis and Dissertation Portal, Education Resources Information Center (ERIC), The National Library of Australia's Trobe (Trove), Academic Archive Online (DIVA), DART- Europe E-Theses Portal, Electronic Theses Online Service (EThOS), Open Access Scientific Repository of Portugal (RCAAP), National ETD Portal, Theses Canada, Theses and dissertations from Latin America.

Selection criteria. The following inclusion criteria were defined for the eligibility of the studies: research related to the types of analysis of validation studies in nursing, studies published in full and available electronically on the CAPES Periodicals Portal through the Federated Academic Community (CAFe). No restrictions were established regarding language and time of publication. Editorials, letters to the editor, opinion articles and reflection articles were excluded. In addition, duplicate materials were counted only once.

Sample definition and Study variables. The study selection process took place in two stages, the first starting from reading the titles and abstracts, which were verified by peer reviewers. In the second stage, the studies were analyzed by reading the material in full and the indicators were extracted: year of publication, country of origin, type of study, level of evidence based on the JBI recommendations,¹¹ which consists of the framework that guides the scoping review, the scientific frameworks for validation and the types of analysis (tests, statistical calculations, etc.)

Data processing and analysis. The selected data were organized in a spreadsheet using the Microsoft Excel 2010® software, evaluated using simple descriptive statistics, and presented in graphs, tables and figures.

Results

From the search in the databases, 809,975 studies were initially identified, which were analyzed in stages that are described in Figure 1, so that a final sample of 881 studies was obtained.

Figure 1 Flowchart of the study search and selection process 

The selected studies consist of articles (841; 95.5%), dissertations (21; 2.3%) and theses (19; 2.2%). With regard to the period of publications, the year 2019 stood out (152; 17.2%) (Figure 2).

Figure 2 Distribution of years in which the 881 studies that made up the sample were published 

Regarding the origin of the studies, Brazil (377; 42.8%), the United States of America (64; 7.2%) and Turkey (47; 5.3%) stood out, as shown in Figure 3.

Figure 3 Countries where the studies were developed 

Regarding the type of method, there was a prevalence of those classified as methodological (352; 39.9%), cross-sectional (163; 18.5%), validation studies (125; 14.2%), descriptive (50; 5.6%), observational (14; 1.5%), cohort and exploratory (6; 0.7%), quasi-experimental (4; 0.4%), accuracy, concept analysis, randomized clinical trial and action research (2; 0.2%) each, case control, sequential triangulation and systematic review (1; 0.1%) each. It is noteworthy that some studies (149; 16.9%) did not describe the method used. The levels of evidence of the studies were evaluated, in which there was a predominance of level five (477; 54.1%), followed by level four (238; 27.0%), level three (9; 1.0%), level two (4; 0.5%), and level one (3; 0.3%). In addition, the data collection indicators (main methodological references applied in the analysis of validation studies in nursing and the tests and statistical calculations performed) were synthesized and presented in Table 1.

There were also variables such as the types of analysis of validation studies in nursing and the amount of analysis used in each research, which are described in Table 2.

Discussion

Regarding the analysis of published studies, the predominance of articles was noticed. This finding is associated with the fact that this type of production is carried out in a shorter period of time compared to the others and, often, they consist of clippings from larger research, such as research reports, dissertations and theses.¹² Regarding the year of publication, 2019 stood out, which is in line with the growing popularization of validation studies to produce and legitimize quality instruments, which are essential to support and structure nursing care, with a view to reinforcing it as a science, as it proposes a practice based on literature and scientific knowledge.¹³ It is noteworthy that such progression in the construction of these studies over the periods may have also been driven by the launch of the “Nursing Now” campaign, which aims to promote the appreciation of the profession by the end of 2020 through investment in the production of knowledge, technical-scientific advancement and health education, thus encouraging the development, the innovation of nursing science and the production of technologies in the area.¹⁴

When analyzing the origin of the materials that made up the sample, a significant portion of Brazilian archives was evidenced, which is supported by the gradual expansion of stricto sensu graduate courses in the health area, which foster the progress of science and technology, which require a careful evaluation before their implementation and, therefore, validation studies become essential as they attest to the quality of the tool developed.¹⁵ Regarding the type of method used in the analyzed studies, there was a prevalence of methodological studies to the detriment of the others, which can be understood by its delimitation, since it allows the researcher to build his validation project structured in stages, from the idealization, search in the literature, elaboration and proof of its reliability for practice, therefore, it is more employable and, thus, more accepted.¹⁶

The data related to the references and benchmarks used in the validation analysis show Polit and Beck as the ones that obtained the highest number of citation. This finding can be explained by the wide dissemination of their works and the international recognition of the authors identified as specialists in research methods in health and nursing.¹⁶ Another highlighted reference was that of Pasquali, who directs his assumptions to psychology, but is constantly adapted by health researchers to the area, generally associated with the need for validation of criteria, content and construct.¹ Pasquali's psychometry is marked by the completeness of its process, as it provides support for the construction and validation of the instrument, thus contemplating the necessary stages in a clear and objective way for the material to be considered usable, such attributes make the reference preterable among the others.^1,17

With regard to tests and statistical calculations, the prevalence of Cronbach's Alpha was verified. This fact corroborates a study¹⁸ carried out in 2017, which states that this is the most suitable for evaluating the estimate of the internal consistency of an instrument and is based on the degree of covariance between the items. Therefore, with Cronbach's alpha, the researcher is able to determine if the instrument developed is in line with the target audience, if it is repetitive and if it is consistent for what it intends to measure.¹⁹ Another highlighted statistical calculation was the Kaiser-Meyer-Olkin (KMO) Test, which refers to whether the Factor Analysis (FA) is appropriate to the research data, and this is directly associated with the Bartlett's Sphericity Test, which presents the adequacy of the FA according to the hypothesis testing of the correlation and identity matrix.²⁰

Thus, research²¹ highlights the importance of the KMO test by allowing the verification and adjustment of variables as a fundamental process in validation studies, as they directly influence the constitution of the factors that represent the construct. Regarding the types of analysis, there was a predominance of factor analysis, which is classified as exploratory and confirmatory. Exploratory factor analysis in a validation procedure allows attesting to the representativeness of the data, thus making the material developed more succinct and objective, by grouping similar items.²²⁾ While confirmatory, as the name implies, confirms the structure and construct of the instrument, considered indispensable in the validation processes, as it directs the management of the necessary factors so that the material does not deviate from its proposal.¹⁸

Still on the types of analysis, the wide use of the Content Validity Index (CVI) was found as one of the most important dimensions in the analysis of a material, since it allows the determination of the validation of the material based on statistical calculations defined according to the pre-established.^18,23,24 The CVI is in line with Polit and Beck, who guide the performance of this calculation in three ways: through the mean of the proportions of the items classified as relevant, through the sum of each CVI individual divided by the total number of items, or the sum of all relevant items divided by the total number of items.¹⁹

However, for Pasquali, this process is understood as Content Validation Coefficient (CVC) and differs in statistical calculations. In this case, each item and the instrument as a whole are validated according to previously established criteria and by means of an expert opinion, which provides the adequacy of the content and semantics of the material to make it understandable to the audience for which it is intended and compatible with the purpose for which it is available.²⁵ Both verify the suitability, representativeness and relevance of the instrument in relation to the phenomenon that it aims to measure. It is highlighted as a relevant characteristic to the subjectivity of these methods when considering the individuality in the interpretation of each judge and, consequently, can result in distortions in the evaluation.^1,26

It is evident that the presence of one more method of analysis in 50% of the studies implies the need for several tests to measure the validation of a material and prove its reliability. These results corroborate an integrative review²⁷ by confirming the use of multi-methods in data analysis. Finally, the data extraction procedures are highlighted as a limitation of the study, since many studies that composed the sample described the data analysis stage succinctly and some did not fully portray the types of analysis and statistical calculations used to validate the material produced, which may imply limitations of the findings presented.

Conclusion

The main types of analysis used in validation studies in nursing are the exploratory factorial and the content validation index. Regarding the methodological framework, Polit and Beck stood out, and as a statistical test, Cronbach's Alpha for structuring and evaluating the tool produced. All these processes allow the improvement of the technologies developed and ensure their quality, in addition to enabling their use in health services, since such materials guide and the routine of professionals and turn it dynamic by subsidizing their practical performance and strengthening the theoretical basis.

Furthermore, as contributions from the research developed, the mapping of the analyzed information is elucidated, considering that from the results obtained, researchers will be able to base the analyses of their validation studies by understanding which types of analyses are most suitable for each type of approach according to the findings of this material, as well as statistical tests and calculations that can be associated and appropriate references and benchmarks