Introduction
Clinical trials (CTs) are studies with an experimental design which require the voluntary participation of people in whom various health treatments or interventions are evaluated to determine their efficacy, effectiveness and/or safety. Accordingly, they allow one treatment or intervention to be compared with another, or with a control, to observe its effect or result 1. Clinical trials are critical for the health sciences because they allow safe, evidence-based treatments and interventions to be implemented and more sound decisions to be made, aimed at quality care 2.
From the onset of the CT methodology, it was determined that a randomly selected experimental and control group needed to be used. However, this division has historically been arbitrary, and only beginning in 1920 were CTs carried out under these conditions. Likewise, clinical epidemiology and evidence-based medicine (EBM) have proven to be the main areas contributing to the development of this design 3.
Clinical trials have been considered a paradigm of epidemiological and clinical research, because their methods try to control different variables as much as possible to allow a cause-effect relationship to be established with the least bias possible, proving the validity of the experiment and its potential generalization 4. These properties position them as one of the best levels of health evidence, contributing to decision making for the treatment, prevention, etiology and identification of damage from a disease, as well as determining the prognosis and natural history of a disease 5.
Controlled CTs may be said to have essentially been used in pharmacology 6. However, various health areas like pediatrics, midwifery 7, and cardiology 8, among others, have used the experimental design to show effectiveness through a systematic method that contributes greater evidence for their application.
Clinical trials have increased significantly worldwide, in Latin America, and in Colombia. The report of studies in Clinical Trials shows that, as of 2020, a total of 344,320 studies had been conducted in 216 countries, 298,287 of which were experimental studies. The North American continent is the main CT producer (133,378), followed by Europe (81,979) and Asia (35,510). In Latin America, Brazil is reported to be the first CT producer (7,286), followed by Mexico (3,631), Argentina (2,762), Chile (1,614) and Colombia (1,338) 9. With regard to the accelerated growth and increasing application of CTs, some authors state that, while the predominance of these studies has prevailed in North America and Europe, Latin American countries are considered attractive for carrying out CTs due to the following factors: a high population and easily accessible racial diversity, the prevalence of chronic diseases and longevity, high medical standards that make it possible to perform CTs, and a strong doctor-patient relationship that favors participant retention 10,11.
In any case, the performance of CTs in the different health areas requires dedication, effort and a series of obstacles to be overcome, which in many cases become the reason for their failure. In this regard, aspects like lack of investigator training, costly methods, scarcity or lack of funding, complex and time intensive approval requirements, difficulties in clinical practice application, and barriers to participant recruitment, among others, become challenges that investigators have dealt with through collaborative strategies (generally academic) which allow these studies to be performed 12-14.
Furthermore, CTs' level of evidence may be said to be conditioned by a rigorous methodological process which requires demonstrating internal and external validity. Therefore, their planning and implementation must show evidence of a sample recruitment and selection process, transparent assignment of subjects to the different branches or groups, masking and/or blinding, standard operating procedures and a detailed analysis of both the variables of interest as well as the control variables 15.
Today, in the health sciences, explanatory (or classical) CTs and pragmatic CTs can be carried out, which intend to show outcomes under ideal conditions (for the first) or under usual practice conditions (for the second) 5,12. Both have different scopes, and therefore their objectives, purpose, recruitment methods, assignment, interventions and analysis differ 16. Back in the 60s, Schwartz and Lel-louch stated that each method (explanatory and pragmatic) answers one of two problems which arise when comparing two treatments; in this regard, explanatory CTs are aimed at "understanding these treatments," seeking to discover the differences between the two; while pragmatic CTs are aimed more at the need to "make a decision" about applying one treatment versus the other, taking into account the contextual conditions of real practice 17. A current example of the use of these two methods was experienced during the COVID-19 pandemic, which required a rapid study of different treatments (including vaccine production), under the explanatory and pragmatic conditions required by the urgency of controlling the pandemic 18,19.
Historically, explanatory CTs have had a higher priority in clinical practice and are the gold standard for evidence-based practice. However, many of the healthcare guidelines are based on small, non-conclusive CTs, and therefore some authors do not consider it practical to wait for all the everyday clinical questions to be answered through this type of CT 20. On the other hand, pragmatic CTs account for only 2% of all the CTs performed and, despite this, have achieved relevance based on the premise that their results may translate better and be more useful in regular clinical practice and for those responsible for policy making 21. However, progress is slow toward pragmatism in randomized clinical trials because its methodology is still unknown, and there is insufficient conviction and data, record and clinical procedure organization to increase the degree of pragmatism in CTs. Therefore, both types of CTs can be said to still have advantages and disadvantages; meanwhile, it is essential for healthcare researchers to have a detailed knowledge of their methodological requirements, to be able to perform these studies more accurately.
This article, the result of a narrative review, describes the methodological aspects to be kept in mind in the design of explanatory and pragmatic CTs, which are essential for either of these experimental studies.
Search materials and methods
The scientific literature in Google Scholar, Scopus and PubMed was reviewed. The following stages were followed to determine the methodology: drafting of a guiding question based on the CT methodological process, identification of descriptors, search for papers in the databases mentioned above, application of filters using the inclusion and exclusion criteria, and analysis of the selected papers.
The following guiding question was used for the search:
What methodological aspects should be taken into account in designing explanatory and pragmatic CTs?
The search was performed using a combination of the following descriptors: Clinical Trials, Pragmatic Clinical Trials, Methodology, Models Statistical. The inclusion criteria were: articles published in the last five years (2016-2020), written in any language, derived from research or reviews, and describing methodological aspects to be taken into account for CTs. Articles showing specific CT results on any health topic, editorials, duplicate papers and articles describing the ethical component of CTs were excluded.
A total of 77 papers were found which were filtered by title and abstract, obtaining a sample of 47 articles for analysis. Figure 1 shows the number of articles found and the reasons for exclusion.
Results
Clinical trials make up a single experimental design, but with many variants. In this regard, the analysis of the selected papers revealed six relevant methodological aspects which show differences in the design of explanatory and pragmatic CTs: 1) the purpose and objective, 2) participant recruitment, 3) participant assignment, masking and/or blinding, 4) intervention, 5) analysis of the data obtained, and 6) internal and external validity (Table 1).
Methodological aspect | Explanatory CTs | Pragmatic CTs |
---|---|---|
Recruitment |
|
|
Participant assignment, masking and/or blinding |
|
|
Intervention protocols |
|
|
Data analysis | Greater control over confounding variables and able to show statistically significant results 24,43,44. |
|
Purpose and objective
In CT design, the term "purpose" refers to the study's nature or matter of concern, while the "objective" is related to the intended aim or goal of implementing the treatment or intervention. The purpose of an explanatory CT lies in testing a treatment or intervention under ideal conditions with highly selected participants 19,20; while its objective is to show the efficacy and/or safety of a controlled treatment in a group of people with similar conditions regarding a health or illness phenomenon 19,21. In pragmatic CTs, the purpose is similar as far as wanting to test a treatment or intervention, but under usual practice (real) conditions in which a large population of people are cared for 19,20. The objective of pragmatic CTs is to show not only the effectiveness of the intervention but also its applicability in usual practice settings and in groups of people with diverse conditions 22-24.
Recruitment
Participant recruitment for CTs is a challenge, as they are often not recruited on time nor with the expected sample size. This may be due to unforeseen problems or an overestimated recruitment rate. There are determinist (conditional and unconditional) and stochastic (Poisson model, Bayesian, simulation model) models for predicting recruitment in the design stage of clinical trials. These models vary considerably in the factors included and their complexity. The determinist models require specification of only a few parameters and, while they are easy to apply, they are likely to be unrealistic. On the other hand, stochastic models allow variation around an average recruitment rate 25.
The most frequently used recruitment methods include social marketing or indirect recruitment (through flyers, pamphlets or web sites) 26 and outreach or direct contact with possible study candidates 27. Along with recruitment, retention strategies are needed to ensure that participants remain in the study. In this regard, successful retention strategies have been used like detailed explanations of the study with its benefits and risks, ongoing contact with the participants through scheduling follow up appointments with reminders, offering flexible appointments, creating a study identity for the patients, assigning recruitment staff with interpersonal skills which show empathy and are culturally competent with the study participants, providing financial incentives and tokens of appreciation for participation (like souvenirs and certificates, in countries where these are legally authorized), involving the community in study design, recruitment and retention 28; and implementing tracking methods for hard-to-reach participants 29. In Colombia, payment for participating in CTs is forbidden by law 10,30.
Regardless of the type of CT to be implemented, the following three aspects should be kept in mind for strategic recruitment planning: CT design and protocol implementation, the viability and selection of the target institution for the CT, and communication with the people involved 31.
Participant assignment, masking and/or blinding
The CT sample size allows results to be extrapolated and increases its potential generalization; thus, it is important to determine the representative number of units of analysis to consider in the intervention groups (arms or clusters) 32. The assignment of interventions to the groups of participants may be randomized or nonrandomized. In a randomized CT, the total sample size is a function of both the number of groups and their size. Invariably, one of these is fixed and the others are determined using pre-set formulas. In this case, the intracluster correlation coefficient (ICC) is considered, which measures the degree to which the observations or measurements are correlated within a cluster 33. Random assignment is applied in almost all CTs which compare two or more treatments or interventions with each other. The random element in the assignment process is used to avoid, or at least minimize, the impact of bias on the estimate (or selection) 34.
The statistical factors which predetermine the risk reduction calculation and the number needed to treat are generally related to a (type I) error, ß (type II) error, the statistical power and the incidence in unexposed groups. These errors may cause false positives or negatives, lead to incorrect rejection of the null hypothesis or to incorrectly estimating the expected frequency of the event in the control group, and to conditioning the expected difference of effect in the outcomes of interest. Therefore, it is important to determine the probability of success based on prior evidence; otherwise, the best option is to establish a 50% probability when prior benchmarks are available 34,35. To calculate the sample size, the following aspects should be kept in mind: writing a research question, stating the null hypothesis and an alternative hypothesis, selecting the primary outcome measure and the corresponding type of statistical test, considering a range of plausible effects (and variability, if applicable), and selecting the type I or II error based on the objective, clinical considerations and/or phase of the study. This calculation can be done using statistical packages available online which have formulas for each statistical model 36.
Further, the concept of masking or blinding can be mentioned as interchangeable terms to show the concealment of interventions from both the participants as well as the investigators, to reduce measurement bias 37. In this regard, it is important to specify that blinding refers to concealing information (for example, the initial results of an assessment prior to the intervention), while masking refers to concealing the treatments or interventions.
Intervention
Treatments or interventions are implemented in each CT, depending on its design. In this regard, various designs have been classically implemented over time to allow the experiment to be applied through different groups: parallel (simultaneous experiments in a series of groups), crossover (with the treatment and control in the same participant), paired (the participants are combined in pairs according to certain characteristics), withdrawal (initial participation in a treatment, followed by random assignment to continue with the treatment or receive a placebo), and factorial (evaluating several interventions in a single trial without increasing the sample size, as long as the treatments function independently) 35,38. For randomized CTs, the current literature also shows adaptive designs 34,39 (which allow for evaluation of preliminary results at intermediate points of the intervention or treatment, to modify the study design). For pragmatic CTs, sequential multiple assignment designs are currently cited (mentioned?) 40,41 (which compare two or more alternative interventions, incorporating a second randomization stage; in this way, the interventions are sequenced based on the person's response).
Data analysis
The primary and secondary results analysis plan should be specified before beginning the study. There are two basic types of analysis in CTs, according to the proposed objectives. One is the analysis according to the protocol, in which results are ideally obtained from participants who successfully concluded the treatments or interventions implemented; and the analysis by intention to treat, for those who only completed part of the treatment or intervention. This second type of analysis is important in that it can show the actual effectiveness of the intervention evaluated with just a few sessions 42-44.
The measures of association to be used in CTs are relative risk (RR), as they are prospective studies. In this case, RR is considered as the ratio of absolute risks between the group of individuals exposed to the intervention and those not exposed. When the RR is equal to 1, it is interpreted as there being no association between the intervention and the outcome; therefore, the resulting confidence intervals do not include the unit. When the figure obtained is greater than 1, it means that it is likely that the expected outcome will occur after the intervention is implemented; on the other hand, if it is less than 1, this likelihood decreases. The interpretation of a favorable or unfavorable outcome should be kept in mind in terms of the variable being measured (for example, an RR greater than 1 will be favorable if some risk factor is expected to be reduced, but unfavorable if mortality is what is being measured) 45. Another measure of association is the odds ratio (OR), or risk reduction, which shows the differences between the intervention group and the control group. The CTs which try to measure efficacy clarify the risk reduction using this measure of association.
Relative risk reduction (RRR) corresponds to the difference in risk between the two groups compared with the control group. The problem with using RRR is that it cannot evaluate the real effect if the rate of events in the control group is unknown 45,46. Finally, the number needed to treat (NNT) is taken as the measure which represents the number of patients who would need to be treated to avoid an adverse event 46,47.
Finally, in the interpretation of the results, it is essential to distinguish between a statistically significant and clinically significant difference. A difference may be significant from a statistical standpoint but have little clinical value or impact.
Internal and external validity
The validity of a CT represents the likelihood of its being bias-free. Internal validity shows the proximity to the truth sought in the results obtained. Thus, to achieve internal validity, the design must show coherence in its question or hypothesis, objectives, uniform characteristics of the participants, correct calculation of the sample size and control of confounding variables 24,48,49. High internal validity means that the differences found between the groups are related to the intervention tested in the trial; this means that the study's final outcome is a result of the treatments or interventions implemented and not of other factors. In summary, the internal validity of a CT is directly related to the design, implementation and presentation of appropriate study reports. The two main threats to internal validity are bias and random error. Bias refers to a systematic error which leads to a deviation of the results. The typical sources of bias are data collection, statistical analysis or data interpretation errors; this leads to the actual difference between the study groups possibly being under or overestimated. The four main sources of bias in clinical trials are selection bias (controlled with randomization), performance bias (controlled through adequate adherence to the treatment or intervention and controlling for confounding variables), detection bias (controlled by blinding) and burnout or attrition bias (controlled through the intention to treat analysis) 35.
On the other hand, external validity helps show to what extent the results obtained are generalizable and can be applied to populations similar to those of the study 24,48,49.
It should be noted that internal validity is a prerequisite for a study to have external validity. If the inclusion and exclusion criteria are carefully established, specific study samples can be selected, but there is also a risk of jeopardizing the generalization 35.
To improve the transparency of clinical trials, databases like ClinicalTrials.gov9 have been created to follow the changes between the planned and published studies, and to keep investigators up to date on the ongoing clinical trials. The studies must be registered in these databases before enrolling the first participant. Most medical journals request registry in a study database as a prerequisite for publication.
Furthermore, as a quality measure, standards have been consolidated for presenting trial reports. For instance, there is the CONSORT declaration, which is a tool that standardizes all the aspects to consider for publishing a CT. CONSORT consists of five domains made up of 22 items which help generate a structured report of the following aspects: title/abstract, introduction, methods, results and discussion. It also has a flow chart to describe the CT stages 50. The objective of the CONSORT declaration is to prevent incomplete CT information and show validity and reliability, which challenges the authors to fully comply with this standard 51. Some specific areas of knowledge have their own extensions in this declaration, which are constantly updated and complemented, with the aim of achieving greater accuracy in study reporting 52,53.
Pragmatic CTs also have the Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) as a tool to quantify the degree to which a CT incorporates pragmatic principles. It is advisable for investigators to self-assess the degree of pragmatism of their proposed trial using this tool.
The PRECIS-2 consists of nine criteria (eligibility, recruitment, adjustment, organization, delivery, adherence, follow up, primary outcome and primary analysis). Each criterion has a Likert scale score of 1 to 5, with 1 being a very explanatory trait and 5 being the highest pragmatic trait 54,55.
Discussion
As has been indicated, CTs are experimental studies whose characteristics vary according to their design. Explanatory CTs have an important role, but interventions in clinical practice rarely fit the strictly controlled environment of this type of study, while pragmatic CTs create the possibility of applying treatments and interventions in routine practice 21.
Recruitment clearly does not only depend on the strategies implemented by the research team, but also on the clinical and structural barriers which increase or decrease the possibility of participant enrollment and retention. In this regard, a meta-analysis of cancer CTs showed that more than half (55.6%) of cancer patients do not participate in CTs because there are none available for their type of cancer or the stage which is being treated, and when a trial is available, an additional 21.5% is not eligible. Taken together, these structural and clinical factors are the reason why more than three out of four patients (77.1%) do not participate 57. All the same, the participation of patients and the public in CT enrollment and retention has been shown to lead to a modest improvement in enrollment when organized recruitment strategies are implemented, especially when people with prior experience with the study's health condition are included 28.
On the other hand, the methodological rigor of CTs may also lead to participant dropout, given the many improvisations which may occur in the implementation of these studies. In this regard, it has also been proven that the dropout rate in some CTs is greater when they have low methodological quality or when there are technical errors, more than due to the effects of the treatments themselves 69.
Regarding sample size and cluster assignments, the investigators have several options when faced with a limited number of groups and a predetermined intracluster correlation coefficient (ICC) indicating that very large cluster sizes may be required to reach the desired power. In this case, the recommended option would be to increase the number of groups, if possible. Otherwise, a decision must be made between having a smaller group size without achieving the ideal 33. These situations may lead to errors in participant randomization; one of the most common errors is involuntary randomization of ineligible participants, especially in Phase III CTs. Less often, people who are suitable for the CT and willing to participate are not randomized. Both errors are serious, both for the person to whom the treatment is administered without meeting the eligibility criteria as well as for the loss of participants who, while suitable for the study, are not chosen. The biggest aggravating factor is the fact that there is very little reporting of these errors in the publications and the CTs do not clearly described the methods used for the randomization sequence, which makes it even more difficult to perceive these errors 70
There are significant differences in the application of the intervention protocols in explanatory and pragmatic CTs. Therefore, it is important to clarify that neither design is better than the other; each simply specifies a desirable intervention or treatment under different conditions, with different objectives, which allows different results in each case, but both contributing to the clinical setting. Thus, each CT will need to clearly specify the explanatory or pragmatic characteristics and their respective analysis, to show how well the therapies work in a routine care setting versus the ideal 60. The objective of each CT can be said to lead to a different primary outcome; one example would be spine surgery, with noticeable differences in that an explanatory CT can show the primary outcome of physical function through the walking ability gained and the performance of physical tasks, while the primary outcome of the pragmatic CT evaluates the degree of disability, including assessment of pain intensity and the impact of symptoms on the patients' social life, sex life, and ability to travel, in addition to the ability to perform physical tasks 21. Consequently, pragmatic CTs not only determine the efficacy of the treatment, but also its effectiveness, evaluated from different individual, family and social aspects which impact people's lives and are improved through the intervention performed.
Today, an increased use of pragmatic clinical trials is advocated in nursing and the various healthcare areas, in general. A sequential multiple assignation randomized trial (SMART) is a valuable design which is currently receiving more attention in nursing because it provides pertinent clinical trials using the comparison of two or more alternative interventions, incorporating a second stage of randomization based on the person's response 40. In this regard, the sequential design becomes essential evidence for the nursing process because it identifies which of the interventions work better, the ideal sequence of interventions for the general population, and in which type of population they function most effectively.
On the other hand, it should be noted that there is a subclass of pragmatic CTs whose results are derived from registries and clinical charts which describe the interventions carried out in actual practice. Essentially, this type of study is attractive because it is inexpensive, since it saves on costly interventions, time investment and qualified staff. However, there is often minimal control over which data to collect and how to do so, and therefore investigators face challenges in methodologically specifying how to handle incomplete, missing or inconsistent data 40,60.
Even if pragmatic CTs are presented as a savior to bridge the existing gap between the controlled requirements of explanatory studies and the scant possibility of their application in the real world, it must be recognized that a current disadvantage of pragmatic CTs is their scarce publication, in which specific guidelines must be met to prove the level of pragmatism. Thus, it is a challenge to write and publish a pragmatic CT, as it must prove to what degree the variables were controlled and the level to which the intervention was allowed to advance to show its effectiveness, regardless of the diversity of the participants. The major challenge for investigators is to specify the method used, since there is no standardized methodological rubric to guide pragmatic CTs, as there is for explanatory CTs.
Some specific health specialties have evolved from the explanatory level to pragmatism over time. One specific case can be seen in cardiology, where the level of pragmatism has increased moderately, showing more flexibility in aspects like participant eligibility, target institutions, the interventions and the primary outcome. Thus, over a five-year period, pragmatic cardiology CTs had more intervention implementation sites, larger sample sizes, longer follow-up, and mortality as the primary final outcome 71.
Conclusion
Both types of CTs can be said to be beneficial today, and both must overcome barriers to their implementation in the clinical setting. Internal and then external validity are directly related to the design, implementation and presentation of adequate study reports. The methodological evolution of CTs calls for small efficacy studies to be increasingly applied in larger settings to achieve the potential for generalization which will help improve decision making in health care and, thus, healthcare quality.