Dear Editor,

We greatly appreciate the comments made by Rojas-Botero et al.¹ on our paper entitled "Critical reflections on the Municipal Epidemiological Resilience Index used for public policy decision-making regarding the control of the COVID-19 pandemic in Colombia".²

We acknowledge that public health, understood as the well-being of the community, should be the focus of all our efforts as health professionals. In this sense, decision-making amid a major emergency, such as the one we are currently experiencing due to the COVID-19 pandemic, requires a balanced and calm judgment that considers both the general context and local specificities, as well as the best available scientific evidence, that is, evidence that not only supports decisions, but also prioritizes benefits over risks.

As in other recent catastrophic events, during the COVID-19 pandemic, it has been necessary to resort to "real-time evidence," to consult continuously the data emerging from observation and the results of health surveillance and interventions, to build statistical and epidemiological models, and to compare scenarios using sensitivity analysis under different assumptions and conditions. All of this has been an ambitious exercise to respond to this emergency with the best tools available.

In this exercise, there may be risks derived from the "biases" that are inherent to epidemiological thinking and are viewed as a, visible or concealed, enemy to be avoided. In our case, the objectives are primarily aimed at controlling or eliminating the transmission of SArS-CoV-2 while avoiding overburdening health systems to mitigate the impact of the pandemic, minimize its social impact and reduce the number of preventable deaths.^2,3

Therefore, critical reflections should be understood as a contribution of academia to society because it seeks to bring a broader perspective to problems and a dynamic strategy that facilitates the understanding of different situations in their multiple dimensions, with its own resources and with independence, which evidently helps to make the most appropriate decisions. Thus, we appreciate the letter to the editor submitted by Rojas-Botero et al.,¹ to your journal since it represents a step forward in the informed, serene and necessary debate on the management of the COVID-19 pandemic in Colombia, as well as a document that healthcare delivery planners will undoubtedly value. Accordingly, we would like to make the following observations about said publication:

First, we would like to go through the definition of "resilience index." Although Rojas-Botero et al.¹ state in their letter to the editor that the definition is clear, the technical documentation of the Municipal Epidemiological Resilience Index (IREM) does not support such a statement, nor does it elaborate on the assumptions used as a basis for such a definition and does not indicate the conflicting theories that support it or the epistemological process that led to such a definition, raising doubts about its validity, as argued below.

This is not intended to start a debate on a definition; however, in the face of such an initiative, it would be helpful to know what theoretical contributions are being made, because de novo definitions and theories must be based on concepts that have been established and widely discussed in the existing literature to ensure that they are built on solid foundations. Moreover, in this specific case, naming the index "epidemiological resilience" does not imply that social variables should not be included. On the contrary, it makes it even more necessary to incorporate epidemiological models of infectious diseases into its design.

It is worth stressing that even the most basic epidemiological model should include social and health care variables. For example, the epidemiological triad model includes variables related to the agent, the host and the environment,^4,5 and the Lalonde report, besides these three factors, considers the response of the health system or eco-epidemiology, which involves the interaction of multiple systems.⁶ In this context, it is essential that epidemiological models of social determinants of health on which, according to official documents, the country's public health policy is based, take these aspects into account. This is a discussion that goes beyond rhetoric to encompass the very concept that is to be measured and used as a tool in decision-making.

As mentioned above, Rojas-Botero et al.¹ state that the definition of the construct to be measured by the IREM is clearly explained in its technical document. An excerpt from that document backs up this claim, stating that "this construct has been defined as Epidemiological Resilience, to the extent that it indicates the capacity of the territory-population dyad, as well as the dynamic process of overcoming stressors in an adaptive way while maintaining functioning."¹ However, although this dyad is referenced in the Ministry of Health and Social Protection's publication, nowhere is it specified how this interrelationship was conceived and measured, nor is it indicated which measurements are applicable to the territories.⁷

The construction of scientific language is done socially from the discussions of the sciences and the exchange of knowledge between them. Thus, a concept's polysemy requires a sound scientific construction that brings together and outperforms the arguments for and against it. Rojas-Botero et al.¹ insist that the definition of resilience proposed in the IREM technical document is entirely valid since it does not allude to other meanings of the term, particularly those based on social and behavioral sciences and among which, according to them, there is no consensus because there are no universally accepted referents for this term and because language is dynamic and changing. We have pointed out that there are multiple indicators of resilience specific to the COVID-19 pandemic, which have already been discussed and used, contradicting the authors' claim of a lack of consensus and international references,¹ as well as their unquestionable conclusion that "the meaning proposed in this index is fully valid." In this regard, it is critical to remember that validity in epidemiology is a solid, demanding, and verifiable process.

Second, the authors reiterate that "validity is the extent to which an instrument truly measures the construct it intends to measure," and that what should be evaluated is the index's construct rather than its name. To that purpose, they present a table outlining the three steps required to validate health measuring instruments.¹ However, the process is actually more complex than just content, face and construct validation because it must also evaluate its dimensionality, reproducibility (internal consistency and test-retest reliability) and construct, which is complemented by convergent, correlational, discriminant and known groups.^8,9

The critical analysis of the IREM performed in our previous publication² only contained the critical assessment of its face, content and construct validity because the methodological documents for such an instrument did not present elements to critically analyze all the steps of its validation. In this regard, it is worth noting that the authors were expected to document and include these steps in the technical document before implementing it, because if an instrument fails the initial face, content, and construct validity phases, it is not appropriate to proceed with the validation process.

The name is an essential component of the anatomy of a good index, along with its justification and the elements that make up it, and it is also part of its metadata.¹⁰ In turn, the data are the sources that feed information about the index's components, and it is expected that those components will be associated with the index's name at the most significant points of the process it measures, rather than just a few.¹⁰ In this sense, the authors' desire to reduce the weight of the IREM name and focus on the construct (the pieces that comprise it) is questionable, which, as mentioned above, is simply one of the steps in the instrument's validation process that, according to our critical analysis, is not properly met.²

Third, Rojas-Botero et al.,¹ state in their example on the immunization variable that the total value of the contribution of this dimension to the IREM of 0.5 becomes 0.2498 when the exemplified values are weighed. However, even though this statement is correct, the authors did not address many of the other aspects mentioned in the critical analysis concerning face, content, and construct validity. Furthermore, the suggested contributions for improving the measurements of each of the proposed dimensions and the addition of variables reflecting variations in infection over time, transmission and impact on mortality and health services are not discussed either.²

Also, we would like to refer to the authors' statement that "what has been observed is that this tool efficiently discriminates the different territories of the country based on their capacity to restart restricted economic, social and cultural activities without having a significant epidemiological impact on the population".¹ This should be carefully reviewed as no supporting data are presented.

Figure 1 shows the rates of new cases and new deaths per million people before and after the implementation of the IREM, which allow us to confront the statement of Rojas-Botero et al.¹ Although it is not possible to speak of a causal relationship between the implementation of this index and the increase in the number of positive cases and deaths due to COVID-19, it is clear that the timing of implementation was not the best in terms of new cases and new deaths, as we noted in our article.² Moreover, it is clear that this situation continued to worsen until the end of the evaluated period. Furthermore, it is important to mention that Colombia went from being the seventh country with the highest incidence of deaths per million inhabitants on June 1, 2021, to being the fourth on July 1, 2021 (Figure 2).

Source: Elaboration based on Ritchie et al.¹¹

Figure 1 New cases and new deaths per million people due to COVID-19 in Colombia as of July 3, 2021. Note: the yellow vertical line indicates the moment of implementation of the Municipal Epidemiological Resilience Index proposed by the Colombian Ministry of Health and Protection.  

Source: Elaboration based on Ritchie et al.¹¹

Figure 2 Deaths per million people in the 20 countries with the highest rates. 

Making public a proposal to resume restricted economic activities may be misinterpreted by the population, worsening adherence to general self-care standards if the messages are not clearly and adequately conveyed.² All this may be reflected in the results shown in Figures 1 and 2.

We believe that the IREM can be improved, both in its conception and its operation, to contribute efficiently to decision-making. For this reason, we consider that this is an opportunity for academia to make contributions to its construction with technical arguments aimed at adding value and community welfare, preserving the attributes of opportunity, efficiency, equity, and utility. However, we are concerned about the general framework of Resolution 777 of 2021 issued by the Ministry of Health and Social Protection¹² and strongly believe that it should be revised, adapted and, if necessary, repealed to take into account all of the above arguments in its implementation, to select the best information and support, and to ensure that the greatest number of people involved accept it.

It is understandable that to facilitate decision-making in the process of resuming all restricted economic and social activities, an instrument to guide such work is needed. Nevertheless, creating this instrument requires a validation process that supports its implementation.

Before concluding, we would like to point out that the IREM design did not take into account the experiences of other countries that use different indicators for decision-making in the management of the pandemic that could be useful for better implementation of elimination strategies over mitigation strategies, which do not appear to be working fully in our context based on the results observed. We would also like to add that other authors have recently published a paper that backs up with their proposal what has been stated throughout our dissertation.¹³

Finally, we would like to clarify that what motivates us to publish these reflections is our desire to make an academic contribution to all the processes related to overcoming the crises caused by the pandemic because only a respectful dialogue will allow us to move forward to find more effective solutions.