1. Introduction

Currently, the medical devices industry is a highly relevant and dynamic healthcare sector. According to the World Health Organization (WHO), worldwide, the yearly sales of this type of technology exceeded USD$ 210,000 million, representing an annual growth rate of approximately 6% [¹]. In the area of clinical diagnosis, the technological advancement of these devices has brought laboratory tests closer to the patient. This type of testing is termed: Point of Care Testing (POCT) [²,³]. Considering the advantages of obtaining timely readings of the biological parameters required to define the patient’s most appropriate treatment, these types of device have managed to permeate the organizational culture of the healthcare institution in departments / critical areas outside hospitals (home care, General Practitioner’s surgery and primary care) and in hospitals (emergency rooms, conventional and intensive care units) [⁴]. As a result, currently, one in four tests is carried out at the patient care site, with a growth rate of 12% per year [⁵]. Specifically, for the global market of POCT diagnostic devices, a projection from USD$ 23.7 billion in 2017 to USD$ 38.1 billion in 2022 is estimated [⁶].

Considering their characteristics, POCT equipment is instrumental to governments in developing economies when drafting proposals for improvement of healthcare quality and coverage in a decentralized manner. In [⁷] an evaluation of the implementation of decentralization policies in three Latin American countries: Chile, Bolivia and Colombia was carried out. In [⁸] the decentralization experience was analyzed in four developing countries: Ghana, Zambia, Uganda and the Philippines. In addition, [⁹] presents the evaluation of decentralization policies in Pakistan. In general terms, the results of these evaluations show the need for technological equipment, such as POCT, to consolidate improvements in these healthcare systems [¹⁰]. Specifically, according to [⁴], the growing demand for this type of analyzer ensures continuous evolution of different aspects, such as: manufacturing, design, cost reduction, portability and reduction of the exposure of personnel to waste, among others. As a consequence, multidisciplinary assessments of technology are increasingly necessary, taking into account the impact on the healthcare institution, benefits, risks and investments [¹¹]. Additionally, decision making for the selection of medical devices should consider the characteristics of the user interfaces, given that the clinical outcomes may be influenced by the training, competence and experience of the end user [¹²-¹⁴]. In this sense, Health Technology Assessment (HTA) is a comprehensive form of research that examines the short and long-term technical, social, economic, ethical and legal consequences derived from the use of technology (direct or indirect) and its desired and undesired effects [¹⁵].

Hospitals have been induced to incorporate robust methods or tools that support the decision-making process in the acquisition of medical devices. In addition to the accuracy of the device and its potential benefits, other indicators concerning the risks incurred by the technology are considered [¹⁶]. In [¹⁶] there is a tendency to evaluate the motivation of those responsible for the requisition of medical equipment, in the drive to implement more reliable methods than empirical ones. Accordingly, this work investigates the criteria for HTA by analyzing the concepts related to multi-criteria tools, specifically incorporating the Analytic Hierarchy Process (AHP) as a methodology for decision making.

2. Methods

In this work, a literature review was carried out in order to identify the assessment models, criteria and institutions involved in HTA. The following search criteria were used: "HTA + medical devices / medical equipment"; "Point of care + HTA"; "Blood gas analyzer + HTA"; "Blood gas analyzer + errors"; "Blood gas analyzer + organization"; "Blood gas analyzer + clinical laboratory"; "Point of care + clinical laboratory"; "Blood gas analyzer + emergency"; "Blood gas analyzer + management"; "Blood gas analyzer + safety", both in Spanish and English languages. In relation to the analysis of the assessment models, and the criteria in institutions involved in HTA, the literature review identified 62 articles with which we compiled a digital database for the project. For this, the information is structured according to the following fields: names of the authors, title of the article, methodologies (HTA or mini-HTA or any other unspecified methodology), multi-criteria techniques and main assessment criteria (health problems and use of technology, description and technical characteristics, safety, clinical effectiveness, costs and economic evaluation, ethical analysis, organizational / institutional aspects and human resources, patient and social aspects, legal aspects, among others). In addition, 14 guidelines and methodologies proposed by international HTA agencies were reviewed, allowing the conceptualization of methodologies used internationally [¹¹,¹²,¹⁶,²³,²⁴,³⁰,³¹,³³-⁴⁴].

For the identification of alternatives, the database of the Emergency Care Research Institute (ECRI Institute) [³²] was used. Subsequently, a refinement of this database was carried out using five requirements: (i) existence of sanitary records in Colombia and the Techno-surveillance program, (ii) availability of technical and safety data sheets for the equipment and supplies, (iii) access to technical / scientific personnel for training or maintenance; (iv) connectivity capacity to the information system of the healthcare institution and (v) technical capacity to meet the operational capabilities required by the health institution.

Concurrently, the current POCT equipment acquisition process was identified in the healthcare institution where the study was conducted. This allowed us to establish the complex integration of the requirements and criteria in an instrument of evolution with 180 indicators. From the information gathered in interviews with the personnel involved in the acquisition process, a SIPOC diagram was made. SIPOC is a modeling tool that facilitates the description of a given process in a simplified way, identifying suppliers, inputs or materials for activities, flow of activities, results of the activities and clients of each activity. All this allows the systematical analysis of the entire process and its environment [¹⁹].

Sequentially, the multi-criteria decision tool from AHP was selected, which is widely used in different areas, such as: health, political, economic, social and management sciences [²⁰-²⁶]. One of the relevant characteristics for the selection of this tool was its natural capacity to perform bi-univocal comparisons through matrix operations. In this way, it is possible to establish priorities among the elements of a level, with respect to an element of the next higher level. Fig. 1 shows the methodological scheme implemented for the blood gas analyzer equipment, based on Saaty and Lee [²⁷].

Source: The Authors.

Figure 1 Methodological scheme for the generation of the AHP model for the blood gas analyzer equipment. 

Step 1: Selection of the group of experts was made on the basis of their level of technical expertise and having the proficiency necessary to assess the criteria and alternatives.

Step 2: The definition of the criteria was based on judgments issued by the group of experts and on the review of the literature described above. Finally, the criteria were refined in conjunction with the healthcare institution management team.

Step 3: At this stage, a group of potential alternatives of POCT blood gas analysis equipment that met the minimum requirements from the healthcare institution was proposed.

Step 4: The results of the previous step were structured in three hierarchy levels. In the upper level the objective was defined. In the middle level the criteria were established. In the lower level the sub-criteria and research questions were determined.

Step 5: For the development of this step, the proposition in [²⁷] was taken into account. As a result, the relative importance of the sub-criteria was calculated according to the proposed criteria. For this, a comparison was made in pairs of the importance of the control criteria in relation to the general objective. Additionally, a ponderation by pairs of the importance of the merits was carried out in relation to each control criterion.

As recommended in [²⁰], once each of the comparison matrices were established by each expert, the information was consolidated into a square matrix quantifying the geometric mean, defined as the n^th root of the product of the comparisons. Subsequently, a numerical value was assigned to each criterion and its importance was quantified by the preference vector (N_i). This definition was cyclically iterated until the preference vector reached a stable state.

Considering the need to define the random consistency index in the AHP model for the selection of the best alternative proposed, it was established that the paired matrices must be consistent (CI < 0.1).

Step 6: In this step, the sub-criteria for each one of the defined criteria were established. For this, comparison matrices were generated based on the questionnaires applied in the previous steps. Then, the information from each of the comparison matrices for each criterion was consolidated in a matrix by quantifying the geometric mean.

Step 7: Based on the vector evaluated for each sub-criterion, the preference vector per alternative was generated following the procedure proposed in [²⁹], which considers the grouping of each sub-criterion according to the defined criteria.

Step 8: Considering the vector evaluated for each criterion, we weighted by the preference vector of each consolidated sub-criterion until the vector preference per alternative was defined.

Step 9: As a final step, the alternatives were ordered in ascending order, with 1 being the highest priority, according to the value C_k found in the previous step.

3. Results

From the literature review, two main assessment models were identified: i) Core model, developed by the European Network for Health Technology Assessment (EUnetHTA), ii) mini-HTA, developed by the former Danish Center for Health Technology Assessment, and the model developed by the project Adopting Hospital Based Health Technology Assessment (AdHopHTA), in the European Union. Fig. 2 shows the criteria frequently used in the assessment of medical devices, highlighting those used in the model. This result shows that internationally, multidisciplinary assessments which consider factors other than purely economic aspects are conducted and include factors such as: technical, safety, clinical effectiveness, organizational, social, ethical and legal.

Source: The Authors.

Figure 2 Criteria reported in the literature. 

In developing the methodology of Fig. 1, the following results were obtained:

Based on the levels of information and expertise, five professionals who participated in the process of technology procurement within the healthcare institution were selected to jointly analyze the opinion of all those involved. The profiles are described in Table 1.

Table 1 Profile of the decision-making group 

Source: The Authors.

Based on the literature review, the Core model, Mini-HTA model, the needs of the health institution and recommendations by the panel of experts, five criteria and eleven sub-criteria were defined. For this, it was verified that each of the sub-criteria were complete, not redundant and independent. A classification and description of this information is presented in Table 2. It is important to highlight that the after-sales service sub-criterion is highly relevant to the healthcare institution, defined as aspects related to training, repairs and requested maintenance.

Table 2 Decision criteria and sub-criteria 

Source: The Authors.

Initially, the identification of technological alternatives considering the Emergency Care Research Institute (ECRI) database resulted in 43 blood gas analyzer POCT devices.

However, a refinement of these options considering the five criteria allowed the assessment of five potential medical equipment alternatives. The safety criteria assessment was performed according to the historical analysis of adverse events presented by the national health agency, FDA, and the structured interview made to the brand representatives in Colombia. The evaluation of the economic criterion was based on the sale price information reported by ECRI and the cost of supplies and technical personnel. The evaluation of the organizational criteria was based on the information declared by the manufacturer, in terms of training and infrastructure needed to operate the equipment. The clinical effectiveness and technique criteria were evaluated based on technical information given by the service manuals of each equipment and by the brand representative in Colombia and also an expert in clinical laboratory test was consulted.

Results of the hierarchical level structuring and the device alternatives are presented in the tree diagram, see Fig. 3.

Source: The Authors.

Figure 3 Decisional tree. Results of the hierarchical level structuring and the alternatives 

For the classification of the information by the experts, a questionnaire was elaborated for the comparison of criteria by pairs. A consolidation of the expert´s answers is shown in Table 3, the preference vector of the criteria, Table 4, and the consistency index, of 1,120, calculated based on [²⁰]. Considering that these results depend upon the institutional orientations and the expertise of the evaluators, a regular update of Fig. 4 is recommended.

Table 3 Consolidated criteria matrix 

Source: The Authors.

Source: The Authors.

Figure 4 Criterial Weights 

The experts evaluated the sub-criteria by pairs, the results were consolidated in four matrices using the geometric mean function as an indicator. As in the previous step, the vector preference for each of the sub-criteria was found, i.e., information was processed to obtain the weights of each of the sub-criteria with respect to the corresponding criteria and the consistency index. This process was carried out from the previously obtained consolidated matrices and the values recorded in Table 4 were achieved.

Table 4.Preference vector 

Source: The Authors.

According to the results, the criteria related to clinical effectiveness, safety and technical aspects are the most relevant. It is important to note that the criteria described above are related to patient safety, while the other criteria are related to administrative and economic aspects.

Each alternative was evaluated with respect to each sub-criterion based on information provided by commercial advisors from the distribution companies, the user manual and the expert's criterion, translated into the tool with the Saaty scale. On this occasion, a series of questions were asked regarding each sub-criterion.

The corresponding values for each criterion were multiplied by the auto vector calculated in step 5 to obtain the vector preference for the alternative and a partial qualification of each alternative with respect to each criterion.

With the result of the preference vector (step 8), the selected alternatives were prioritized, the results are presented in Table 5.

Table 5 Results of the assessment 

Source: The Authors.

Based on the results, the option prioritized was Alternative 2, a piece of equipment that was characterized by different aspects, such as: after-sales service, ease of integration with the information system of the health institution, among others.

4. Conclusions

The proposed model, presented a systematic approach for the multi-criteria evaluation of alternatives for POCT blood gas analysis equipment in a healthcare institution using the hierarchical analysis process. The general problem was summarized by criteria and sub-criteria which were evaluated based on technical information, information from health agencies and the information declared by the brand representatives in Colombia. which allowed the identification of the best alternative via a mathematical tool, thereby helping to reduce subjectivity and uncertainty from the decision-making process.

Multi-criteria tools allow us to recognize and work within the framework of the existing organizational culture, for example, in the pilot healthcare institution, the staff are aligned with the organizational strategy, regardless of the area in which they work.

When comparing the existing process and the one proposed here, it was observed that there is an 88% reduction in the questions asked to the equipment supplier, which allows the process to be completed in less time and decisions can be made in an efficient way, based on information collected clearly and consistently, resulting in reliable data. The bibliographical references do not consider the after-sales service due to the culture of the countries where they were developed. In Colombia, this criterion is necessary considering that the need for training, maintenance, and short response times may be affected in cities where there are no brand representatives.

For the institution, the most relevant criteria were those related to the patient and their safety, while administrative aspects are considered secondary.