Introduction

As we could see in the previous articles of the economic evaluation series,^1-4 there is a large set of information required for decision makers on health expenditures and outcomes, and the way they spread over time. In order to obtain this information, epidemiological, economic, mathematical and statistical methods are used, but they all have limitations which are inherent to any scientific method. During the development process of economic evaluation, some uncertainties may arise, which can substantially impact the main findings of the analysis.⁵ When uncertainty is considered, researchers try to quantify the influence of data and adopted assumptions on the conclusion of the research. This article outlines three types of uncertainty in economic evaluation: methodological, structural and parameter.

Methodological uncertainty

The methodological uncertainty arises when there are different perceptions about how the ideal model for economic evaluation should be.⁶ The study developer chooses a set of methodological decisions,^1-3 such as analysis perspective, length of time horizon, discount rate, type of health outcome and method for valuing costs. These choices influence the result and, consequently, the decision-making. For instance, not all interventions impact on patient’s survival, such as those related to hearing loss and erectile dysfunction, although they substantially influence their quality of life. Thus, if the researcher chooses clinical outcomes (life years gained) instead of utility (quality of life), he or she would probably rule out the positive effects of interventions, affecting decision-making.

The way to deal with this kind of uncertainty is to adopt national good practice guidelines on how to conduct economic evaluation studies; or even, in the absence of such guidelines, find support in international recommendations. By following regulatory guidelines, the ability of the developer to influence the results of the analysis is reduced. In addition, comparability of results between different analyzes increases.

Structural uncertainty

It happens when the available evidence on the natural history of the disease and the impact of the strategies under investigation are non-existent, limited or contradictory.⁷ In the absence of good quality evidence, the analytical model will be built⁴ inappropriately. In this case, among the most common errors, we can mention: (i) disregard of any relevant health state; (ii) choice of constant transition probabilities when they are variable; and (iii) use of fragile assumptions to extrapolate short-term results in long-term results.

This structural uncertainty can be overcome when considering at least two hypotheses in which one has a more favorable assumption to the intervention under investigation whilst the other portrays less favorable assumptions. There is also the possibility of introducing a random parameter to the analytical model, which would signal the probability of each assumption to be true.⁶

Parameter uncertainty

It is defined as the inability to employ true numerical values of the parameters used in the analytical model,⁸ such as transition probabilities, quality-adjusted life year and costs. Parameter uncertainty arises from several factors, mainly: (i) some parameters are unknown at the time of completion of the analysis, for example, the price of a medication not available in the health system; (ii) the main consequences of a given intervention are unknown at population level, since scientific evidence derived from samples or biased studies; and (iii) the reliability of the information available may be questionable.

Parameter uncertainty is examined by sensitivity analysis, which can be deterministic or probabilistic. In general, the difference between those two lies in the way of representing the variation of parameters. In the case of deterministic sensitivity analysis, a set of specific values that express the plausibility of variation of parameters is used. It works as a confidence interval in which there are lower and higher values than the measure of central tendency (mean or median, for example). In the case of probabilistic sensitivity analysis, we use random distributions instead of specific values, in the parameters variation.⁹

There is also uncertainty regarding the variability among individuals, whilst some extracts or social groups respond differently to the intervention or have distinct perceptions and values. For example, differences between youngsters and the elderly. One way to overcome this uncertainty is the subgroup analysis.⁷

Concluding remarks

Uncertainty is inherent to any economic evaluation. Thus, it is necessary to analyze its impact on the outcome of the study. Each type of uncertainty has its peculiarities, some of them summarized in Figure 1. In the case of methodological uncertainty, it must follow national and/or international guidelines. As for structural uncertainty, we recommend the use of alternative analytical models, if there is limited information on the natural history of the disease or the result of interventions over time. And as for the parameter uncertainty, we suggest the sensitivity analysis.

Figure 1 - Types of uncertainty in economic evaluation studies