Introduction

With effect from 2004, public policies have been put into place in Brazil with the aim of improving access to and qualification of the Brazilian National Health System (SUS). As such, the Ministry of Health has scaled up its guidance for health service managers on the process of planning, executing, evaluating and monitoring their actions. Different programs and projects have been established, along with their appropriate health indicators.^1-3

With effect from the first decade of the 2000s, increasing importance has been placed on oral health within the Public Health scenario. Furthermore, the process of SUS decentralization has favored actions being planned according to local reality, requiring health service managers to have knowledge of the health situation and health system performance prior to making decisions.⁴

Within this context, the effectiveness of indicators provided to evaluate health system policies, actions and performance are being questioned. There is a scarcity of studies about them, including the data needed to calculate them and their source. A previous publication analyzed the evolution of oral health indicators present in the SUS Interfederative Pacts between 1998 and 2016.⁵

The objective of this study was to present the indicators proposed by the Ministry of Health for monitoring and evaluating SUS oral health action qualification between 2000 and 2017.

Methods

Documental research was conducted on governmental guidelines, i.e. projects, programs and indices available about SUS qualification monitoring and evaluation systems issued between 2000 and 2017 containing oral health indicators.

Initially the data were obtained using the Ministry of Health website search engine (http://portalsaude.saude.gov.br/), in August 2017 and October 2018, using the following descriptors, together, and their correlates: indicators; and oral health. Further searches were then performed on the above mentioned source based on the references found. When reading the references, we checked whether they cited other Ministry publications covering SUS qualification evaluation; a further search was then performed to identify whether such publications contained information about oral health indicators. If these publications were not found on the Ministry website, we searched for them using the Google search engine.

The guidelines found were analyzed with regard to the following aspects:

As some guidelines presented indicators classified in categories while others did not, we opted to reclassify and/or group together all the indicators in three general categories: access to care; service availability; resolutive capacity and continuity. The data collected on oral health indicators were classified according to these three categories and presented in table format.

This research was not submitted to the appraisal of an ethics committee as the documents analyzed were available through a public domain website address.

Results

Our searches identified 186 publications of the governmental ordinance, information and guideline type. The Ministry of Health guidelines comprising the SUS qualification monitoring and evaluation systems and which contained oral health indicators between 2000 and 2017 were: Health Services Performance Evaluation Methodology Project (PROADESS); SUS Qualification Evaluation Program; National Program for Improving Primary Care Access and Quality (PMAQ) - Cycles 1, 2 and 3; and SUS Performance Index (IDSUS).

The indicators relating to each of these guidelines can be found in Tables 1, 2 and 3.

Discussion

The results provide evidence of diverse indicators for SUS health care qualification. The majority of the oral health indicators fell into the access to care category. Over the years analyzed few indicators of service resolutive capacity and continuity were proposed, and in programs that had more than one cycle some indicators were excluded.

A literature review¹⁴ identified studies on evaluation of dental care in health services in Brazil published as at June 2010. The majority of the 23 studies published between 2002 and 2010 evaluated Primary Care, while care quality and access to services were focused on by three and four of those studies, respectively.¹⁴

Access indicators demonstrate ease or difficulty in getting health care.¹⁵ Our study identified different indicators in this category. A literature review of oral health indicators used in Brazil for the period 2000 to 2012,¹⁶ identified that the indicator of access to dental appointments was one of the most frequent.

The service availability category corresponds to resources available to meet service users’ demands.¹⁷ The most frequent indicator in this category was the estimated population coverage of Family Health Strategy Oral Health teams. The Brazilian National Health System (SUS) Qualification Evaluation Program, proposed in 2011 to evaluate SUS performance,⁸ included this indicator.

The resolutive capacity and continuity category refers to the capacity to identify health risks, needs and demands until the problem is solved.¹ One of the indicators identified in this category was the proportion of tooth extraction among procedures performed. Tooth extraction enables evaluation of the extent to which dental practices involve mutilation.¹⁶ Another indicator identified in this category was the ratio between concluded treatments and first programmatic dental appointments, representing adherence to treatment and treatment conclusion.

Set up in 2001, the mission of PROADESS⁶ is to evaluate health service performance based on the principle of equity, using a matrix categorizing indicators into four dimensions: health determinants; health conditions; health system structure; and health service performance.

The principle of equity has been evaluated according to two main dimensions: health conditions; and health service access and use.¹⁸ PROADESS classified oral health indicators according to the latter dimension. With regard to the implementation of equitable health policies, three of their aspects stand out: resource distribution; access availability or timeliness; and service use.¹⁹

Models proposed for health service evaluation have matrices that include dimensions, subdimensions, indicators and, in some cases, calculation formulae and indicator parameters.²⁰ In our study we identified guidelines using different models and indicators, some of which had parameters. PMAQ came into being in 2011⁸ and proposes evaluation and monitoring of the processes and results achieved in Primary Care, in order to ensure access and qualify health care offered to the population.⁹ It is noteworthy that the 3^rd PMAQ cycle incorporated indicator parameters, as well as an indicator of the existence of a minimum number of oral health services.¹ The incorporation of these elements can contribute to oral health team work process management and organization.

IDSUS was introduced in 2011 with the aim of ‘evaluating SUS performance with regard to compliance with its principles and guidelines’.² However, oral health indicators were only proposed in the service availability and resolutive capacity and continuity categories and these are insufficient to achieve this evaluation.

Indicator parameters were introduced in the last PMAQ cycle and in IDSUS. However, few studies on dental care evaluation have specified the criteria, indicators or parameters used. Where parameters have been presented, they were based on local service targets, or Health Ministry performance targets, or even established by the authors themselves.²⁰ Different oral health evaluation models are found in the literature, approaching diverse indicator dimensions, names and calculation formulae.^20-26 A methodological proposal for service action and performance evaluation and monitoring must contain elements that favor analysis of compliance with SUS principles, so that its managers strengthen and qualify the system. In the field of Oral Health, however, the indicators cover few dimensions.

We conclude that oral health indicators were proposed in four governmental guidelines in the period from 2000 to 2017. Although they provide important support for Oral Health management, new indicators need to be incorporated, capable of broadening the scope of the evaluation of the quality of Public Health System service delivery and performance.