Serviços Personalizados
Journal
Artigo
Indicadores
- Citado por SciELO
Links relacionados
- Similares em SciELO
Compartilhar
Epidemiologia e Serviços de Saúde
versão impressa ISSN 1679-4974versão On-line ISSN 2237-9622
Epidemiol. Serv. Saúde vol.29 no.1 Brasília 2020 Epub 07-Fev-2020
http://dx.doi.org/10.5123/s1679-49742020000100002
RESEARCH NOTE
Indicators proposed by the Brazilian Ministry of Health for monitoring and evaluating oral health actions in the National Health System: documentary research, 2000-2017*
1Secretaria Municipal de Saúde, Gerência de Atenção Primária, Goiânia, GO, Brasil
2Universidade Federal de Goiás, Faculdade de Odontologia, Goiânia, GO, Brasil
3Universidade Federal de Goiás, Instituto de Patologia Tropical e Saúde Pública, Goiânia, GO, Brasil
Objective:
to present the indicators for monitoring and evaluation of oral health actions in the Brazilian National Health System (SUS), proposed in the period 2000-2017.
Methods:
documental research conducted on the Ministry of Health website regarding government guidelines on oral health monitoring and evaluation systems; the indicators were classified according to the following categories: access to care; resolutive capacity and continuity; and availability of oral health services.
Results:
oral health indicators were identified in the following guidelines: ‘Health Services Performance Evaluation Methodology Project’, ‘SUS Qualification Evaluation Program’, ‘National Program for Improving Primary Care Access and Quality’, and ‘SUS Performance Index’; most of them refer to access to services and resolutive capacity and continuity of care.
Conclusion:
oral health indicators in the four government guidelines identified provide important input for health management, but new indicators are needed for effective monitoring and evaluation of oral health actions.
Keywords: Indicators of Health Services; Oral Health; Health Management; Government Document
Introduction
With effect from 2004, public policies have been put into place in Brazil with the aim of improving access to and qualification of the Brazilian National Health System (SUS). As such, the Ministry of Health has scaled up its guidance for health service managers on the process of planning, executing, evaluating and monitoring their actions. Different programs and projects have been established, along with their appropriate health indicators.1-3
With effect from the first decade of the 2000s, increasing importance has been placed on oral health within the Public Health scenario. Furthermore, the process of SUS decentralization has favored actions being planned according to local reality, requiring health service managers to have knowledge of the health situation and health system performance prior to making decisions.4
Within this context, the effectiveness of indicators provided to evaluate health system policies, actions and performance are being questioned. There is a scarcity of studies about them, including the data needed to calculate them and their source. A previous publication analyzed the evolution of oral health indicators present in the SUS Interfederative Pacts between 1998 and 2016.5
The objective of this study was to present the indicators proposed by the Ministry of Health for monitoring and evaluating SUS oral health action qualification between 2000 and 2017.
Methods
Documental research was conducted on governmental guidelines, i.e. projects, programs and indices available about SUS qualification monitoring and evaluation systems issued between 2000 and 2017 containing oral health indicators.
Initially the data were obtained using the Ministry of Health website search engine (http://portalsaude.saude.gov.br/), in August 2017 and October 2018, using the following descriptors, together, and their correlates: indicators; and oral health. Further searches were then performed on the above mentioned source based on the references found. When reading the references, we checked whether they cited other Ministry publications covering SUS qualification evaluation; a further search was then performed to identify whether such publications contained information about oral health indicators. If these publications were not found on the Ministry website, we searched for them using the Google search engine.
The guidelines found were analyzed with regard to the following aspects:
documents - title and year of publication; and
indicators - category, calculation method, collection source and purposes.
As some guidelines presented indicators classified in categories while others did not, we opted to reclassify and/or group together all the indicators in three general categories: access to care; service availability; resolutive capacity and continuity. The data collected on oral health indicators were classified according to these three categories and presented in table format.
This research was not submitted to the appraisal of an ethics committee as the documents analyzed were available through a public domain website address.
Results
Our searches identified 186 publications of the governmental ordinance, information and guideline type. The Ministry of Health guidelines comprising the SUS qualification monitoring and evaluation systems and which contained oral health indicators between 2000 and 2017 were: Health Services Performance Evaluation Methodology Project (PROADESS); SUS Qualification Evaluation Program; National Program for Improving Primary Care Access and Quality (PMAQ) - Cycles 1, 2 and 3; and SUS Performance Index (IDSUS).
The indicators relating to each of these guidelines can be found in Tables 1, 2 and 3.
Health Services Performance Evaluation Methodology Project (PROADESS)
PROADESS6 had four indicators (Table 1) for oral health service performance evaluation. Based on the categories proposed in this study, three of the indicators were classified in the access to care category and one in the service availability category.
Category | Indicators and sources | Guideline | Calculation formula | |
---|---|---|---|---|
Numerator | Denominator | |||
Access to care | Dental appointment Sources: PNAD and PNS | PROADESS | No. of people who had a dental appointment in the last 12 months x 100 | Total resident population |
Percentage of people who never had a dental appointment Sources: PNAD and PNS | PROADESS | No. of people reporting never having had a dental appointment x 100 | Total resident population | |
Percentage of the population reporting having used health services in the last 15 days, in a given geographical space. Sources: PNAD and PNS | PROADESS | No. of people seeking health services in the last 15 days and who received care x 100 | Total resident population | |
Coverage of first programmatic dental appointment a Sources: SIA/SUS and IBGE | SUS Qualification Evaluation Program | Total number of first programmatic dental appointments a attended in a given place and period x 100 | Population in the same place and period | |
Service availability | Number of dental surgeons per 100,000 inhabitants, in a given geographic space, in the year under consideration. Source: CNES | PROADESS | No. of dental surgeons x 100 mil | Total resident population |
Estimated population coverage of the Family Health Strategy Oral Health teams Sources: FCES and IBGE | SUS Qualification Evaluation Program | No. of Family Health Strategy Oral Health teams implanted (Modalities I and II) x 3,450 people | Population in the same place and period x 100 |
Sources: Ministry of Health; 2011a6; 2011b7.
a) ‘First dental appointment’ is considered to be when the preventive treatment plan is prepared. Urgent/emergency care is not taken into consideration.
Legend:
PNAD: National Household Sample Survey.
PNS: National Health Survey.
CNES: National Health Establishment Registry.
SIA/SUS: Brazilian National Health System Outpatient Information System.
IBGE: Brazilian Institute of Geography and Statistics.
FCES: Health Establishment Registry Form.
Brazilian National Health System (SUS) Qualification Evaluation Program
The SUS Qualification Evaluation Program7 is comprised of diverse indicators, divided into two dimensions: access and quality. In our study, these indicators were classified into two groups: access to care; and service availability (Table 1).
National Program for Improving Primary Care Access and Quality (PMAQ)
The PMAQ cycles occurred during the period analyzed: the 1st Cycle in 2011;8 the 2nd Cycle in 2013;9 and the 3rd Cycle in 2015.10 Different oral health indicators were identified. The 1st and 2nd Cycles used the same indicators11,12 (Table 2), the majority of which were classified into the access to care category.
Category | Indicators | Calculation formula (and sources consulted) | Parameter | |
---|---|---|---|---|
Numerator a | Denominator | |||
1st and 2nd Cycles | ||||
Access to care | Coverage of first programmatic dental appointment | No. of first programmatic dental appointments in a given place and period x 100 | Population registered in the same place and period Source: SIAB Form A | Not provided |
Coverage of first dental care appointment for pregnant women | No. of pregnant women having first appointment with Oral Health team dental surgeon in a given place and period x 100 | No. of pregnant women registered in the same place and period Source: SIAB SSA2 Report | Not provided | |
Average urgent dental care session per inhabitant | No. of urgent dental care sessions performed by the Oral Health team dental surgeon in a given place and period | Population registered in the same place and period Source: SIAB Form A | Not provided | |
Oral mucosa alteration incidence rate | No. of diagnoses of oral mucosa alteration in service users care for the by Oral Health team in a given place and period x 1,000 | Population registered in the same place and period Source: SIAB Form A | Not provided | |
Service availability | Average supervised collective dental brushing actions | No. of people taking part in the supervised collective dental brushing action performed in a given place and period x 100 | Population registered in the same place and period Source: SIAB Form A | Not provided |
Resolutive capacity and continuity | Average dental prosthesis fitted | No. of dental prostheses fitted by the Oral Health team in a given place and period | Population registered in the same place and period x 0.03 Source: SIAB Form A | Not provided |
Ratio between concluded treatments and first programmatic dental appointments | No. of treatments concluded by the Oral Health team dental surgeon in a given place and period | No. of first programmatic dental appointments carried out by the Oral Health team dental surgeon in the same place and period Source: PMA2 Report - SIAB Complement | Not provided | |
3rd Cycle | ||||
Access to care | Coverage of first programmatic dental appointments | No. of first programmatic dental appointments attended x 100 | Population registered | 15% first programmatic dental appointment attended/year 1.25% first programmatic dental appointment attended/month |
Service availability | Percentage of services made available by the Oral Health team | Quantity of actions and service performed by the Oral Health team x 100 | Total actions and services expected by the Oral Health team | 70%/month |
Resolutive capacity and continuity | Ratio between concluded treatments and first programmatic dental appointments | No. of treatments concluded by the dental surgeon | No. of first programmatic dental appointments | 0.5-1.0 treatment concluded/month |
Sources: Ministry of Health, 20171; Brazil, 2011c8; Brazil, 2013a9; Brazil, 201510; Ministry of Health, 201211; Ministry of Health, 2013b12.
a) Sources of 1st and 2nd Cycle numerators: PMA2 Report - SIAB Complement.
Legend:
SIAB: Primary Care Information System.
In the 3rd PMAQ Cycle,1 despite their reduced number - when compared to the preceding cycles -, different indicators were incorporated and these were classified into three categories: access to care; service availability; resolutive capacity and continuity (Table 2).
Performance Index (IDSUS)
With regard to oral health, IDSUS has indicators for access potential (measures service delivery availability) or access obtained (measures services delivered), and effectiveness (evaluates the result obtained).13 We classified these indicators into two categories: service availability; resolutive capacity and continuity (Table 3).
Category | Simple indicators and sources | Calculation formula | Parameter | |
---|---|---|---|---|
Numerator | Denominator | |||
Service availability | Population coverage estimated by the Oral Health teams Sources: CNES and IBGE | Average monthly number of Family Health Strategy Oral Health teams plus Average monthly number of qualified equivalents per each 60 hours/week of non-Oral Health team dental surgeons x 3,000 inhabitants | Population resident in the municipality | 50% coverage taking one Primary Care Oral Health team per each group of 3,000 inhabitants in the year assessed |
Average supervised collective dental brushing actions Sources: SIA/SUS and IBGE | No. of people taking part in the supervised collective dental brushing action in the 12 month period, in a municipality, divided by 12 x 100 | Population in the municipality in the period evaluated | 8 participants per 100 inhabitants - average of municipalities covered by Oral Health teams or THD/ACD teams greater than 60% | |
Resolutive capacity and continuity | Proportion of tooth extraction in relation to dental procedures Sources: SIA/SUS and IBGE | No. of teeth extractions performed in a municipality and year evaluated x 100 | Total individual preventive and curative procedures, selected per municipality and year evaluated. | 8% (percentage close to the average for the year 2010: 8.6%) of all Brazilian cities that performed dental procedures on the SUS |
Sources: Ministry of Health; 2013c13.
Legend:
ACD: Dental Consulting Room Auxiliary.
CNES: National Health Establishment Registry.
IBGE: Brazilian Institute of Geography and Statistics.
SIA/SUS: Brazilian National Health System Outpatient Information System.
THD: Dental Hygiene Technician.
Discussion
The results provide evidence of diverse indicators for SUS health care qualification. The majority of the oral health indicators fell into the access to care category. Over the years analyzed few indicators of service resolutive capacity and continuity were proposed, and in programs that had more than one cycle some indicators were excluded.
A literature review14 identified studies on evaluation of dental care in health services in Brazil published as at June 2010. The majority of the 23 studies published between 2002 and 2010 evaluated Primary Care, while care quality and access to services were focused on by three and four of those studies, respectively.14
Access indicators demonstrate ease or difficulty in getting health care.15 Our study identified different indicators in this category. A literature review of oral health indicators used in Brazil for the period 2000 to 2012,16 identified that the indicator of access to dental appointments was one of the most frequent.
The service availability category corresponds to resources available to meet service users’ demands.17 The most frequent indicator in this category was the estimated population coverage of Family Health Strategy Oral Health teams. The Brazilian National Health System (SUS) Qualification Evaluation Program, proposed in 2011 to evaluate SUS performance,8 included this indicator.
The resolutive capacity and continuity category refers to the capacity to identify health risks, needs and demands until the problem is solved.1 One of the indicators identified in this category was the proportion of tooth extraction among procedures performed. Tooth extraction enables evaluation of the extent to which dental practices involve mutilation.16 Another indicator identified in this category was the ratio between concluded treatments and first programmatic dental appointments, representing adherence to treatment and treatment conclusion.
Set up in 2001, the mission of PROADESS6 is to evaluate health service performance based on the principle of equity, using a matrix categorizing indicators into four dimensions: health determinants; health conditions; health system structure; and health service performance.
The principle of equity has been evaluated according to two main dimensions: health conditions; and health service access and use.18 PROADESS classified oral health indicators according to the latter dimension. With regard to the implementation of equitable health policies, three of their aspects stand out: resource distribution; access availability or timeliness; and service use.19
Models proposed for health service evaluation have matrices that include dimensions, subdimensions, indicators and, in some cases, calculation formulae and indicator parameters.20 In our study we identified guidelines using different models and indicators, some of which had parameters. PMAQ came into being in 20118 and proposes evaluation and monitoring of the processes and results achieved in Primary Care, in order to ensure access and qualify health care offered to the population.9 It is noteworthy that the 3rd PMAQ cycle incorporated indicator parameters, as well as an indicator of the existence of a minimum number of oral health services.1 The incorporation of these elements can contribute to oral health team work process management and organization.
IDSUS was introduced in 2011 with the aim of ‘evaluating SUS performance with regard to compliance with its principles and guidelines’.2 However, oral health indicators were only proposed in the service availability and resolutive capacity and continuity categories and these are insufficient to achieve this evaluation.
Indicator parameters were introduced in the last PMAQ cycle and in IDSUS. However, few studies on dental care evaluation have specified the criteria, indicators or parameters used. Where parameters have been presented, they were based on local service targets, or Health Ministry performance targets, or even established by the authors themselves.20 Different oral health evaluation models are found in the literature, approaching diverse indicator dimensions, names and calculation formulae.20-26 A methodological proposal for service action and performance evaluation and monitoring must contain elements that favor analysis of compliance with SUS principles, so that its managers strengthen and qualify the system. In the field of Oral Health, however, the indicators cover few dimensions.
We conclude that oral health indicators were proposed in four governmental guidelines in the period from 2000 to 2017. Although they provide important support for Oral Health management, new indicators need to be incorporated, capable of broadening the scope of the evaluation of the quality of Public Health System service delivery and performance.
REFERENCES
1. Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Manual instrutivo para as equipes de atenção básica e NASF: Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ), Terceiro ciclo 2015-2017 [Internet]. Brasília: Ministério da Saúde; 2017 [citado 2017 out 21]. 78 p. Disponível em: Disponível em: http://189.28.128.100/dab/docs/portaldab/documentos/Manual_Instrutivo_3_Ciclo_PMAQ.pdf [ Links ]
2. Ministério da Saúde (BR). IDSUS - Indicadores de desempenho do sistema único de saúde [Internet]. Brasília: Ministério da Saúde ; 2014 [citado 2018 jul 30]. 50 p. Disponível em: Disponível em: http://idsus.saude.gov.br/documentos.html [ Links ]
3. Oliveira PTR, Sellera PEG, Reis AT. Monitoramento e a avaliação na gestão do Ministério da Saúde. Rev Bras Monit Aval [Internet]. 2013 jan-ju [citado 2018 jul 30];(5):114-29. Disponível em: Disponível em: http://idsus.saude.gov.br/documentos/Artigo_M%26A_RBMA_5.pdf [ Links ]
4. Lessa CFM, Vettore MV. Gestão da atenção básica em saúde bucal no Município de Fortaleza, Ceará, entre 1999 e 2006. Saúde Soc [Internet]. 2010 set [citado 2018 jul 30];19(3):547-56. Disponível em: Disponível em: http://www.scielo.br/pdf/sausoc/v19n3/07.pdf . doi: 10.1590/S0104-12902010000300007 [ Links ]
5. França MASA, Freire MCM, Pereira ED,Marcelo VC. Oral health indicators in the Interfederative Pacts of the Unified Health System: development in the 1998-2016 period. Rev Odontol UNESP Soc [Internet]. 2018 Jan [cited 2018 Dec 30];47(1):18-24. Available from: Available from: http://www.scielo.br/pdf/rounesp/v47n1/1807-2577-rounesp-1807-257708417.pdf . doi: 10.1590/1807-2577.08417 [ Links ]
6. Ministério da Saúde (BR). Fundação Instituto Oswaldo Cruz. Laboratório de Informações em Saúde. Instituto de Comunicação e Informação em Ciência e tecnologia. PROADESS - Avaliação de desempenho do sistema de saúde brasileiro: indicadores para monitoramento. Relatório final [Internet]. Rio de Janeiro: Fiocruz; 2011 [citado 2018 jan 21]. 287 p. Disponível em: Disponível em: http://www.proadess.icict.fiocruz.br/SGDP-RELATORIO_FINAL%20_com_sumario_atualizadorev%202014.pdf [ Links ]
7. Ministério da Saúde (BR). Secretaria Executiva. Departamento de Monitoramento e Avaliação do SUS. Coordenação-Geral de Monitoramento e Avaliação. Programa de avaliação para a qualificação do Sistema Único de Saúde [Internet]. Brasília: Ministério da Saúde ; 2011 [citado 2018 jan 21]. 26 p. Disponível em: Disponível em: http://www.cebes.org.br/media/File/Programa_Avaliacao_Qualificacao_06-04-2011.pdf [ Links ]
8. Brasil. Ministério da Saúde. Portaria MS/GM nº 1.654, de 20 de julho de 2011. Institui no âmbito do Sistema Único de Saúde, o Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ) e o Incentivo Financeiro do PMAQ-AB, denominado Componente de Qualidade do Piso de Atenção Básica Variável - PAB Variável [Internet]. Diário Oficial da União, Brasília (DF), 2011 jul 20 [citado 2018 jul 30];Seção 1:79. Disponível em: Disponível em: http://bvsms.saude.gov.br/bvs/ saudelegis /gm / 2011/prt1654_19_07_2011.html [ Links ]
9. Brasil. Ministério da Saúde. Portaria MS/GM nº 635, de 17 de abril de 2013. Homologa a adesão dos municípios ao segundo ciclo do Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ) [Internet]. Diário Oficial da União, Brasília (DF), 2013 abril 18 [citado 2018 jan 19];Seção 1:39. Disponível em: Disponível em: http://189.28.128.100/dab/docs/portaldab/documentos/pmaq/prt_635_17_04_2013.pdf [ Links ]
10. Brasil. Ministério da Saúde. Portaria MS/GM nº 1.645, de 2 de outubro de 2015. Dispõe sobre o Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB) [Internet]. Diário Oficial da União, Brasília (DF), 2015 out 10 [citado 2018 jan 19];Seção 1;668. Disponível em: Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2015/prt1645_01_10_2015.html [ Links ]
11. Ministério da Saúde (BR). Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): manual instrutivo - Anexo ficha de qualificação dos indicadores [Internet]. Brasília: Ministério da Saúde ; 2012 [citado 2018 jan 19]. 80 p. Disponível em: Disponível em: http://sistemas.fesfsus.ba.gov.br/guiatrabalhador/TrabESF/artigos/pmaq_manual_instrutivo_anexo.pdf [ Links ]
12. Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Saúde mais perto de você - acesso e qualidade. Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): manual instrutivo [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2017 ago 30]. 38 p. Disponível em: Disponível em: http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_instrutivo_PMAQ_AB2013.pdf [ Links ]
13. Ministério da Saúde (BR). Secretaria-Executiva. Departamento de Monitoramento e Avaliação do SUS. Coordenação-Geral de Monitoramento e Avaliação do SUS. Índice de desempenho do Sistema Único de Saúde (IDSUS). Fichas técnicas dos indicadores [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2018 jan 21]. 16 p. Disponível em: Disponível em: http://idsus.saude.gov.br/assets/simplificadas.pdf [ Links ]
14. Colussi CF, Calvo MCM. Avaliação da atenção em saúde bucal no Brasil: uma revisão da literatura. Saúde Transf Soc [Internet]. 2012 [citado 2019 mar 19];3(1):92-100. Disponível em: Disponível em: http://pepsic.bvsalud.org/pdf/sts/v3n1/v3n1a15.pdf [ Links ]
15. Travassos C, Castro MSM. Determinantes e desigualdades sociais no acesso e na utilização de serviços de saúde. In: Giovanella L, Escorel S, Lobato LVC, Noronha JC, Carvalho AI, organizadores. Políticas e Sistema de Saúde. Rio de Janeiro: Editora Fiocruz; 2008. p. 215-24. [ Links ]
16. França MASA. Indicadores utilizados em saúde bucal no Brasil [dissertação]. Goiânia (GO): Universidade Federal de Goiás; 2013. Disponível em: https://repositorio.bc.ufg.br/tede/handle/tede/4232 [ Links ]
17. Bordin D, Fadel CB. Pacto pela saúde no Brasil: uma análise descritiva da progressão dos indicadores de saúde bucal. Rev Odontol UNESP [Internet]. 2012 set-out [citado 2019 abr 11];41(5):305-11. Disponível em: Disponível em: http://www.scielo.br/pdf/rounesp/v41n5/a02v41n5.pdf . doi: 10.1590/S1807-25772012000500002 [ Links ]
18. Escorel S. Equidade em saúde. In: Pereira IB, Lima JCF. Dicionário da educação profissional em saúde [Internet]. 2. ed. rev. ampl. Rio de Janeiro: EPSJV; 2008 [citado 2019 mar 20]. p. 202-10. Disponível em: Disponível em: http://www.epsjv.fiocruz.br/sites/default/files/l43.pdf [ Links ]
19. Viana ALA, Fausto MCR, Lima LD. Política de saúde e equidade. São Paulo Perspect [Internet]. 2003 jan-mar [citado 2019 abr 11];17(1):58-68. Disponível em: Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-88392003000100007 . doi: 10.1590/S0102-88392003000100007 [ Links ]
20. Medina MG, Silva GAP, Zulmira RA, Hartz MA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV, organizadores. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador: EDUFBA; Rio de Janeiro: Editora Fiocruz ; 2005. p. 41-63. [ Links ]
21. Colussi CF, Calvo MCM. Modelo de avaliação da saúde bucal na atenção básica. Cad Saúde Pública [Internet]. 2011 set [citado 2019 mar 19];27(9):1731-45. Disponível em: Disponível em: http://www.scielo.br/pdf/csp/v27n9/07.pdf . doi: 10.1590/S0102-311X2011000900007 [ Links ]
22. Bueno VLRC, Cordoni Júnior L, Mesas AE. Desenvolvimento de indicadores para avaliação de serviço público de odontologia. Ciênc Saúde Coletiva [Internet]. 2011 jul [citado 2019 mar 19];16(7):3069-82. Disponível em: Disponível em: http://www.scielo.br/pdf/csc/v16n7/07.pdf . doi: 10.1590/S1413-81232011000800007 [ Links ]
23. Grimm SCA, Tanaka OY. Painel de monitoramento municipal: bases para a construção de um instrumento de gestão dos serviços de saúde. Epidemiol Serv Saúde [internet]. 2016 jul-set [citado 2019 mar 19];25(3):585-94. Disponível em: Disponível em: http://www.scielo.br/pdf/ress/v25n3/2237-9622-ress-25-03-00585.pdf . doi: 10.5123/s1679-49742016000300014 [ Links ]
24. Machado FCA, Souza GCA, Noro LRA. Proposição de indicadores para vigilância da saúde bucal de adolescentes. Ciênc Saúde Coletiva [Internet]. 2018 jan [citado 2019 mar 19];23(1):187-202. Disponível em: Disponível em: http://www.scielo.br/pdf/csc/v23n1/1413-8123-csc-23-01-0187.pdf . doi: 10.1590/1413-81232018231.20842015 [ Links ]
25. Martins LGT. Desenvolvimento de um modelo de avaliação da atenção à saúde bucal da criança [dissertação]. Tubarão (SC): Universidade do Sul de Santa Catarina; 2014. Disponível em: https://www.riuni.unisul.br/handle/12345/3600 [ Links ]
26. Pires DA, Colussi CF, Calvo MCM. Avaliação da gestão municipal da saúde bucal na Atenção Básica: precisão do instrumento de pesquisa. Ciênc Saúde Coletiva [Internet]. 2014 nov [citado 2019 mar 19];19(11):4525-34. Disponível em: Disponível em: http://www.scielo.br/pdf/csc/v19n11/1413-8123-csc-19-11-4525.pdf . doi: 10.1590/1413-812320141911.15412013 [ Links ]
Received: December 11, 2018; Accepted: August 28, 2019