Introduction

The Mortality Information System (SIM), created in the 1970s, is frequently evaluated by means of quality attributes, with emphasis on reliability, completeness and coverage.^1,2 These investigations seek to provide evidence of common obstacles, especially in the less developed regions of Brazil.^3-5 However, highlighting specific dimensions of the system limits knowledge on the production of vital data and its level of deployment, with repercussions for information accuracy and use.⁶

The need for evaluations targeting the generation of information on SIM has been evidenced by studies aimed at identifying difficulties with its operationalization in the institutional environment.^7,8 As a result, evaluations that consider the structure and process required to obtain data in the context of health services contribute positively to the use of theoretical and methodological principles of health evaluation in the improvement of information systems that undertake continuous recording.^6,9,10

Evaluation of vital information system processes has enabled explanatory factors regardingthe results achieved to be specified. The potential of these analyses was expanded once they began to involve normative aspects as one of their steps.^6,9 This methodological procedure has allowed the degree of structure and process adequacy to be determined, based on criteria and standards, as well as to identify whether expected results correspond to those observed in the context of services, in addition to making interventions and their way of operating more understandable.¹¹

Although evaluative experiences have incorporated aspects of the operationalization of health information systems in the organizational environment,^6,9 empirical gaps restrict knowledge as to the entire process of SIM data production, its specific problems and deployment implications, with regard to the results achieved. Analyzing the system from the perspective of its deployment may overcome the limitations of evaluations commonly carried out, the focus of which is restricted to results achieved, in addition to guiding actions aimed at improving structural and procedural aspects, with the possibility of having impact on the quality of the system.^6,9

This study aimed to evaluate the implantation of SIM in Pernambuco, Brazil, in 2012.

Methods

This was an evaluation study based on implantation analysis, assessing the influence of SIM deployment variation on the results found.¹² The strategy we adopted was a single case study¹³ in Pernambuco, both in relation to the state as a whole and also its health regions.

SIM is in operation in the Health Departments of the 185 municipalities of Pernambuco. These are distributed over 12 political-administrative regions corresponding to the state’s health regions. The municipal health departments are responsible for collecting, processing, transmitting and disseminating information on mortality, which is produced based on 'Death Certificates’ issued. The flow of information is hierarchical: data is sent by municipal SIM managers to the State Health Department and to the Ministry of Health.¹

In this study, we included 184 of the state’s municipalities and excluded only one, as it does not have technicians and managers to implement and manage SIM routines. Our evaluation was carried out in four stages:

Stage 1 - Updating the SIM logic model

In order to be able to detail the intervention assessed, the previously prepared¹⁴ SIM logic model was updated using official system documents in a normative approach that included Ministerial Ordinances No. 116/2009, No. 17082013 and No. 47/2016, Death Certificate procedure and completion manuals, and the planning of actions undertaken in the different spheres of SIM system management. Together these components covered the structure dimension (human and physical resources) and the process dimension (activities performed), divided into six components involved in the generation of information: management; distribution and control; issuing and filling in; collection; processing; and analysis and dissemination (Figure 1).

a) CID-10 International Statistical Classification of Diseases and Related Health Problems- 10^th edition.

Figure 1 - Summary of the Mortality Information System logic model 

Stage 2 - Preparation of the matrix of indicators and judgment criteria, and data collection

The indicator matrix and judgment criteria were prepared based on the SIM system logic model. The indicators were selected taking into consideration content validity, its relevance, availability, ease of retrieval, simplicity of calculation and timeliness. Structure and process indicators were established for each component of the logic model expressing the degree of implantation and results. Parameters derived from norms were defined for the indicators of structure and process, when such norms were available in institutional documents of the Ministry of Health and/or the Pernambuco State Health Department. If the norm did not contain a clear definition, the criteria were empirically derived according to the researchers’ judgment and stipulated in accordance with the service routine.

The variables and indicators are shown in (Figure 2), a matrix of 76 indicators distributed according to components: 23 for structure, 33 for process and 20 for the results, with their respective judgment criteria. Based on the indicators selected, we created the data collection instrument according to the SIM system components that were to be evaluated.

a) Source of empirical verification.

b) Source of normative verification.

Figure 2 - Matrix of Mortality Information System (SIM) indicators and judgment criteria, Pernambuco, 2012 

The primary data were collected in November and December 2013 - relating to information for the year 2012 - by administering a structured questionnaire with those responsible for SIM in the municipalities studied, in addition to direct observation of structural and procedural aspects in all the municipal health departments included in the study. The secondary data, relating to deaths occurring in 2012, were retrieved from the Pernambuco state SIM database. The information obtained through the primary and secondary data were consolidated by health region and for the state as a whole, and included in the indicators that informed the evaluation of the degree of SIM implantation and the influence of this on results.

Results

SIM was found to be partially implanted in Pernambuco (70.6%), varying between 66.3 and 74.8% according to health region. Similarly, its 'management' (75.1%), 'issuing and filling in' (79.1%) and 'processing' (71.7%) components were found to be partially implanted. The 'collection' component was 'implanted' in the state (80.7%) and in most regions, with the exception of five, three of which were 'partially implanted' and two had 'incipient' implantation. Implantation of the 'distribution and control' component was 'incipient' in the state as a whole (49.7%) and 'not implanted’ in three regions, while the 'analysis and dissemination' component was 'incipient' in the state as a whole (58.0%) and partially implanted in five of its regions (Table 1).

The 'management' component showed that 69.6% of the state’s municipalities monitored at least 50.0% of the operational indicators; and that 62.0% carried out continuing education, varying between 35.0 and 100.0% between regions (Table 2). The 'distribution and control' component showed that 90.1% of Death Certificates issued and registered were input to SIM, this result being greater than or equal to 79.0% between the health regions. The 'issuing and filling in' component in relation to the state as a whole had 94.9% of deaths with a defined underlying cause, with similar values for all but three health regions. Consolidated Death Certificate completeness for general deaths for the state as a whole was 51.7%; while block IV relating to infant deaths was 66.8% complete, with higher results in three regions. The 'collection' component showed 100% coverage in the state as a whole and also in its regions; the proportion of municipalities with coverage above 90% for this component was 82.1%.

The 'processing' component showed 94.3% of municipalities with monthly data transfer and 90.1% of Death Certificated input and forwarded on a timely basis, with similar results between the regions. The indicators for the notification of deaths of women of childbearing age, maternal, fetal and infant deaths within 30 days after death remained below 50% in the state as a whole. Four regions did not notify maternal death in a timely manner. The 'analysis and dissemination' indicators showed a proportion of less than 40%, in both the state and in most regions (Table 2).

Discussion

SIM was found to be partially implanted in Pernambuco, with variations between the components of the system according to health regions, demonstrating consistency with the results found: the greater the degree of implantation, the better the results. Incomplete achievement of SIM’s objectives arises from inadequacies in the distribution, control, analysis and dissemination of data, although the system has been established for more than 30 years and despite recognition of its relevance for the analysis of the health situation and its monitoring and evaluation.^1-3

The fact of state-level management having updated the SIM logic model and prepared the matrix of indicators, without the participation of the federal level of the system, may have bestowed a transitory nature on the judgment criteria so that they may require periodic revisions. In spite of the impossibility of the study results being extrapolated, the internal validity of the logic model of the system is explicit owing to a theoretical construct that shows the relations of interdependence between the historically constructed components and their contents, so that it can be replicated in other contexts, with adjustments.^12,13

The 'distribution and control' and 'analysis and dissemination' components showed reduced adherence of procedural indicators to established standards, pointing to a not very satisfactory work process. This finding, while being similar to that found for the Brazilian Live Birth Information System (SINASC) in the municipalities of Minas Gerais,⁹ is opposite to that found for SISNASC in Pernambuco, where structural problems prevailed.⁶

Performing analysis by component identified shortcomings, such as municipalities with low control of Death Certificates distributed, used and canceled, which can favor misuse, loss and flow deviation. This insufficient control over the Certificate forms not only increases the cost of production and meeting the needs of the system, but also reveals reduced management capacity. In the same way, studies of active tracing indicate problems with the operationalization of vital information systems, such as inadequate flows and use of non-official forms, reflecting little normative knowledge, signaling the need to enhance the capacity of professionals responsible for the operationalization of these systems.^4,6,7,15

Even though inter and intra-regional differences were revealed in SIM coverage in the municipalities evaluated, high system data capture points to an intensive data collection work process. This finding corroborates the evidence about the expansion of SIM coverage in the state of Pernambuco, with repercussions on the reliability of using its data to calculate mortality indicators,^16-18 thus allowing adequate monitoring of national and international pacts.¹⁹

The better results found for indicators of regular batch transfer and data sending, which are linked to funding being released, to the detriment of timely notification of deaths of women of childbearing age, and maternal, fetal and infant deaths, express the preponderance of financial rationality over the epidemiological situation with regard to SIM processing. These findings are similar to those of a previous study, focused on small-sized municipalities of the state of Rio Grande do Sul.²⁰ Delay in knowledge of reportable events hinders surveillance actions, as well as the reorientation of care and health services to meet the needs of the population,^21,22 frequently limiting the scope of municipal actions, inadequately directed towards collection and transmission of data to other health management levels.²³

Death Certificate incompleteness, scant dissemination of information and reduced continuing education found in Pernambuco’s health regions, resulting from the overvaluation of data collection and transmission routines, are related to the lack of teams and lack of technical qualification, thus hindering the consolidation of the system.^20,23 As expected, the better filling in of Death Certificates, found in more developed areas of the metropolitan region of Recife (the state capital) and its surrounding territories as well as in the state’s larger municipalities, contrasted with Death Certificate completeness found in regions and localities with lower access to health services, lower access to health professional capacity building and lower access to health surveillance services recovering data variables.^24-26

The results of the 'issuing and filling in', 'collection' and 'processing' components of the state SIM system are linked to the influence of the degree of implantation achieved, reflecting adherence by municipalities to the system’s operational routines. Conversely, the lower consistency of the 'management', 'distribution and control' and 'analysis and dissemination' components signals restrictions in managerial activities, given the inadequate training of technical staff, inadequate planning and monitoring of actions and information dissemination. These findings reflect the centralizing and fragmented conception of health information systems, historically designed to establish a network of local production of information that respond to national policies.^1,27

Notwithstanding the role of executing the actions promoting the empowerment of municipal management,²³ such decentralization does not just mean deconcentration of activities but also the sharing of decision-making power, thus promoting management and work process autonomy.²⁷ Empowering health regions and municipalities technically and administratively would be a suitable alternative for consolidating SIM. This procedure, combined with the incorporation of new routines, favors the operationalization of the system, with a reduction of omission, overlap or partial execution of actions.^6,28

SIM was found to be partially implemented in the state of Pernambuco, due to inadequacies in the distribution, control, analysis and dissemination of data, negatively influencing the effects observed, while data collection, issuing and filling in obtained the best scores. SIM’s objectives can be achieved by reorganizing the entire data production process, particularly within services, where health care is provided, and where deaths occur. To this end, additional investments are needed in the health service infrastructure, in strengthening of hospital epidemiology hubs and improving the work process. Such initiatives, coupled with periodic evaluations, are indispensable for the qualification of vital information systems and, therefore, for ensuring accuracy in the analysis of the health conditions of the population.²⁹