Introduction

The implementation of the Information System on Live Births (Sinasc) in Brazil happened gradually in the federation units, since the early 1990s.¹ The main objective of the system is to subsidize the plan for maternal and child health care based on the epidemiologic profile, according to the information inserted into the Certificate of Live Birth (CLB).²

Such information is the basis for the calculation of important health indicators, such as child and maternal mortality rate, birth rate and vaccine coverage, besides allowing health diagnosis, surveillance and monitoring of newborn at risk, identify the availability of services that perform childbirth and analyze the territorial and time distribution of these events, besides supporting public health policies in the field of maternal and child health.^3,4

In this context, it is essential to assess the attributes of Sinasc and several studies have analyzed the quality of its data (completeness and validity of the records), its acceptability (the willingness of people and institutions to participate and contribute to the system), representativeness (precision of the data), opportunity (punctuality), stability (coverage) and usefulness(data source for indicators).^5-11

To assess the reliability of the information in SINASC, we can compare it to other databases, such as the National Hospital Information System (SIH/SUS) and the Mortality Information System (SIM), to information in hospital records or to data collected in field studies.^12,13

Incomplete or absent data in the CLB records is one of the greatest challenges to overcome in order to achieve a high quality information system.¹⁴ A literature review on the use of Sinasc described that studies conducted between 1996 to 2002 reported a common problem: the high rate of incompleteness of CLB fields.¹⁵ Other studies evaluated the characteristics of the system, focusing in analyses of its coverage, quality and reliability.^4-6

Considering the complex nature of the processes involved in collecting the data entered into the CLB, it is essential to assess the quality of the information recorded in the system, to then develop interventions to improve it to evaluate the quality of the information recorded in the system, aiming to the adoption of measures to improve it.

The objective of this study was to evaluate the coverage, completeness and the reliability of the information about live births (Sinasc) in maternity wards of the public health network in the municipality of São Paulo, Brazil.

Methods

This cross sectional study was conducted in four public maternity wards of hospitals from the Brazilian National Health System (SUS) in the municipality of São Paulo. This is a sub-project of a previously published study ("The study of the mother-child binomium").¹⁶

São Paulo is the largest municipality of Brazil and in 2011 it had approximately 11 million inhabitants, according to estimates of the Brazilian Institute of Geography and Statistics (IBGE).¹⁷ In that same year, 193,383 live births occurred in the municipality, according to the SINASC records. Amongst these births, 169,492 were from mothers who lived in São Paulo.¹⁸ The Municipal Human Development Coefficient (MHDI) was 0.805 in 2010, occupying the 28^th position among the 5,565 Brazilian municipalities.¹⁷

Since Sinasc was implemented in the municipality of São Paulo, the hospitals that have deliveries fill and type the CLB directly into an electronic database, which facilitates data collection, minimizes sub-notification and especially the recording delay. The information used to fill the CLB and feed the SINASC system are obtained by nurses and/or administrative personnel by checking medical records and through interviews with the mothers.¹⁹

In the 'The study of the mother-child binomium: an imperious necessity to reach the Millennium Development Goals', the data were collected in four maternity wards: Amparo Maternal Hospital, Maternity School Municipal Hospital Vila Nova Cachoeirinha Dr. Mário de Moraes Altenfelder Silva and Leonor Mendes de Barros Maternity Hospital, in the period from August 1^st to October 31^st, 2011; and São Paulo University Hospital, from September 1^st to November 30^th, 2011. The four maternity wards participating in the study, selected by convenience, are part of the SUS network: two are under municipal management and two are under state management. They contributed with 12.6% (24,357) of the total number of live births that occurred in the municipality of São Paulo in 2011 (Table 1).

Characteristics of women in the immediate pregnancy/postpartum cycle and the product of their pregnancy were analyzed. The data collection was made through interviews with the postpartum women, by nurses, and through searches in the medical records, prenatal care cards and other hospital records to complement the information, later typed in appropriate forms that contained open and closed questions.¹⁶

Since several maternals and newborn variables collected for the "Study of the mother-child binomium" are also part of the CLB, we compared the data entered into the SINASC database with the data collected for the study. We considered the information collected in the "Study of the mother-child binomium" as the gold standard.

Our study had three separate phases: standardization of the variables, data pairing and evaluation of the agreement, described below:

a) ‘Study of the mother-child binomium: an imperious necessity to reach the Millennium Development Goals’.

b) Apgar score: assessment of the newborn's vital signs after one and five minutes of life.

Figure 1 - Variables and categories in the evaluation of coverage, completeness and reliability of the data of the Information System on Live Births (Sinasc) in comparison to the Study of the mother-child binomium,^a in four hospitals from the Brazilian National Health System's (SUS) network, São Paulo, SP, 2011 

The quality evaluation was based in three aspects -coverage, completeness and reliability -, defined below:

For data organization and treating process, the software Excel® was used, and for kappa coefficient calculation, the PASW Statistics 17.0.

The ‘Study of the mother-child binomium: an imperious necessity to reach the Millennium Development Goals’ was conducted by the Epidemiology Department of the Public Health School of São Paulo University (FSP/USP) in 2011, after being approved by the FSP/USP Ethics Research Committee on August 24^th, 2011, under the No.253/2011 (ID of the project in the Brazilian National Commission of Ethics in Research [Conep]: 0002.0.220.207-11). All participating mothers gave written informed consent.

Results

During the study period, 5,772 live births were recorded in Sinasc in the partipating hospitals while the ‘Study of the mother-child binomium’accounted 5,579 interviews (excluding nine mothers who refused to participate), distributed as follows: 1,626 in Amparo Maternal, 1,603 in Cachoeirinha Maternity Ward Hospital, 1,597 in Leonor Mendes de Barros Hospital and Maternity Ward and 753 in São Paulo University Hospital.

A total of 5,566 registers were paired between Sinasc and the ‘Study of the mother-child binomium’. To calculate the coverage, the paired registers were added to the remaining registers of each database, for a total of 5,785 LB. Sinasc had a coverage of 99.8% (5,772) and "The study of the mother-child binomium" had a covrage of 96.5% (5,579) (Figure 2).

a) ‘Study of the mother-child binomium: an imperious necessity to reach the Millennium Development Goals’.

b) Registers present in the Study of the mother-child binomium database and absent in the Sinasc database.

b) Registers present in the Sinasc database and absent in the Study of the mother-child binomium database.

d) Registers found in both databases.

Figure 2 Coverage of the Information System on Live Births (Sinasc) and from the Study of the mother-child binomium^a (N and %) in four hospitals from the Brazilian National Health System's (SUS) network, São Paulo, SP, 2011 

To calculate the completeness of information, the 5,566 matching registers were considered, representing to 96.2% of the total. In the CLB, the percentage of absent information in the variables studied was very low compared to the corresponding percentages found by the ‘Study of the mother-child binomium’, specially for: number of prenatal care visits(0.02% versus 3.72%), pregnancy duration (0.05% versus 0.52%), mother's education level(0.04% versus 0.31%), Apgar escore at 1 minute (0.16% versus 0.49%) and at 5 minutes (0.16% versus 0.34%) of the newborn's life. The following variables had 100% of completeness in Sinasc: mother's age, previous live birth or stillborn children, types of pregnancy and delivery, weight at birth and sex (Table 2).

To calculate agreement, the ignored or unfilled data were excluded. According to the kappa coefficient, the agreement between the existing information in Sinasc and in the ‘Study of the mother-child binomium’ was classified, in accordance to kappa coefficient, as excellent for the mother's age (0.99), type of pregnancy (0.98), weight at birth (0.98), sex (0.98), Apgar at 1^st (0.96) and at 5 minutes (0.95) and previous live births (0.87). Agreement for maternal educational level was good (0.62) while prenatal care visits (0.60) and duration of pregnancy (0.56) had a moderate agreement. The agreement was weak for stillborn children (0.09) (Table 3).

Discussion

In the evaluated hospitals, Sinasc proved to be reliable and had a high coverage (99.8%), confirming the low under-registration of deliveries and the excellent completeness of the variables, in the comparison with the ‘Study of the mother-child binomium’. The Ministry of Health recommends that all CLB fields should be filled -some are mandatory, others are not-to avoid CLB with incomplete data in the system. This condition justifies the relevance of the measurement of this ocurrence.⁶

A study conducted in 2002, analyzed Sinasc in eight Brazilian states, including São Paulo and reported excellent completeness (99%) for the selected variables, especially birth weight, type of pregnancy and mother's age, with values similar to those found in this study, while duration of pregnancy and maternal educational level had the largest percentages of incomplete data.¹²

A nationwide study evaluating the quality of Sinasc epidemiologic and demographic variables for the year 2002 revealed that in most of the Brazilian states, completeness was excellent (>95%) for mother's age, sex of the live born and mother's marital status, and good (from 90 to 95%) for education level, birth weight and number of prenatal care visits.⁵

Ignored or non-filled variables can be caused by a series of isolated and/or simultaneous problems: unintelligible writing, lack of information or difficulties in finding it in the medical record; a deficient flow of information between the hospital sectors (admission, obstetric center, maternity ward, neonatology, medical archives); lack of knowledge about certain information by the mother or her companions. On the other hand, unfilled variables can also be due to lack of attention, reckless and/or lack of knowledge -of the person who fills the CLB-about its epidemiologic and legal meaning.⁸

The reliability of, the variables pregnancy duration (0.56) and number of prenatal care visits (0.60) had moderate agreement, which may be attributed, in part, to the less accurate way to collect information in Sinasc, since they were collected in predefined group categories. The CLB form was modified in 2011, but the older version was still used in the evaluated hospitals during the study period. The CLB form currently used collects information on gestational age and the number of prenatal visits as continuous variables (in weeks and the exact number of visits) instead of collecting this infomation in categorical intervals, which was used in the older CLB version.²²

It is difficult to establish the accuracy of the gestational age to be informed in the CLB, because there are three different possible ways of reporting it: LMP (last menstrual period), obstetric ultrasound or clinical examination of the newborn (Dubowitz, New Ballard, Cappurro). This may lead to inconsistent information that can compromise the acuracy of an important indicator for prematurity.^20,22,23

The agreement for the variable number of previous stillborn infants was weak (0.09). The lack of of a clear definition of the meaning of this field does not contribute to the correct collection of this information in the CLB, causing confusion between the terms miscarriage and stillborn or even about live births who shortly died, informed in this field destined to stillborn children, generating misinterpretation or even reckless in this variable's filling.⁴ With the CLB model change in 2011 the terms: 'fetal losses/miscarriages' were added to the pregnancy's history.¹

The high level of agreement (over 0.61) in eight of the twelve evaluated variables demonstrates the data reliability and its adequacy for use, reflecting the compromise of these hospitals with the inclusion of the data in the system.

A qualitative study aimed at understanding the mechanisms and contexts involved in the CLB data production in the municipality of São Paulo, which used the collective discourse method to analyze the social representation of the health professionals responsible for filling these forms, revealed that they recognize themselves as part of Sinasc's information generation process, not only committed in the search for solutions before eventual filling difficulties, but also valuing the monitoring and support they receive from a higher level of the system. The aforementioned research also verified that the periodic qualification promoted, which have as target the team responsible for the municipality's Sinasc management, under a permanent education perspective, were considered space for devolution, essential to solve doubts and understand the meaning of the task made in their everyday work, the purpose and the uses of the information produced by them.¹⁰

There is a consensus that continuous education is essential to ensure the completeness and reliability of the data recorded in the system, especially because of the high turnover and the different types of professionals involved in filling the LBC. Responsible for filling the Live Birth Certificate. Periodic evaluation and monitoring also must be part of the health information systems management routine.^6,9,22,24

The Sinasc Label, introduced in the municipality of São Paulo in 2009, was a strategy created to conciliate evaluation, monitoring and encouragement, recognized through an annual certification bestowed upon health establishments that fulfill a pre-established coverage threshold (total of live birth deliveries), punctuality (inclusion of all the deliveries of the month in the system in at least 15 days after the delivery) and completeness of the variables (95 to 100%).^18,25

However, some challenges still exist: (i) improve the information with weak to moderate agreement and (ii) maintain the quality of the information rated as good. This evaluation is useful, because it contributes to guide the efforts and subsidize the review and to develop interventions to improve the completeness of the fields 'gestational age' (pregnancy duration), 'number of prenatal care visits' and 'number of previous stillborn children'.

Other evaluations of reliability would be opportune and enriching in the sense of overcoming the limitations of this study, possibly with the expansion of the period and number of included hospitals, allowing a more representative of the municipality or even matching Sinasc records in a regular and systematic way with the hospitals systems of information.

This study results, focused in four hospitals from the SUS network in the municipality of São Paulo, pointed to a high coverage, completeness and agreement of the researched variables, confirming the ability from the Information System on Live Births-Sinasc -to fulfill its aim: to provide objective analyses of the health situation and through it to subsidize the health care plans for pregnant women, mothers and children.