Introduction

In the historical context, with the recognition of having saved thousands of lives, mass vaccination has been considered to be one of the greatest achievements of Public Health, from fundamental research developed in the late nineteenth century, to current studies and implementation of national immunization programs.¹ Despite the evidence of years of high vaccination coverage (>95%), in 2016, Brazil showed a fall in vaccination of approximately 10 to 20 percentage points.²

Vaccination coverage estimates are usually made by the indirect or administrative method: where (i) the aggregated data containing the total doses administered or distributed are the numerator, and (ii) a population estimate is used as the denominator. This method is useful for planning immunization program actions, but quite fragile in terms of accuracy.³ Computerized immunization records, in the same way as vaccination coverage surveys, allow individual data about the vaccinated population to be obtained as well as identification of low coverage micro-areas, i.e., pockets of susceptible people.^3,4

Some countries have been using this instrument for a long time, i.e., Canada, the United Kingdom and the United States have had computerized immunization records since the 1970s,⁵ while Latin American countries began implementation of computerized immunization records of national scope to improve vaccination coverage estimates with effect from 2006, with the exception of Mexico and Uruguay, which have been using this instrument since 1987 and 1991, respectively.⁶

In Brazil, the National Immunization Program (PNI) computerized records gathered by the PNI Information System (SI-PNI), have been being developed since the 1990s. However, it was only with effect from the 2010s that the SI-PNI began to be implemented nationwide.⁷

Prior to the SI-PNI being put in place, there were already some municipal initiatives in Brazil, such as the University of São Paulo (USP) Araraquara Special Health Service (SESA). Araraquara, a municipality in the state of São Paulo, has had computerized immunization records since 1987. SESA is now called the Juarez System and is probably the oldest service of this type in Brazil.⁵ Some studies have already been conducted using data from the Juarez System,^8-10 however its potential to analyze the spatial distribution of vaccination coverage, as well as the validation of this system regarding the data contained in vaccination record booklets, has not been explored yet.

The aim of this article was to describe agreement between the Juarez System immunization data and information from vaccination record booklets and vaccination coverage in children aged 12 to 24 months.

Methods

This was a descriptive study, with analysis of secondary data from the Juarez System and primary data from the household survey performed in the municipality of Araraquara, state of São Paulo, in 2018.

Araraquara had 236,072 inhabitants in 2019, with approximately 3 million live births per year and an infant mortality rate of 10.46 deaths per 1 million live births;¹¹ its municipal human development index (HDI) is 0.815¹² and 97.2% of the population is located in the urban area.¹³ In 2020, the municipality had 34 health centers, three general hospitals, two emergency rooms, a psychiatric hospital and a specialty outpatient clinic, covering about 60% of the local population. The municipality has also had a vaccination program identified by its high vaccination coverage, which is reflected in the control of vaccine-preventable diseases since the 1990s.¹⁴

Located in the municipality of Araraquara, SESA/USP has had electronic immunization registry since 1987, when it already had the individualized vaccination record of the child, with father’s and mother’s full name, date of vaccination and the type of vaccine, adverse events after vaccination, among other information, having improved in this regard over time. In 2011, SESA implemented municipal electronic medical records that include, in addition to the nominal vaccine record, consultation data and records of compulsorily notifiable diseases. That same year, electronic immunization registry started being called the Juarez System. It became online from 2012 and since then all health centers have accessed and recorded vaccination data in real time on the system. Vaccine coverage is 99.6% for children born in the municipality.⁹ The vaccination schedules, including the date and batch of each dose of vaccine administered, are verified via the Juarez System for each individual registered on it.

The object of this study was comprised of children born and living in Araraquara in 2015, registered on the Juarez System. Children whose mothers did not present the child’s vaccination record booklet during the vaccination survey, conducted with a probability sample extracted from the Juarez System by selecting at random without replacement, were excluded. Each of the 15 weighting areas of the Brazilian Institute of Geography and Statistics (IBGE) was considered a stratum sample,¹⁵ data were collected in all 15 strata, by means of interviews with the mothers of 3,394 children born that year and living in the municipality, proportionally distributed among the areas weighted by the IBGE.

The following were taken into account for the calculation of the minimum sample size: 95% confidence level, the value of which was 1.96 for a 0.05 alpha; 40% frequency of vaccination coverage, based on the study by Ferreira et al.⁹ that analyzed the vaccination coverage of 49,741 children under 2 years of age, born from 1998 to 2013 in Araraquara; and 0.05 maximum error in absolute value.¹⁶ Thus, a minimum sample size of 369 children was obtained.

Of a total of 3,394 children registered on the Juarez System in 2015, 3,054 had their addresses geocoded and categorized in their respective IBGE weighting areas,¹⁵ in order to make up the final population for the random sample (Figure 1). From this final population, 450 children were randomly selected; 20% of the minimum sample size calculated was added to compensate for possible losses.

Figure 1 Recruitment and exclusion process of children under 2 years of age, Araraquara, São Paulo, Brazil, 2018 

In its evaluation of the completeness of the vaccination schedule, the following recommendations for vaccination schedule were considered by the PNI:¹⁷

Valid doses were considered to be those administered respecting minimum and maximum age at dose administration, as well as the appropriate interval between doses for multidose vaccines (Figure 2).

The vaccine survey data collection from took place between August and October, 2018, by means of a questionnaire administered at home by 10 field interviewers and four supervisors, who had taken a theoretical-practical training with a workload of 10 hours, distributed over two consecutive days. Previously, a pilot study had been conducted to test the instrument’s adequacy, involving 20 mothers from the municipality of Araraquara who were not included in the study. Secondary data were obtained from the Juarez System.

The levels of agreement between the Juarez System vaccination data and the information from the vaccination record booklets collected through the vaccine survey were verified by the Kappa test, categorized as follows: almost perfect agreement when the Kappa coefficient was ≥0.80; substantial agreement, 0.60 to 0.79; moderate agreement, 0.41 to 0.59; fair agreement, 0.21 to 0.40; and poor agreement, when the Kappa coefficient was <0.20.

The Geographic Information System (GIS) was used for the spatial description of vaccine coverage data, calculated for each of the 15 weighting areas of the municipality and used for drawing maps showing its distribution. To this end, QGIS 3.4.1 software was used taking an IBGE cartographic database.¹⁸

The present study is part of a larger project called ‘Maternal vaccine hesitancy and the vaccination status of children up to two years of age’ conducted in 2018. It was approved yet again, this time by the Research Ethics Committee of the School of Public Health of the University of São Paulo (CEP/FSP/USP): Opinion No. 3.617.912, October 3, 2019; Certification of Submission for Ethical Appraisal (CAAE) No. 20721819.0.0000.5421. Only children whose mothers signed the Informed Consent Form before the start of the interview questions forming the basis of the vaccination survey were included in the study.

Results

Of the total of 450 children admitted to the study, 4.7% were excluded: 429 children comprised the final sample (Figure 1).

Agreement was found between the two data sources, in a proportion varying from 84.1% to 99.1%. There were more doses recorded in vaccination record booklets than on the Juarez System, and this difference was less than 5% for most vaccines (74.1%) (Table 1).

The municipality of Araraquara had vaccination coverage ranging from 86.1% to 100%. When considering the full schedule, completeness of vaccination coverage at 12 months of age was 77.1%, while for vaccination completeness at 24 months of age, coverage was 68.8% (Figure 3).

Figure 3 Updated vaccination coverage per dose of vaccines and complete schedule at 12 and 24 months of age (n=429), Araraquara, São Paulo, Brazil, 2018 

The spatial distributions of vaccination coverage did not show a defined pattern and vaccination coverage was heterogeneous: ranging from 25.7% to 100%. In addition, most weighting areas had vaccination coverage above 78% at 12 months old; this was not repeated for the vaccination schedule at 24 months old, when coverage above 78% was restricted to only four areas (Figure 4).

a) IBGE: Brazilian Institute of Geography and Statistics.

Figure 4 Spatial distribution of complete vaccination coverage at 12 and 24 months of age, according to IBGEa weighting area (n=429), Araraquara, São Paulo, Brazil, 2018 

Discussion

There was agreement between the Juarez System and the records in the vaccination record booklets. There was also high vaccination coverage, especially vaccines administered up to 12 months of age. The spatial distributions of vaccination coverage did not show a defined pattern; in fact, distribution was heterogeneous, both for the 12-month and 24-month schedules.

The use of secondary data can be considered a limitation of this study, given the possibility of showing incompleteness or inconsistencies in the records. However, by using these data together with the primary data of the vaccine survey, it was possible to analyze the quality of the computerized immunization records, both structured and consolidated. The sample loss, which could also represent a limitation for this study, was not only small but also remained within the acceptable margin provided for in the sample calculation, and did not have power to interfere in the results obtained.

The high vaccination coverage found has also been described in other studies on those born in Araraquara in 1998 and 2013.^9,10 Such coverage may be related to the organization of the municipal immunization program and good performance of the Juarez System, which has been under continuous improvement over time. It may also be related to the use of efficient tools in the monitoring of vaccination coverage. This is the case of the absentee/late (recall) report, which allows active tracing to get the vaccination schedule up to date, and children’s scheduled vaccines (reminder) report, for increasing adherence and taking advantage of vaccination opportunities.⁹

In the same way as the results of this study, the 2007 Brazilian vaccine survey conducted in the 26 state capitals and the Federal District showed that 82.6% of children received all vaccines by 18 months of age.¹⁹ Other studies, such as that conducted in São Luís, capital of the state of Maranhão, in 2006, by means of a household survey, found 71.9% vaccination coverage at 12 months of age;²⁰ in Pelotas, RS, data from the 2015 live birth cohort showed 77.0% coverage^.21 These studies emphasize the presence of good vaccination coverage.

Spatial distribution allows the heterogeneity of vaccination coverage between the weighting areas to be visualized and, therefore, micro-area vaccine coverage to be monitored, this being a fundamental aspect for PNI success. This method has been increasingly used in the health area, having contributed not only to the improvement of surveillance activities, but also to the identification of risk areas requiring intensification or prioritization of vaccine coverage measures.^22,23

Also with regard to vaccination coverage, the heterogeneity found was also reported in a study conducted in the state of Ceará, where, despite high vaccination coverage against measles, four of the 21 health regions did not reach the goal of 95%.²⁴ A study dedicated to the analysis of a time series (2013 to 2017) of vaccine coverage in a border state in the Brazilian Amazon, based on data available on the PNI Information System, showed the difficulty in maintaining homogeneous vaccination coverage on a national scale, given the low vaccination coverage in some regions of the country.²⁵ These results can be explained by the characteristics of Brazilian territory, its great social and cultural diversity, and the influence of sociodemographic factors, such as job/occupation - especially among mothers - family income, public education for children, low education of their guardians, high number of children per mother and birth order of these children, understanding of the reasons and importance of vaccination, trust in the public health system, and other equally determinant factors, such as physical distance and access to health care, in addition to programmatic issues, such as provision of vaccines and supplies via the cold chain.^26,27

The excellent agreement observed between the Juarez System and the child’s vaccination record booklets demonstrates that in Araraquara, the computerized immunization registry data on vaccination coverage is highly accurate. This finding corroborates the understanding that SESA’s pioneering experience in developing and using the Juarez System over more than three decades has been both successful and has also contributed to the quality of the PNI and its good vaccination coverage in Araraquara. The efficiency in the use of computerized immunization records, in addition to favoring the increase in vaccination coverage, as has already been reported,²⁸ reaffirms the importance of accuracy and representativeness of population data records, which can be used in action planning^.29

The results achieved enable this study to guide public policies for the rest of the country and to inform the National Immunization Program Information System. They highlight computerized record potential for expanding the coverage and qualification of ongoing immunization programs, as an efficient tool for monitoring and evaluating vaccine coverage, providing identification of micro-areas with low vaccination coverage and pockets of people susceptible to vaccine-preventable diseases.

Taking these results, it can be concluded that the Juarez System is a very reliable computerized immunization registry, regarding its data, and that it is useful for vaccination coverage monitoring and surveillance. Araraquara, moreover, has good vaccination coverage, although its spatial distribution has proved heterogeneous.