Introduction

Immunization is a priority, effective and strategic action of Primary Health Care. In Brazil, immunization services have been changing since the National Immunization Program (NIP) was created in 1970.¹ NIP is a source of pride, because of its inclusive characteristic, aiming to assist the entire population, through actions for standardization, supervision and development of policies and strategies that provide access to immunization by all the population.^1-6 NIP aims to provide safe immunization to 100% of children under one year old, and also ensures vaccination for all children under five years old who had not been vaccinated or did not complete the basic immunization schedule in their first year of life. It seeks to improve health situation as well as cost reduction concerning the treatment of vaccine preventable diseases.^1,7-9

Immunization services are ruled by the Brazilian Ministry of Health, but the effective structuring and organization within the Brazilian National Health System (SUS) are responsibilities of the states and municipalities. The municipalities play an important role in the operation and responsibility for all activities of the Primary Health Care, including those regarding planning and organizing immunization at the local level. The municipalities are responsible for providing appropriate spots for storage and administration of the vaccines, as well as having trained nursing professionals to manage, maintain and manipulate properly the immunobiologicals. They must also monitor and assess immunization activities in vaccine rooms.^1,5,9-14

The assessment of health services is an important way to gather information that may subsidize future actions in health care. Health managers have been advocating that services should establish tools to measure the quality of health care by adopting proper evaluation systems and performance indicators to support the State in decision making, aiming to improve the quality of health care services. Quality assessment, structure and operation of vaccine rooms is a complex activity but necessary to identify crucial elements in the development of the work, and it is also a way of checking if the services offered in those rooms meet NIP recommendations.^{6,11,13,15,16}

NIP states that the assessment of vaccine rooms must be systematic and conducted by applying a semi-structured questionnaire of the Ministry of Health's Evaluation Program of the Vaccine Room Supervision Tool (PAISSV).¹⁰ Such questionnaire will verify the compliance with standards that may contribute to improve the quality of the services provided and, therefore, to the success of the Program.^{1,2,5,9-11,13,15,17} However, the assessment made by the PAISSV has been carried out irregularly and sporadically, making it impossible to identify weaknesses of NIP activities related to organizational and operational aspects of immunization. The implementation of a permanent evaluation system concerning the quality of structure, organization and operation of vaccine rooms is a necessary strategy to improve the quality of those services.^12,18,19

The present study aimed to assess the quality of organization and operation of vaccine rooms in Montes Claros, Minas Gerais, Brazil, in 2015.

Methods

A descriptive evaluation study was conducted from June to August, 2015, in Montes Claros, a municipality located in the north region of Minas Gerais, which had a population of about 394,350 inhabitants in 2015.²⁰ In that same year, there were 68 Primary Health Care Units (PHU) in the municipality, which were visited by users registered in 125 teams of the Family Health Strategy (FHS).

All vaccine rooms of Montes Claros public health network were included in the study. This information was provided by the Municipal Department of Health.

Data were collected through interviews that were carried out by two undergraduates from the nursing course. They were previously trained and conducted the data collection in the presence of a nurse or a nursing technician in each room visited. For the interview, a version of the PAISSV 2.0 questionnaire from the NIP was used. This version was previously tested and adapted after a pilot study conducted in five vaccine rooms of another municipality in the same region.¹⁰ The undergraduates who carried out the interviews observed procedures and routines of every vaccine room for four hours to obtain information regarding the physical structure of the rooms, and knowledge and practices of the nursing team.

The instrument of data collection gathered questions related to five dimensions:

After conducting the pilot study, the researchers made some changes concerning the response pattern: the options 'does not know', 'did not answer' and 'ignored' were added, once the original questionnaire had only 'yes' and 'no' answers.

In the tabulation of results, the replies received weighs, according to the criteria of the NIP General Coordination and PAISSV guidelines.^1,10 Absolute frequencies of quantitative variables were described and then a score was assigned to each question: one point to adequate or positive items and zero to inadequate or negative items. The final score was determined by the sum of values received in the items, in each of the five aforementioned dimensions. The quality classification of each room was estimated as follows: a score from 90 to 100% was considered ideal (concept = 3); from 76 to 89%, good (concept = 2); from 50 to 75%, regular (concept = 1); and less than 50%, insufficient (concept = 0).¹ The higher the score, the more items were in accordance with the NIP standards and guidelines, that is, the room quality was better.¹

The analysis of the data was performed using the following applications: SPSS®20.0 version for Windows® and Microsoft® Office Excel® 2010.

The study was approved by the Research Ethics Committee of the State University of Montes Claros (Unimontes), Report No. 313,870/2013, dated June 24^th, 2013, in compliance with the Resolution No. 466 of the National Health Council (CNS), dated December 12^th, 2012.

Results

Montes Claros has 18 vaccine rooms, operating in the morning and afternoon shifts at Primary Health Care units. Those units count with 74 professionals of the FHS to assist the population’s demands. Among the 18 vaccine rooms, 17 were within Primary Health Care units located in the urban area, and presented identification, that is, a board or a signpost outside the door.

Fifteen rooms opened from Monday to Friday, for 8 hours a day, and three rooms worked 6 hours a day, during the morning and afternoon shifts, closing at lunchtime. All rooms were exclusive to vaccination. However, in 17 of them, not all the routinely recommended vaccines were available.

With regard to human resources, the staff of vaccine rooms was composed of 74 professionals: 54 nursing technicians, 2 nursing assistants, and 18 nurses. All nurses worked both as managers of the health units and supervisors of the vaccine rooms. Only 29 out of 74 professionals had taken some training course, regardless of the year of accomplishment, to work or perform activities at vaccine rooms; 21 had received training in adverse events following immunization and 28, in cold chain.

With regard to general aspects and operation of vaccine rooms and technical procedures/ indication to vaccines, 16 out of the 25 items evaluated received insufficient or regular concepts: most of them did not have washable walls (n=14), resistant (n=12), non-slip (n=12), waterproof (n=12), and washable floors (n=12), adequate ventilation (n=10) and temperature between 18 and 20°C (n=16). In the assessment regarding technical procedures and indication for vaccines, most rooms did not consider two critical aspects, according to the PAISSV¹⁰: the active search for defaulters was not mentioned by 11 individuals; and the selective packaging of waste was not cited by 14 (Table 1).

Table 1 - Description and distribution of scores attributed to the general aspects and operation of vaccine rooms (n=18), technical procedures and indications to vaccines, Montes Claros, Minas Gerais, 2015 

a) Critical aspects according to the Evaluation Program of the Vaccine Room Supervision Tool (PAISSV).

With regard to the cold chain, 5 out of the 35 evaluated items obtained insufficient or regular concepts. When professionals were questioned about the procedures related to temperature changes, most of them reported that they used to inform immediately to their superiors (n=16), however 10 of them did not fill in the form of immunobiologicals subjected to non-recommended temperatures. Among the 18 rooms, just five had a preventive and (or) corrective maintenance program for the refrigerator that stores the vaccines (Table 2).

Table 2 - Description and distribution of scores attributed to conditions of the cold chain of vaccine rooms (n=18), Montes Claros, Minas Gerais, 2015 

a) Critical aspects according to the Evaluation Program of the Vaccine Room Supervision Tool (PAISSV)

b) PVC: polyvinyl chloride

Regarding characteristics and estimates of the scores given to vaccine rooms, 12 out of the 36 evaluated items received insufficient or regular concepts. Only one room monitored and sought to understand vaccine coverage, only one room was aware of the proportion of abandonments, and also only one room was aware of the occurrence of vaccine-preventable diseases in its coverage area, whilst five scheduled the vaccines properly (Table 3).

Table 3 - Description and distribution of scores attributed to vaccine rooms (n=18) regarding the information system, standardization, post-vaccination adverse events, special immunobiologicals and epidemiological surveillance, Montes Claros, Minas Gerais, 2015 

a) It was considered critical point by the Evaluation Program of the Vaccine Room Supervision Tool (PAISSV)

Note: The probable inconsistencies between the number of rooms with items which characterize the appropriate conditions for the operation of the cold chain, and the number of rooms with reports on their correct filling occur due to the fact that the presence of items was verified by observation during the research, whilst the information on the correct filling capacity was asked to the supervisors of the rooms, even when the item was not present.

As to health education in the public network, 2 out of the 8 evaluated items obtained insufficient or regular concepts. Half of the rooms (n=9) had partnerships with several social segments to disclose immunization actions. Another information that stood out was the fact that professionals of only 10 out of the 18 vaccine rooms reported to be aware of the importance of being properly vaccinated (Table 4).

Table 4 - Description and distribution of scores attributed to vaccine rooms (n=18) in relation to health education in the public network, Montes Claros, Minas Gerais, 2015 

a) It was considered critical point by the Evaluation Program of the Vaccine Room Supervision Tool (PAISSV)

With regard to the scores attributed to vaccine rooms, all rooms obtained insufficient or regular concepts in at least 3 of 9 items in the evaluation process. The assessment of special immunobiologicals and epidemiological surveillance was considered regular or insufficient in all rooms. The assessment of adverse events following immunization and health education was considered ideal or good in all rooms (Table 5).

Table 5 - Distribution of scores attributed to vaccine rooms (n=18) according to items from the evaluation process of the Evaluation Program of the Vaccine Room Supervision Tool (PAISSV), Montes Claros, Minas Gerais, 2015 

As to the general scores of the municipality vaccine rooms, 6 rooms were assessed as ideal, 5 as good, 4 as regular, and 3 as insufficient (Table 5).

Discussion

The technical standards on immunization defined in most evaluated dimensions, have not been fulfilled in the vaccine rooms of Montes Claros, as established and determined by the NIP.⁶ Not all rooms opened from Monday to Friday, for eight hours a day, and the lunch break of the staff may hamper the access of those users who seek for treatment at lunchtime. There is a need to review the working hours of the rooms, although the NIP suggests as ideal six to eight hours a day. The better timetable is essential to ensure the population access to immunization services and, therefore, it is an important indicator of the assessment of a public service.^1,5,21

The staff reported that they had not been informed about the importance of being vaccinated, showing the fragility of professionals in the vaccine rooms. This fact deserves greater commitment of managers to articulate epidemiological services and health education to the NIP. The lack of identification signs observed in some rooms can make it difficult to the users to find and access them, and this may lead them to giving up immunization.^5,21

The nurses’ work, acting as unit managers and supervisors of the vaccine rooms, can compromise the population immunization due to their overwork. Although all rooms were exclusive for vaccination, most of them did not offer all vaccines. The recommendation is the daily offer of all vaccines of the basic schedule suggested by the NIP. Such inadequacy may probably have happened due to specific facts related to BCG vaccine: usually, that vaccine is given in maternity hospitals right after the child birth. In case the infants have not been vaccinated at the hospital, the immunization should be scheduled at the vaccine rooms.^8,9,22

The lack of training observed in the staff responsible for vaccine rooms shows that there may be operation difficulties concerning clinical indication and contraindication, management of side effects and of adverse reactions to immunobiologicals. Those professionals need to develop a critical and reflexive view of their work, as well as receive constant updates on the services to be provided in vaccine rooms. It is worth noting that introducing new vaccines into the NIP schedule demands training for the development of a range of skills and knowledge by those professionals.^7-9,11,21

A great part of the rooms showed serious structural problems. Similar problems were identified previously in 2008-2009, in the municipality of Marilia, São Paulo State,⁵ and in 2011, in Pernambuco State,² mainly regarding the walls (color, permeability and being washable), cleaning conditions and room preservation. These are important recommendations concerning the continuous manipulation of immunobiologicals and the need of a flawless sanitation of the place.⁹

The spot destined for handling and administration of immunobiologicals must follow - strictly - conservation and cleaning standards, ensuring security to users. Most vaccination spots may present susceptibility, which makes necessary its adequate sanitation. Moreover, vaccines, when exposed to temperature variation, may lose power rating; some of them even have their aspects altered due to changes in their physical-chemical characteristics.^9,23

Not all syringes and needles were properly packed, nor all vaccine rooms had examination tables with impermeable mattresses protected with disposable material. According to the NIP recommendations, syringes and needles must be packed and kept in closed cupboards, preferably; in the absence of cupboards, they can be replaced by clean plastic containers with lids.^2,5,9

With regard to the technical procedures/indication to vaccines,⁶ some evaluated items did not meet the standards established by the NIP: more than half of professionals answered that they did not carry out active search for susceptible cases; they did not check the expiration dates of syringes and needles before administering the vaccine and did not pack properly and separately the various types of waste. It seems to be common to find vaccine rooms without adequate containers for the disposal of contaminated waste, which leads to improper disposal. In the Manual of rules for vaccine rooms, it is stated that the waste coming from immunization activities, such as biological material, perforating and infectious residues, is classified as dangerous, whilst the other wastes are considered common. Due to the specificities of this type of waste, it must be discarded separately. These actions are part of the planning, provision of adequate conditions of stocking and final destination of waste.⁷

The lack of preventive maintenance of refrigerators may affect the effectiveness of vaccination, which not only is restricted to its production, but also implicates on how strictly the standards of storage, transportation and conservation are followed.^5,21,24-26 In some rooms, for example, the refrigerators were not working properly.

All rooms observed performed daily temperature control. Moreover, all professionals reported the importance of carrying out daily temperature reading and stated that they used to do it at the beginning and the end of their working hours, in a printed form, notifying supervisors in case of changes. However, it was found that in half of the rooms observed, the daily temperature control map was not displayed in a visible place, which could affect the correct monitoring on the internal temperature of the refrigerator.^{3,13,15,16,27}

Difficulties in the management of activities hamper a proper practice of the process of vaccine conservation, possibly because of the absence of standard operational procedures that can be supervised, monitored and assessed. The discussion on this item calls attention to the need of a control of activities and monitoring of processes involving the manipulation of such substances. This could be resolved by implementing education actions in the workplace, resulting in possible corrections and adjustments in the training process with professionals being properly guided by supervisors of vaccine rooms and municipal health managers. Appropriate working conditions must be ensured as well, so that the recommendations of the Ministry of Health are followed.

In the assessment related to the quality of records, we found that most professionals did not schedule vaccines correctly; besides that, about half of the rooms did not have forms to evaluate vaccines under suspicion. Vaccination schedule must be performed accurately because it defines whether or not the individual needs to receive doses of vaccines. Only with these data it is possible to prevent people from receiving unnecessary doses, increasing the possibility of local reactions due to immunization. When seriously and strictly conducted, this procedure can also prevent overdoses of immunobiologicals and unnecessary expenses.²⁸

Almost all professionals were unaware of vaccine coverage and proportion of abandonments in the area covered by their health unit, in disagreement with the NIP: professionals who work with immunization are responsible for performing the calculation of vaccine coverage and proportion of abandonment; the capacity of the service to convene defaulters, perform vaccination and complete schemes are based on these data.^24-26

With regard to adverse events following immunization, most professionals identified, notified and investigated adverse events related to vaccination. Notifying adverse events is indispensable, once this indicator works as instrument for improving the quality of the NIP. Vaccines are not 100% effective, nor 100% safe.²⁹

Regarding special immunobiologicals, almost three-quarters of the professionals were aware of the existence of the Reference Center for Special Immunobiologicals (CRIE), as well as the indication and availability of immunobiologicals. Nevertheless, little more than a quarter of them knew how to request these special immunobiologicals. Professionals of vaccine rooms have to understand the importance of these immunobiologicals. They also have to guide and refer people with indications for their use, due to the possibility of adverse events.

As to the epidemiological surveillance actions, almost all professionals were unaware of the number of cases concerning vaccine-preventable diseases in their coverage area, and little more than half was aware of the incidence of vaccine-preventable diseases and their relation with vaccine coverage. A similar fact was found in a previous study that assessed vaccine rooms in the primary health care of the municipality of Marilia, São Paulo State, in 2008 and 2009.⁵ It is common to hear professionals mentioning they do not know about new cases of vaccine-preventable diseases in their coverage area and corresponding vaccine coverage.

Regarding health education actions, in most vaccine rooms we observed the participation in partnerships with programs in the health units, as well as several events that aimed at spreading the immunization program actions. However, it should be emphasized the need for vaccine rooms professionals to perform activities in partnership with several social segments in order to disseminate immunization actions, even those programs already exist. Those partnerships occurred in only half of the analyzed vaccine rooms, and this is one of the main obstacles to overcome to achieve the goals proposed by the Ministry of Health, in order to ensure the control of vaccine-preventable diseases.

As limitations of the present study, it is important to highlight the use of an instrument that does not take into account the sociodemographic and economic condition of each region, service and community. Another limitation is that the information on professionals' knowledge was obtained only from the employees working in the vaccine room at the time the data were collected. In vaccine rooms where other professionals worked, no information was obtained on the knowledge of who was not interviewed. As positive aspects, we can mention the inclusion of all municipal vaccine rooms and the observation in loco of procedures and routines performed, which reinforces the internal validity of the study.

Thus, we recommend the systematic implementation of supervision, monitoring and assessment of vaccine rooms of Brazilian municipalities, because there are few studies with this approach.^2,5 Some items considered relevant for the performance of immunization activities were in disagreement with the standards of the NIP, reinforcing the importance of assessment to improve the actions offered by health services.