A pilot study of routine immunization data quality in Bunza Local Government area: causes and possible remedies

A pilot study of routine immunization data quality in Bunza Local Government area: causes and possible remedies

Semeeh Akinwale Omoleke^1,&, Menberu Getachew Tadesse¹

 

¹World Health Organisation, Office of Country Representative, Abuja, Nigeria

 

 

^&Corresponding author
Semeeh Akinwale Omoleke, World Health Organisation, Office of Country Representative, Abuja, Nigeria

 

 

 

 

In the presence of effective vaccines in the national schedule, childhood deaths from vaccine-preventable diseases (VPDs), such as pneumonia, diarrhoea, and measles, accounted for about 40% of all deaths among children under-five in developing countries including Nigeria [1, 2]. Evidence from the National Immunization Coverage Survey (NICS) indicates that variations exist in routine immunization (RI) performance across the country’s zones, with the South West (76%) and South East (91%) zones showing higher RI performance, and the North West (60%) and North East (46%) showing lowest performance [2]. This disparity ultimately impacts on national RI coverage.

 

The low or sub-optimal RI coverage is mainly due to factors like weak demand, poor service quality, inadequate human resource and poor local leadership and accountability [1,3]. More importantly, the reported administrative coverage is bedevilled with poor data quality in the face of the large pool of susceptible under-fives, which could lead to outbreaks of vaccine-preventable diseases (VPDs) at various administrative levels, i.e., Local Government Area (LGA) and State [2, 4]. Consequently, this could delay or reverse the gains made through supplemental immunization activities (SIAs) for diseases that are targeted for eradication and elimination such as Polio and Measles respectively [5].

 

According to administrative coverage reports, routine immunization showed consistent and increasing trend in Kebbi State, including Bunza Local Government Area (LGA). However, findings from community-level survey put in doubt the validity of these administrative figures, as there are disparities between these data sources. Perhaps, reported cases of VPDs, and sometimes outbreaks could be an indication of immunity gaps suggesting that the administrative coverage often reported may not be correct. Further, the surveillance and outbreak response reports in the state corroborated these findings. Similarly, national survey results following measles follow-up campaign in 2015 also revealed a great discrepancy between the reported administrative coverage and coverage surveys (104% versus 80.4%) [6]. Likewise, observations from community surveys during RI supervision by senior supervisors from the state (government and partners) showed huge difference among fully immunized children compared to the high reported administrative coverage in the same catchment health facility providing service to same settlement where surveys were conducted.

 

In the light of the above, World Health Organization recommended data quality checks at different levels [7]. Consequently, partners are making effort to support the government to address this issue. For instance, in Kebbi State, the attention of programme managers sand relevant stakeholders has been drawn to data quality issues during the routine technical review meetings and supportive supervision by partner agencies. Sadly, there was no formal assessment to specifically investigate the basis of the discrepancies and proffer possible remedies. Hence the need to pilot a study seeking to understand how the discrepancy originates along recording and reporting line, the extent of the discrepancy, and to identify the root cause(s) as well as recommend action points towards improving data quality. This study will also form a foundation for a more robust study, perhaps a state-wide assessment on this subject.

 

 

A cross-sectional facility and community level study was carried out in five health facilities (HFs) and selected settlements under each facility of Bunza LGA, Kebbi state, Nigeria between August and September 2016.

 

Study area

 

Bunza is one of the 21 LGAs in Kebbi state with a total population of 164,825 based on 2006 census population and 6,593 under-one population. The LGA shares borders with Kalgo, Arewa, Dandi and Zuru LGAs. There are 4 Traditional District Heads and 10 political wards under Bunza LGA. However, most of the wards are hard-to-reach for health services. There are 22 health facilities (HFs) in the LGA providing RI services through fixed and outreach strategies. The health facilities under study were selected based on their reported coverage of RI in August and September, 2016; the five health facilities reported maximum performance based on coverage.

 

Data collection

 

A semi-structured questionnaire was used to collect primary and secondary data from health facilities. The questionnaire was adapted from multiple tools such as the WHO Data Quality Assurance and RI monitoring tools while important areas believed to be relevant from supervisory observations were also included. For the household survey, the standardized community survey tool was improvised by including the proposed recommendations from mothers/care takers to address the reasons mentioned for not vaccinating or defaulting from immunization sessions.

 

For all the sessions conducted in September 2016, Penta 3 and measles antigens were checked in the Tally sheet, RI register, HF summary and the same copy sent to the LGA were included in the data check. In addition, same summary was checked at the LGA (received from HF and sent to the state). To explore the reasons for discrepancies of the RI data across the different data tools and at different levels, RI providers and RI programme managers were asked about the reason(s), the presence of regular monitoring mechanisms and feed backs for data quality, and what to do to address the issues.

 

One settlement was selected from each HF catchment area and 10 under-one year children were surveyed for their immunisation status giving a total of 50 children sampled from all the settlements. Similarly, mothers/caregivers of children who did not complete their antigen for their age and/or not started at all were interviewed about the reasons for the status of immunisation and their recommendations.

 

The WHO Field Volunteers and Local Government Area Facilitator were trained at the LGA Level and deployed for data collection. The tools were pre-tested at one of the HFs. The data collection processes were supervised by senior personnel from a partner agency, specifically, WHO Cluster Coordinator, and the entire processes were overseen by the State Coordinator.

 

Data analysis

 

Quantitative primary and secondary data from all sources were summarized, analysed and presented in tables and graphs while comparisons and conclusions made from summaries. Discrepancies of data across the different data tools were calculated by subtracting the values between two data tools with maximum difference taken as ‘numerator’, then the value in a data tool with minimum record taken as ‘denominator’, and then presented in percentages. Qualitative data analysis was made after developing thematic framework using responses, coded, compiled and summarized manually, then complemented with the quantitative results of primary and secondary data sources. Finally, the findings from the quantitative and qualitative data were presented in narrations and tables.

 

Ethical consideration

 

We obtained consent from all the RI providers, mothers/care takers and RI program managers at LGA level who participated in the study, after explaining the purpose of the study to them. We also got ethical clearance from the Research and Ethical Committee of the Kebbi State Ministry of Health.

 

 

A total of five health facilities were assessed using the DQA tool, all RI providers of the respective facilities, and one Local Immunisation Officer were interviewed with a response rate of 100%. Profiling the health workers showed that 50% of them were Community Health Extension Workers and the rest were Environmental Health Technicians (akin to an Ordinary Diploma) by qualification.

 

Data quality assessment results

 

From the Data Quality Assessment (DQA) at the HFs and LGA, discrepancies across the different RI data tools were observed in 100% of the HFs (Table 1). The difference was also observed in all the different data tools (tally, registration and HF summary). Maximum difference for penta 3 was observed between the HF monthly summary and RI registration book (820% and 767% difference) at MCH Bunza and PHC Balu respectively. Minimum difference was observed at Loko Dispensary (3%). For measles, maximum difference was noted between HF summary and registration at MCH Bunza (614%), and lowest at Loko Dispensary (43%) (Figure 1).

 

Reasons for data discrepancy

 

Interviews with RI providers on the reasons for data discrepancy include the following: non-availability of data tools at outreach sessions (poor logistics funding), no regular feedback on data quality from LGA, paucity of personnel leading to work overload (limited time available to record and complete data tools due to competing clinical responsibilities). These were the commonest reasons mentioned by 100%, 60% and 85% of the respondents respectively. Additional personnel during RI sessions (service delivery) deployed from LGA and the need to review and check data tools before sending to the next level were mentioned by RI providers as potential remedies for improving data quality.

 

Community survey findings

 

As can be seen from the table, findings of the community survey showed only 20%-40% of children were fully immunized for their age while 60-80% of children were either not vaccinated or partially immunized. The overall coverage for the LGA appeared to be 38% fully vaccinated (Figure 2). Mothers/caregivers provided reasons for not vaccinating their children or defaulting during RI sessions. Common reasons stated by mothers were succinctly captured as thus: “Sessions were not informed ahead of the schedule, sessions were interrupted, fear of side effects and minor complaints after injections and lack of awareness on benefits of vaccines” were among major reasons mentioned by mothers/caregivers. They also suggested “prior announcement and reminder for planned sessions should be made for them not to default their child from subsequent antigens administration”.

 

 

Routine immunisation services are being delivered at 22 health facilities in Bunza LGA via outreach and fixed post strategies. On the average, each HF conducts 8 sessions in a month. All sessions conducted with one RI provider who is otherwise known as RI-in-charge; meaning that he or she is responsible for all RI activities at that HF. Sadly, nearly half of the RI providers (50%) are environmental health technicians. According to the national guideline in Nigeria, Environmental Health Technicians are not expected to provide RI services [8]. It should be noted that all health facilities in Kebbi State developed ‘Reaching Every Ward’ (REW) micro plan every quarter for their catchment area and also used RI data tools (RI tally, registration, child health card and summary sheets) in documenting RI sessions.

 

Over all, the reported RI administrative coverage was satisfactory at the LGA level, despite the fact that Bunza is one of the least performing LGAs when compared with the other LGAs in the state [9]. DQA provides specific information as to which reporting units contributes more for variation and inconsistency in the reporting system and diagnose specific weakness that, if addressed, could improve the precision and data quality [7].

 

According to the data analysis finding at the two reporting levels (HF and LGA), differences were observed for antigens checked (Penta 3 and Measles) in all RI data tools (tally, registration, and summary). In a data quality audit conducted in Tunisia, re-counts from immunization registration books seem to provide more accurate data [10], hence, the basis of comparison or variation using the RI register in our study. The maximum difference for penta 3 was observed between the HF monthly summary and RI registration book at MCH Bunza and Balu (820% and 767% difference) respectively. This variation is the highest as compared with similar studies conducted in Nigeria and other African countries [10-12]. Minimum difference was at Loko Dispensary where only 3% variation was observed. For measles, maximum difference was between LGA summary and registration at MCH Bunza (614%) and lowest at Loko (43%). Likewise, the maximum variation for measles was 614% which is much higher than findings from data quality assessment conducted in Mozambique, where 268% variation was reported [11]. Different assessments on precision of data reported similar finding on data inconsistencies and missed data, and our assessment showed 79% accuracy ratio at health facilities which is less than 85% reported in Tunisia [10].

 

The cumulative reported administrative coverage for HFs and LGA was more than 100% whereas community survey findings showed greater discrepancies where the maximum coverage was found to be 70% at Tsamiya HF and lowest 20% at Loko HF, using survey data of fully immunised children. The LGA coverage from this HFs was 38% (fully immunized for their age) and 62% were either not immunized (32%) or partially immunized (30%). This finding is consistent with regular RI supervisory findings at community level using the same approach and tools (analysis of RI abridged checklist administration). Furthermore, this finding corroborates findings from the National Primary Health Care Development Agency (NPHCDA)-supported 2015 measles mass vaccination coverage survey that reported coverage of 80.4% by history and card and 58% by card only [6].

 

The above findings have implication on achieving herd immunity. Herd immunity is believed to be maintained if 90-95% “true” coverage is achieved and maintained in any defined geographical area for the different RI antigens [3]. In view of above findings, the eradication and elimination goal for targeted VPDs is in doubt, if these negative findings (poor data quality) are not addressed.

 

Based on our finding, heavy work load on RI providers was the major contributing factor for untimely tallying and registration on the RI data tools, particularly at outreach sites, where one provider could not be able to carry all the tools to the outreach session, and strained to register the client. In addition, efforts from high level supervisors were mainly targeted at improving overall coverage rather than the consistency of data using different data tools and at different levels. Paying more attention to data quality, de-emphasising on high coverage and use of multiple data sources to validate administrative coverage could be some of the proffered solutions towards halting and reversing these ugly findings.

 

According to the response from the mothers/parents of unimmunized and partially immunized children on the reasons for partial or non-immunisation of children, the fact that parents were not informed and were not aware of the sessions was the leading contributing factor, indicating a weak community link to health facilities and inadequate social mobilization. This is in spite of the existence of mobilizers and VDCs as part of primary health care (PHC) structure. These parts of PHC structure, however, needs revitalisation and greater community engagement. Previous studies [1, 13-15] also corroborated these findings (weak VDC and inadequate social mobilisation).

 

Parents’ fear of side effects after the first or second dose of an antigen was a contributing factor for partially immunized children. The reasons that emerged from this study is in agreement with the study conducted in Sudan [14]. The Sudanese study also pointed out that lack of awareness on benefits, not informed of the sessions and providers absent at service delivery points [14]. This underscores the need to improve the knowledge, attitude, and interpersonal communication skills of RI providers.

 

Further, it seems that RI in-charges and LGA officials are lacking in their understanding of the relevance of ensuring good data quality as it impacts on population immunity. Therefore, it becomes imperative to educate and emphasise to RI providers and programme managers at the LGA level on the (direct and indirect) consequences of poor data quality, unreliable routine administrative coverage in prevention and control of vaccine preventable diseases. This was also emphasised in the recent (2013-2015) National Routine Immunisation Strategic Plan [15].

 

Limitations of the study

 

The research conducted was limited to one LGA (Bunza) which may not be entirely representative of the degree of poor data quality in routine immunisation reporting in Kebbi State. However, the study provided a clue to understanding the origin of the discrepancy along the reporting chain, the magnitude of the discrepancy, the cause of the observed disparities, and the suggested evidence-based recommendations to mitigate the negative findings. Secondly, the study was limited to LGA-level data, however, it could have been more informative if state-level data were included.

 

 

This study revealed a significant data discrepancy in all antigens examined at all levels of the reporting line. Differences were also observed in the community survey where majority of the sampled children were found either not immunized or partially immunized for their age, which is a sharp contrast to administrative coverage data. Lack of focus on the data quality, poor understanding of the impact of poor data quality, inadequate logistics support, especially for conduct of outreach services, and high workload on RI providers were identified as contributing factors to the huge discrepancies observed in RI data. Hence, cognizant to the prevailing situation and the findings above, recommendations are provided as follows: 1) The immediate and medium term consequences of poor data quality should be emphasized through evidence-based advocacy to Local Government Authorities, Program Managers and Health workers (RI providers) while efforts should be geared towards swiftly addressing the identified problems. 2) Data quality component in the routine supportive supervision should be at the front burner while enforcing the use of RI data tools and emphasising the relevance of high data quality. 3) Staffing to match workload should be looked into to facilitate effective data collection and reporting, especially at the facility level, and further assessment on the quality of RI personnel delivering RI services should be considered. 4) Strengthening community linkage activities and demand creation for RI services remain vital for improved RI uptake and ultimately, vaccine preventable diseases prevention. 5) In view of the magnitude and its importance, detailed analysis of the root cause(s) of data discrepancy with a wider scope is also highly recommended.

 

 

The authors declare no competing interest.

 

 

SAO conceived the idea, was involved in study design, wrote parts of the manuscript and coordinated the entire manuscript writing; MGT was involved in study design, secondary data management, field supervision and wrote part of the manuscript. All authors read and approved the final manuscript.

 

 

We appreciated the efforts of the LGA Facilitator and other field workers who supported data collection as well as the Local Immunisation Officer. Our appreciation also goes to Miss Jubilee Odunola Ajiboye, a mentee from Campbell University, North Carolina, United States of America who also read through the manuscript and made valuable inputs.

 

 

Table 1: data from the different RI data tools by Health Facilities at Bunza LGA in September 2016

Figure 1: bar chart showing data from different data tools in five health facilities and LGA summaries

Figure 2: community survey results for RI at different settlements under selected HFs at Bunza LGA, September 2016

 

 

1. Eboreime E, Abimbola S, Bozzani F. Access to Routine Immunization: A Comparative Analysis of Supply-Side Disparities between Northern and Southern Nigeria. PLoS One. 2015; 10(12): e0144876. PubMed | Google Scholar


2. Health NPHCDAFMo. Major Causes of Poor Routine Immunisation Performance (2013-2015). In National Routine Immunisation Strategic Plan. 2013. Google Scholar


3. Anyene B. Immunisation in Nigeria: The role of politics, religion and cultural practices. African Journal of Health Economics. 2014; 3(1): 9. Google Scholar


4. Dunkle SE, Wallace AS, Macneil A, Mustafa M, Gasasira A, Ali D, Sandhu HS. Limitation of using Administratively Reported Immunisation Data for Monitoring Routine Immunisation Performance in Nigeria. Journal of Infectious Diseases. 2014; 210. Google Scholar


5. Negussie A, Kassahun W, Assegid S, Hagan AK. Factors associated with incomplete childhood immunization in Arbegona district, southern Ethiopia: a case--control study. BMC Public Health. 2016 Jan 12; 16: 27. PubMed | Google Scholar


6. Health NPHCDAF Mo. National Measles Strategic Planning Meeting, Abuja, Nigeria. 2016 November 21-23, 201 Report No. Google Scholar


7. Organisation WH. Immunisation Data Quality Audit (DQA) Procedure. The Department of Immunisation VaBWHO, Geneva, Switzerland; 2003. Google Scholar


8. Analysts FHS. The state of routine immunization services in Nigeria and the reasons for the current problems. 2005. Google Scholar


9. Agency KSPHCD. Kebbi State Routine Immunisation Administrative Data Coverage. 2016. Google Scholar


10. Chahed MK, Bellali H, Alaya NB, Mrabet A, Mahmoudi B. Auditing the quality of immunization data in Tunisia. Asian Pacific Journal of Tropical Disease. 2013 Feb; 3(1): 65-70. PubMed | Google Scholar


11. Mavimbe JC, Braa J, Bjune G. Assessing immunization data quality from routine reports in Mozambique. BMC Public Health. 2005 Oct 11; 5: 108. PubMed | Google Scholar


12. Ronveaux O, Rickert D, Hadler S, Groom H, Lloyd J, Bchir A et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bulletin of the World Health Organization. 2005; 83(7): 503-10. PubMed | Google Scholar


13. Adeiga AO, Audu SA, Sanni RA, Lakehinde SA, GF, Balogun O. Infant Immunisation coverage in difficult-to reach area of Lagos Metropolis. African Journal of Clinical and Experimental Microbiology. 2005; 6: 227-317. Google Scholar


14. Ismail IT, El-Tayeb EM, Omer MD, Eltahir YM, El-Sayed ET, Deribe K. Assessment of Routine Immunization Coverage in Nyala Locality, Reasons behind Incomplete Immunization in South Darfur State, Sudan. Asian J Med Sci. 2014 Feb 25; 6(1): 1-8. PubMed | Google Scholar


15. Agency NPHCD. Strategic Framework, In Nigerian National Routine Immunisation Strategic Plan (2013-2015). Accessed on 01 February 2017