Assessing the incremental costs and savings of introducing electronic immunization registries and stock management systems: evidence from the better immunization data initiative in Tanzania and Zambia

Assessing the incremental costs and savings of introducing electronic immunization registries and stock management systems: evidence from the better immunization data initiative in Tanzania and Zambia

Mercy Mvundura^1,&, Laura Di Giorgio¹, Elisabeth Vodicka², Robert Kindoli³, Chipo Zulu⁴

 

¹Medical Devices and Health Technologies Program, PATH, Seattle, USA, ²Center for Vaccine Innovation and Access, PATH, Seattle, USA, ³Country Program Office, PATH, Dar Es Salaam, Tanzania, ⁴Country Program Office, PATH, Lusaka, Zambia

 

 

^&Corresponding author
Mercy Mvundura, Medical Devices and Health Technologies Program, PATH, Seattle, USA

 

 

 

 

Immunization has proved to be the most cost-effective public health intervention through reducing childhood mortality and morbidity attributable to vaccine-preventable diseases. Despite immunization being such an effective public health tool, not all children are being reached with the lifesaving vaccines they need [1]. One key challenge faced by immunization programs, especially in the sub-Saharan African region, is the stagnation of coverage rates [2]; coverage rates for the third dose of diphtheria, tetanus and pertussis-containing vaccine have plateaued in the 70% percentile since 2010 [3]. In addition, drop out rates between the first and second dose of measles containing vaccine can be high and this has implications for the ability of countries to achieve disease elimination.

 

Several factors have been identified as inhibiting immunization program performance improvement, including the poor quality of data and the poor use of existing data [2, 4-7]. Data quality challenges include inconsistencies and inaccuracies in reported data, which impact key program metrics such as target populations and coverage rates. Poor use of data includes failure to use existing data to inform planning, which can result in low product stock or stockouts and delays in transmission of data to program managers. In addition, programs have challenges tracking which children have received which vaccines and hence during campaigns, vaccines are given to all children in the target age group because there is no data to inform the program about which children are fully vaccinated through routine immunization. Embedded in these data challenges are data formats that make it difficult for health workers to easily identify and track children who are due for vaccinations or track children who move from one area to another, which hinders the provision of optimal services to intended recipients. In addition, low- and middle-income countries are plagued by a shortage of health care workers, who lack the infrastructure to effectively and efficiently manage their programs [8].

 

Given these challenges with immunization program data, there is a global effort to strengthen country immunization systems by supporting the collection of better-quality data and better use of these data to inform program decision-making. One such effort is through the Better Immunization Data (BID) Initiative [9], led by the Ministry of Health, Community Development, Gender, Elderly and Children in Tanzania and the Ministry of Health in Zambia, in partnership with PATH and funded by the Bill & Melinda Gates Foundation. The initiative is designed to shed light on the challenges surrounding data collection, quality, and use and has identified solutions to improving immunization program data - and potentially applying them to other health areas. The BID initiative worked with the governments of Tanzania and Zambia to develop data quality and use solutions, which include a package of interventions that contains an electronic immunization registry with supply chain information, which enables automatic report generation; data use campaigns; online peer support networks and targeted supportive supervision for health workers. These interventions were implemented at the health facility and district levels. Several research studies were conducted to evaluate the impact of the BID initiative, including monitoring and evaluation of the impact of the BID interventions and costing studies. This article focuses on the findings from the costing studies.

 

Very few studies have evaluated the costs of interventions aimed at improving data quality and use in other countries that have implemented similar interventions. Hence, we sought to provide some evidence on these costs using data from Tanzania and Zambia. Our objective was to estimate the economic costs of immunization program logistics and service delivery before and after the implementation of the BID interventions, and use these data to estimate the incremental costs or savings attributable to the interventions. The findings from this study are intended to inform the scale-up of such interventions within the two countries and across other countries in the region.

 

 

Overview of the baseline system and the BID initiative interventions implemented

 

Table 1 provides an overview of the immunization registry before and after implementation of the BID interventions. At baseline, health facilities in Arusha Region, Tanzania, and Southern Province, Zambia, were using paper immunization registers, tally sheets and vaccine stock ledgers. Child health cards were used to document vaccines given and these cards were kept by caregivers. Monthly immunization reports were compiled manually using paper report templates. Through BID, tablets were provided to health facilities, which contain software for an electronic immunization registry that include functionality for immunization registration, tallying, stock management and reporting. Tablets were provided to all health facilities in Southern Province. Initially, the tablets were provided only to high-volume facilities in Arusha Region, while low-volume facilities implemented a simplified paper system that helped to streamline data entry and reporting. However, by the end of the project, low-volume facilities had adopted the electronic system, due to challenges of the simplified paper version. The electronic registry is integrated with data use interventions, including an online peer network platform (WhatsApp) and provision of data use job aids to health workers. District staff also provided targeted supportive supervision for health workers. A barcode/quick response code was added to child health cards so that health workers can scan the barcode to retrieve the vaccination record for any given child from the registry. The electronic registration system also automatically generates the monthly reports on the standard immunization reporting metrics.

 

Facility- and district-level costing

 

We conducted a micro-costing study [10] to estimate the annual economic costs of resources used for immunization logistics and service delivery before and after implementation of the BID initiative in Arusha Region in Tanzania and Southern Province in Zambia. The study focused on the health facilities and districts in which the BID interventions were implemented and hence did not include regions/provinces or the national level.

 

We developed primary data collection tools to identify resources used for transporting and storing vaccines, staff time for logistics and service delivery, office equipment and communications, and printing and office supplies. Similar questionnaires were used to collect data on the resources used at district level, focusing on activities related to the logistics and management of health facilities. The tools used in the two countries were similar, but adaptations were made to reflect country-specific characteristics of each immunization system.

 

We collected data from a sample of health facilities in each district and a sample of districts in each region/province. We included 4 of the 7 districts in Arusha Region and 6 of the 13 districts in Southern Province. Health facility sample sizes are shown in Table 1. Baseline and post-intervention data were collected from the same sample of facilities and districts, which we selected using a purposive sampling approach based on key characteristics expected to affect the costs of providing immunization services. These parameters included average number of monthly immunizations dichotomized into low (< 50 children) and high volume (≥ 50 children), location (rural versus urban) and distance from the district immunization office. BID staff administered the questionnaires through in-person interviews at each facility.

 

At the time of post-intervention data collection, the facilities in Tanzania had been using the electronic immunization registry for an average of 8 months (range 4 to 11 months); in Zambia, the average was 3.5 months (range 1 to 8 months). In addition, at the time of post-intervention data collection, health facilities in both countries were still using the paper-based system as back-up because policy decisions had not yet been made to eliminate the paper system and solely rely on the electronic system. Therefore, we asked health workers to assess the change in resource use under a scenario in which only the electronic system was in use.

 

Types of costs included in the costing study

 

We collected immunization service delivery costs across five main categories: (1) human resources; (2) cold chain equipment; (3) communications, printing and office supplies; (4) facility office equipment and (5) transport. Human resources costs included salaries and per diems for staff working in the immunization program. Staff were asked to self-report the time spent on providing fixed and outreach immunization services, logistics and stock management for the immunization program and data reporting. The costs of cold chain equipment captured the capital costs of refrigerators, freezers, cold boxes, and vaccine carriers used in the immunization program, and the annual costs of electricity or gas to run the cold chain equipment, as relevant. Costs of office equipment and communications included capital costs for computers, tablets, printers, scanners, and other equipment used by the immunization program and communication and printing costs. Finally, transport costs reflected the costs to collect vaccines and immunization supplies from the district and transport them to facilities or to conduct outreach services (hired vehicles, public transportation and capital and fuel costs for vehicles owned and maintained).

 

Data analysis

 

For resources shared with other programs, costs were allocated to the immunization program based on the reported percentage spent or use of the immunization program. Capital costs were annualized using different lifespans: 3 years for office equipment; 5 years for vehicles; and 10 years for cold chain equipment. All local cost data were collected in local currencies and converted into 2017 US dollars using average exchange rates for the year [11]. As necessary, we updated prices for inflation using consumer price indices from the World Bank [12]. Unit prices (Table 2) were obtained from various sources, including local data sources, World Health Organization (WHO) Comprehensive Multi-Year Plans [13], online databases [14-17] and BID Initiative project records.

 

Resource use data were combined with unit costs to calculate economic costs at the district and facility levels. All cost estimates (baseline, post-intervention and incremental) were reported as annual economic costs per facility or district. Our incremental cost estimates relied on the cross-sectional data from the two surveys conducted, one at baseline and one at post-intervention. These surveys were conducted at different time points. However, given that time use was self-reported and subject to recall bias, at post-intervention, along with asking survey respondents to estimate time spent on immunization activities with the BID interventions, we asked them to recall and report how much time they had been spending on these same activities before the implementation of the BID interventions. We used these data to provide an alternative set of estimates for the analysis. In addition, we conducted univariate sensitivity analyses to identify the cost drivers and time use for the interventions.

 

 

Health facility costs

 

Table 3 shows the economic cost estimates for the health facilities, comparing the costs of the resources used using the data from the baseline and post-intervention surveys. In Tanzania, we estimated at baseline that the annual average economic costs per health facility totaled US$17,318 (95% confidence interval [CI]: $12, 113–$24, 289]. Post-intervention average facility costs were $7,082 [95% CI: $4, 506–$10, 116], reflecting estimated annual savings of $10,235 per facility each year due to efficiencies generated in the immunization supply chain, service delivery, and time spent on immunization activities at the facility level (Table 3). These savings are attributable to reductions in several areas, including human resources costs because of reductions in time spent on immunization reporting and management activities and emergency trips for vaccine resupply, and elimination of printing costs for paper registers and tally sheets not required for the BID interventions. The one cost category that increased because of the BID interventions was office equipment, because of the provision of tablets and barcode or quick response code readers at each health facility. Capital costs for cold chain equipment remained unchanged from pre- to post-intervention because the interventions had no impact on these costs. Using the time use data for baseline and post-intervention based on responses from only the post-intervention survey, we found that for the health facilities in Arusha Region, Tanzania, the estimated savings in salary and per diem costs were $6,642, lower compared to the $10,245 reported when using the responses from the baseline and post-intervention surveys.

 

In Zambia, we estimated similar trends as for Tanzania but Zambia’s baseline costs were lower at $5,324 [95% CI: $1,506-$7,209]. Post-intervention costs were estimated at $4,695 for Zambia [95% CI: $2,982-$6,999]; therefore, savings attributable to the BID interventions were smaller (we estimated a savings of approximately $628 per facility each year). This represents a 12% reduction in costs per facility with BID compared to baseline. Similar to Tanzania, the largest savings would be achieved through reduction in staff time on immunization activities. We also estimated a reduction in annual transport costs of $14 per health facility, resulting from a reduction in emergency trips to the district vaccine store to collect vaccines and transportation for immunization-related outreach activities.

 

We conducted univariate sensitivity analyses to assess uncertainty and identify influential parameters on changes in resource utilization due to the interventions (Figure 1). Influential parameters were considered any variables with uncertainty ranges wider than 20% of the total incremental change in health facility costs. Across all Tanzanian facilities, time spent on provision of routine and outreach immunization services and total number of staff allocated to the immunization program (stratified by nurses and other staff) appeared to be the largest drivers of the incremental cost per health facility (Figure 1 A). Sensitivity analyses in Zambia derived similar results (Figure 1 B). Incremental costs appeared to be driven by the time spent on paperwork and providing fixed immunization services, as well as the estimated number of immunization sessions per month.

 

When results were stratified by health facility characteristics -rural versus urban and low volume versus high volume- we found that in Tanzania, there was a smaller variation across health facility categories based on location (results not shown in tables). For example, rural facilities were estimated to save an average of $8,144 [95% CI: $6,812-$9,820] with the introduction of the interventions, compared to $9,423 [95% CI: $5,787-$13,972] savings in urban facilities. The variation was slightly higher based on volume served; we found that low-volume settings were estimated to save approximately $7,683 [95% CI: $5,116-$10,964] per facility compared to $9,367 [95% CI: $7,671-$11,560] in high-volume facilities. In Zambia, the immunization volume categorized as low or high volume resulted in much larger variation between strata, with savings of $275 [95% CI: $205-$298] in low-volume facilities compared to $2,177 [95% CI: $1,408-$3,037] in high-volume facilities. Stratified cost savings were $380 [95% CI: $612 cost savings to $247 increased costs] and $776 [95% CI: $2,218 cost savings to $89 increased costs] in rural and urban facilities, representing 9% and 13% decreases in costs, respectively.

 

District-level costs

 

Table 4 shows the results of the baseline and post-intervention costing analysis at the district level. At baseline, the average logistics and service delivery costs in Tanzania were $23,001 per year, excluding the value of vaccines. Human resources accounted for the largest share of costs, at more than 50%. Based on the responses provided by staff at the district level about the impact of the interventions, we found that the interventions did not have any impact on the costs of cold chain or transport. The BID interventions impacted the costs of the following items: communications, printing and office supplies, office equipment and human resources. The interventions resulted in an average increase in communication costs of $167 per district. Equipment costs increased by about $491 per district because a tablet, printer and scanner were provided to each district immunization office. Other equipment costs were not expected to change with the introduction of the interventions. Human resources costs were the most impacted by the interventions, as district office staff reported a significant reduction in time spent on estimating vaccine needs, processing orders and distributing vaccines to health facilities. A few human resources activities saw an increase in time use, such as supervision and support, due to more time being spent with facilities to provide technical support with the electronic registry. On average, a district office in Tanzania estimated that human resources time valued at $7,200 would be saved per year when compared to the system without the interventions. Overall, we estimated that the rollout of the interventions would result in net savings in the amount of $6,542 per year at the average district office in Arusha Region, Tanzania. This represents a 28% reduction in costs after the introduction of the interventions.

 

The estimated savings in Zambia from the implementation of the interventions were more modest. The new equipment increased district annual costs by $491 and communication costs by $62. Most of the savings were derived from reduced labor time and fewer printing costs for immunization registers, stock ledgers and tally sheets. We estimated that time valued at $789 would be saved per year by staff at each district office. Overall, the rollout of the interventions was estimated to result in annual net savings of $236 for the average district office in Southern Province.

 

 

This study aimed at estimating the cost implications of introducing electronic immunization registries and stock management systems in Arusha Region, Tanzania, and Southern Province, Zambia. In both countries, we found that electronic systems may result in savings compared to paper immunization and stock registers. Savings were mostly attributable to reduction in health workers’ time spent on immunization activities, such as administrative tasks and reporting. Efficiencies gained due to electronic registration and reporting were, as expected, higher in absolute terms in high-volume facilities compared to low-volume facilities.

 

To our knowledge, this study is the first to estimate the savings to be realized using electronic immunization registries and stock management systems in resource-limited settings. Few studies published in the literature evaluated interventions similar to those introduced through the BID Initiative [18-24], most of which were based in high-income countries. Due to the vast differences in financial resources, immunization programming, and health care delivery systems, these findings provide little opportunity for comparison or use in decision-making for immunization programs in sub-Saharan Africa. In terms of benefits, the US-based studies identified increases in administrative efficiency such as the reduction in reporting burden. We found similar efficiencies from reductions in time spent on daily registration during immunization sessions and on monthly reporting. The dearth of studies in low- and middle-income countries highlights a lack of evidence on the costs and benefits of electronic immunization registries and relevant immunization interventions in these countries.

 

While not having a direct financial implication for health ministry budgets, the reduction in health worker time represents an important finding and suggests that human resources could be freed up at health facilities so that staff could spend more time on patient care rather than administrative tasks. Also, given the competing time demands of health workers who work across different programs, the benefits of the time savings could be spread to other programs. At the district level, the time saved from the automatic generation of monthly reports is expected to allow district immunization officers to divert energy to other activities, such as supervision.

 

We found that the estimated savings were much larger for districts and health facilities in Arusha Region, Tanzania, than those in Southern Province, Zambia. While there could be other reasons for these differences, we suspect that the shorter evaluation time in Zambia (between the deployment of the BID initiative interventions and the collection of the post-intervention costing data) may partially explain these findings. In fact, we hypothesize that staff were still adjusting to the new system in Zambia and hence had not gleaned the full benefits of the new system.

 

Our study has several limitations. First, the cost estimates are not representative, as a purposive sampling approach was used and samples were relatively small. To address this limitation, we conducted sensitivity analyses and varied input values over low and high ranges to assess how the cost implications would differ under varying assumptions. Second, the time between rollout of the interventions and post-intervention data collection was short, especially in Zambia. This choice was driven by the project timelines and the broader delays experienced during rollout of the interventions. Therefore, estimates reported here are likely to capture learning costs and thus underestimate savings from the BID interventions. A longer-term evaluation of the BID initiative may be warranted to provide more accurate estimates of its cost implications. Third, all time use data included in this analysis were self-reported and thus may have been under- or overestimated. Also, at the time of post-intervention data collection, countries were continuing to use both their paper-based system and the electronic system, making it difficult to assess system changes. This is because the relevant ministries of health had not yet made the decision to solely rely on the electronic system. As a consequence, we relied on staff’s assessment of what their time use would be if they were using only the electronic system. Fourth, due to the lack of data or complexity in assessing them, the study did not include all benefits that could result from the BID interventions, further underestimating its benefits. For example, potential benefits in terms of a decrease in the number of stockouts and wastage, better forecasting, more timely immunizations and higher coverage, and improved decision-making could not be taken into account because of the shortness of time between deployment and evaluation. In addition, our ability to capture the costs of the immunization program relied on the availability and quality of the data. Finally, in this study we did not include the upfront costs of implementing the BID interventions, such as the system development costs or the costs of rolling out the BID interventions to facilities and districts and the costs of maintaining the system. These costs will be reported in a separate analysis.

 

 

The introduction of electronic immunization registries and stock management systems through the BID initiative was estimated to be cost saving in Tanzania and Zambia. These savings were primarily due to time efficiencies and associated staff cost savings. Information generated through this work provides evidence for key stakeholders in Tanzania and Zambia to inform decision-making for the scale-up of the BID interventions in these countries and to inform decisions in other countries that may be interested in similar interventions.

 

 

The authors declare no competing interests.

 

 

Mercy Mvundura (MM) and Laura Di Giorgio (LDG) were responsible for the conception and design of the work; Robert Kindoli (RK) and Chipo Zulu (CZ) were involved in the acquisition of data for the work; MM, LDG and Elisabeth Vodicka (EV) were responsible for data analysis; MM and EV were responsible for drafting the manuscript; all authors (MM, LDG, EV, RK and CZ) reviewed and provided important feedback on the various iterations of the manuscript and all authors provided approval of the final version submitted to this journal.

 

 

The authors gratefully acknowledge the contributions of the health workers and Expanded Program on Immunization staff who were willing to answer the study questions and provide data to inform this research. They also gratefully acknowledge the support of the ministries of health in Tanzania and Zambia for allowing them to engage in the costing data collection.

 

 

Table 1: sample size for the health facility analysis and overview of the immunization registry system before and with the BID interventions

Table 2: selected unit prices (in 2017 US dollars)

Table 3: average annual cost per facility for the immunization program at baseline and post-intervention (in 2017 US dollars)

Table 4: average annual cost per district for the immunization program at baseline and post-intervention (in 2017 US dollars)

Figure 1: univariate sensitivity analysis evaluating influential parameters on incremental savings from introducing electronic immunization registries and stock management systems into Tanzania and Zambia

 

 

1. Arevshatian L, Clements CJ, Lwanga SK, Misore AO, Ndumbe P, Seward JF et al. An evaluation of infant immunization in Africa: is a transformation in progress?. Bull World Health Organ. 2007;85(6):449-57. PubMed | Google Scholar


2. Strategic Advisory Group of Experts on Immunization. 2014 Assesment report of the Global Vaccine Action Plan. Geneva: World Health Organization. Accessed on January 22, 2018


3. Feldstein LR, Mariat S, Gacic-Dobo M, Diallo MS, Conklin LM, Wallace AS. Global routine vaccination coverage, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(45):1252-1255. PubMed | Google Scholar


4. Machingaidze S, Wiysonge CS, Hussey GD. Strengthening the Expanded Programme on Immunization in Africa: looking beyond 2015. PLoS Med. 2013;10(3):e1001405. PubMed | Google Scholar


5. Shen AK, Fields R, McQuestion M. The future of routine immunization in the developing world: challenges and opportunities. Glob Heal Sci Pract. 2014;2(4):381-394. PubMed | Google Scholar


6. Ronveaux O, Rickert D, Hadler S. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull World Health Organ. 2005;83(7):503-510. PubMed | Google Scholar


7. Bosch-Capblanch X, Ronveaux O, Doyle V, Remedios V, Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Trop Med Int Heal. 2009;14(1):2-10. PubMed | Google Scholar


8. Ranson MK, Chopra M, Atkins S, Dal Poz MR, Bennett S. Priorities for research into human resources for health in low- and middle-income countries. Bull World Health Organ. 2010;88(6):435-43. PubMed | Google Scholar


9. Path. Product Vision for the Better Immunization Data (BID) Initiative. 2014. Accessed on February 5, 2018.


10. Frick KD. Microcosting quantity data collection methods. Med Care. 2009;47:S76-S81. PubMed | Google Scholar


11. OANDA. Average exchange rates. Accessed on February 12, 2018.


12. The World Bank. Inflation, consumer prices (annual %). Published 2016. Accessed on January 22, 2018.


13. World Health Organization.Country information: Status of National Comprehensive Multi-Year Plans (cMYPs). Published 2016. Accessed on January 22, 2018.


14. PATH. Cold Chain Equipment Manager. Published 2012. Accessed January 22, 2018.


15. Global Petrol Prices. Retail energy price data. Accessed on January 22, 2018.


16. Statista. Average retail electricity prices in Africa in 2016, by select country (in U.S. dollars per megawatt hour). Accessed on July 1, 2018.


17. The World Bank. Fact Sheet: the World Bank and Africa.


18. Rask KJ, Wells KJ, Kohler SA, Rust CT, Cangialose CB. Measuring immunization registry costs: promises and pitfalls. Am J Prev Med. 2000;18(3):262-267. PubMed | Google Scholar


19. Slifkin RT, Freeman VA, Biddle AK. Costs of developing childhood immunization registries: case studies from four All Kids Count projects. J public Heal Manag Pract JPHMP. 1999;5(5):67-81. PubMed | Google Scholar


20. Fontanesi JM, Flesher DS, De Guire M, Lieberthal A, Holcomb K. The cost of doing business: cost structure of electronic immunization registries. Health Serv Res. 2002;37(5):1291-1307. PubMed | Google Scholar


21. Horne PR, Saarlas KN, Hinman AR. Costs of immunization registries: experiences from the All Kids Count II projects. Am J Prev Med. 2000;19(2):94-98. PubMed | Google Scholar


22. McKenna VB, Sager A, Gunn JE, Tormey P, Barry MA. Immunization registries: costs and savings. Public Heal Reports (Washington, DC 1974). 2002;117(4):386-392. PubMed | Google Scholar


23. Bartlett DL, Molinari N-AM, Ortega-Sanchez IR, Urquhart GA. Economics of immunization information systems in the United States: assessing costs and efficiency. Cost Eff Resour Alloc. 2006;4:15. PubMed | Google Scholar


24. Patel M, Pabst L, Chattopadhyay S, Hopkins D, Groom H, Myerburg S et al. Economic review of immunization information systems to increase vaccination rates: a community guide systematic review. J Public Heal Manag Pract. 2015;21(3):253-262. PubMed | Google Scholar