Validation of verbal autopsy methods for assessment of child mortality in sub-Saharan Africa and the policy implication: a rapid review

Validation of verbal autopsy methods for assessment of child mortality in sub-Saharan Africa and the policy implication: a rapid review

Chigozie Jesse Uneke^1,&, Henry Chukwuemeka Uro-Chukwu¹, Onyedikachi Echefu Chukwu¹

 

¹African Institute for Health Policy and Health Systems, Ebonyi State University, PMB 053 Abakaliki, Nigeria

 

 

^&Corresponding author
Chigozie Jesse Uneke, African Institute for Health Policy and Health Systems, Ebonyi State University, PMB 053 Abakaliki, Nigeria

 

 

 

 

Within the past few decades there has been a substantial progress in the reduction of child mortality globally. A 2017 report from the United Nations Inter-agency Group for Child Mortality Estimation (IGCME) indicated that, global under-five mortality rate dropped to 41 deaths per 1,000 live births in 2016 from 93 in 1990, which is a notable 56% decline [1]. However, disparities still exist in the global trend of child mortality reduction across different regions of the world. Recent statistics from the World Health Organization (WHO) Global Health Observatory Data Repository showed that in sub-Saharan Africa, infant, neonatal and under-five mortality rates per 1000 live births reduced from 93.5, 40.9 and 154 in 2000 to 53.9, 27.7 and 79.5 in 2015 respectively [2]. Despite these appreciable reductions, the child mortality rate in sub-Saharan Africa remains unacceptably high. The sub-region, is still the home to all six countries (Central African Republic, Chad, Mali, Nigeria, Sierra Leone and Somalia) with an under-five mortality rate above 100 deaths per 1,000 live births [1, 2]. Furthermore, the probability that a child, aged 5 dies before reaching his or her fifteenth birthday in sub-Saharan Africa is 17 times higher than the average in high-income countries [1].

 

Clearly, sub-Sharan Africa requires accelerated progress if the region will achieve the Sustainable Development Goal (SDG) targets in child survival. According to WHO, reliable data on the levels and causes of child mortality are cornerstones for building a solid evidence base for health policy, planning, implementation, budgeting, resource allocation, monitoring and evaluation, which are core prerequisites to improving child health outcomes [3]. Unfortunately, accurate, reliable and timely available data on Cause of Death (COD) among children is essentially lacking in most of sub-Saharan Africa. One major reason for this drawback in the sub region as in other Low and Middle-Income Countries (LMICs) is the widespread near absence of well-established and functional Civil Registration and Vital Statistics (CRVS) systems [4]. All across the sub-Saharan African countries especially in the rural and underserved areas and even in urban areas where the CRVS systems do not function, many of the child deaths and the COD are not recorded. This is largely because the majority of deaths occur at home and there is little chance that deaths occurring away from health facilities will be recorded and the cause of death certified [3, 5]. As a partial solution to this problem, Verbal Autopsy (VA) has gained global prominence as a primary strategy for obtaining information about COD in populations where CRVS and medical certification are absent [3, 6-10]. The WHO defined VA as a method used to determine the COD based on an interview with next of kin or other caregivers of the deceased [3]. According to Desai and colleagues [7], VA tools typically consist of a structured survey administered to a close relative or associate of the deceased by a trained field worker, to record the signs and symptoms that occurred before death and this information is used to assign the most probable COD, most often via Physician-Certified Verbal Autopsy (PCVA) coding.

 

The WHO is promoting the application of VA especially in regions such as the sub-Saharan Africa where the CRVS systems and medical certification are weak [2, 10]. This is because the increased use of VA for routine application in national health information systems has been reported by Murray and co-workers [6], to have the potential to greatly improve the availability of reliable and essential information on causes of death for disease control programs. The WHO noted in a recent publication on VA standards, that there has been a proliferation of interest, research and development in all aspects of the VA process, including VA data-collection systems, VA questionnaires content and format, cause of death assignment process, coding and tabulation of causes of death, and validation of VA instruments [3]. Consequently, there exist various methods of VA application [11] and many more methods are being developed. Some of the methods which have been reported in literature include: the PCVA [12], artificial neural networks model [13], data-derived algorithms [14], probabilistic approach [15], InterVA model [16], symptom pattern method [17], King Lu method [18], robust metrics model [19], cause-specific mortality fractions model [20], tariff method [21] and random forests model [22]. Despite the value of VA as a potential alternative for COD data for child mortality for resource constrained setting like the sub-Saharan Africa, there are a number of fundamental issues associated with its accuracy, reliability and routine applicability.

 

First, Murray and colleagues argued that there are widespread concerns about VA in terms of the dependability of symptom information collected from families and the practicality of relying on physicians to review anonymous symptom-based questionnaires [6]. They further noted that confidence in VA as a legitimate data collection mechanism is limited because it is not known how accurately the method can diagnose the underlying cause of death compared with hospital-based procedures or how different approaches to VA perform in assigning causes of death [6]. Second, Menendez and co-workers noted that verbal autopsies have been questioned because they are subject to a high degree of misclassification errors, especially for conditions with poor diagnostic specificity such as most causes of peri-neonatal deaths [23]. Third, Byass and colleagues observed that there has been substantial methodological heterogeneity involved in VA, which can magnify existing uncertainties over cause-specific mortality [24]. Fourth, Leitao and co-workers noted that the expanding use of VA in generating mortality data has led to a proliferation of different VA instruments (comprising a set of questions/indicators that elicit pertinent information on signs, symptoms and circumstances preceding death and a corresponding list of COD that has impaired data comparability across sites and over time [5]. Another major challenge with the use of VA in resource scarce setting has to do with concerns about its validity. VA validation has been defined as a process that compares an Underlying Cause of Death (UCOD) derived from the VA with a reference UCOD for the same death whereby the reference UCOD can be derived from pathological autopsy which is considered as a “gold standard”, or from clinical records which are considered as the next best alternative [25, 26]. Due to the importance of the validation of VA, a number of studies have been conducted in sub-Saharan Africa on the subject with variations in the findings of the investigations. Thomas and colleagues [4] advised that the findings from such studies need careful consideration, while Tran and co-workers [25] noted that establishing the validity of VA methods is important when considering to use these methods on a routine basis to improve the availability and quality of mortality data in the country. The objective of this paper is to review studies that investigated the validation of verbal autopsy methods for assessment of child mortality in sub-Saharan Africa with the view to providing scientific evidence relevant to policy making as it affects COD and child survival in sub-Saharan Africa.

 

 

We conducted a MEDLINE PubMed search of literature in June 2018 and sought studies published in English that investigated the validation of VA methods in sub-Saharan Africa within the last three decades (1990-2018). We used the following keywords for the search: “verbal autopsy”, validation, child, mortality, Africa; these resulted to 17 publications. The identified publications were subjected to study inclusion criteria which included: (i) study must be conducted in any of the countries in sub-Saharan Africa; (ii) the study must not be earlier than 1990; (iii) the study must be based on the validation of one or more methods of VA; (iv) the study must be focused on children, infants and newborn; (v) the study must be a primary study either prospective or retrospective. One of the authors (CJU) reviewed the 17 publications identified and extracted data and other relevant information from those that fulfilled the study inclusion criteria. The other two authors (HCU and OEC) subsequently performed independent review of the identified publications and selected those that are in line with the study inclusion criteria from which data was extracted. Thereafter all the authors discussed and harmonized selected publications and arrived at a consensus regarding included studies. A total of nine out of the 17 identified publications fulfilled the study criteria and were included in the review. The reference sections of all the included studies were reviewed for any additional relevant study and other information. This process yielded additional five studies which satisfied the study inclusion criteria and making the included study a total of 14. The included publications were thereafter categorized into three groups (1990-1999; 2000-2009; 2010-2018) in Table 1, Table 2 and Table 3 according to the following information: author/year of publication; country; scope of validation; study design; target population; evidence generated; and policy implication.

 

 

Physician-certified verbal autopsy most widely used verbal autopsy method

 

The result of this review clearly showed that PCVA is the most widely used VA method to determine child mortality COD in sub-Saharan Africa. This explains why all the studies [27-35, 36-40] with the exception of one [36] focused on the validation of the PCVA. Although PCVA has proven to be a very useful strategy to determine COD for children, infants and neonates in many parts of sub-Saharan Africa where there is no other better alternative, the limitations of the tool are well-established. Among the major drawbacks of the PCVA as noted in numerous reports from sub-Saharan Africa and other resource poor regions of the world is that, it is very expensive and often inefficient because it is time consuming [11, 12, 41]. According to Leitao and co-workers [42], the PCVA typically involves at least two physicians examining each record, with adjudication done by a consensus review or by a third physician. This may constitute a significant challenge in situations where there are numerous and often voluminous VA forms collected from a large population. Lozano and colleagues [12], noted that the usefulness of PCVA is based on the premise that physicians can interpret reported signs and symptoms correctly and occasionally household recall of Healthcare Experience (HCE) to accurately assign causes of death. This is not an easy venture because apart from the uncertainty associated with obtaining reliable household recall of HCE, the interpretation of the reported signs and symptoms highly depends on the skill and competence of the physician.

 

Available reports have indicated that operationalization differs by the number of physicians reading each VA, the methods used to adjudicate when different physicians disagree, and the procedures to map International Classification of Diseases (ICD) codes to the physician-assigned underlying cause of death [12, 43, 44]. Despite the fact that the use of PCVA is fraught with these drawbacks, a number of studies have shown that it still remains the only reliable method for determining COD from VA questionnaire data in locations where CRVS systems are lacking or dysfunctional [24, 31, 42]. Writing from experiences in the Nouna health and demographic surveillance site of Burkina Faso, Yé and colleagues [41] argued that the PCVA deficiencies should not prevent countries requiring information on causes of death from benefiting from the use of VA when no practical alternative for obtaining these data exists. However, Lozano and colleagues [12], advised that more rigorous standardization of questionnaire implementation, tests of diagnostic skill, and training might be able to improve concordance and perhaps increase Cause Specific Mortality Fraction (CSMF) accuracy.

 

Validation of Physician-certified verbal autopsy with hospital records

 

As noted previously, the wider use of PCVA has raised a lot of concern regarding its validity. Consequently, a number of PCVA validation studies have been conducted in different LMICs to ascertain the extent of reliability of the method whereby the COD for individuals derived from PCVA is compared to the COD obtained from hospital records [19, 24, 25] (Table 1, Table 2 and Table 3). In the present VA validation review of child mortality in sub-Saharan Africa, a total of eight studies [31-33, 35, 37-40] out of the 14 selected studies focused on the comparison of PCVA with hospital records. Seven (87.5%) of the eight studies that compared the PCVA with hospital records were earlier studies conducted between 1990 and 2010. In Uganda, Mpimbaza and colleagues [31] compared PCVA with hospital records and noted that the sensitivity of PCVA in determining malaria death was 61% in Tororo and 50% in Kampala, its specificity was >88% in both locations, while the Positive Predictive Value (PPV) varied widely between the two locations; 83% and 34% respectively. In Ghana, PCVA performed poorly in stillbirth diagnosis and for neonatal deaths, its sensitivity was >60% for major causes of death while its specificity was 76% for birth asphyxia but >85% for prematurity and infection [32]. In Tanzania PCVA underestimated stillbirth and overestimated malaria death in post newborn [33]. Khan and colleagues [35] noted from their study in South Africa that PCVA had sensitivity, specificity and PPV of 69%, 96% and 90% respectively for communicable diseases whereas for non-communicable diseases the values were 75%, 91% and 86% respectively. The respective sensitivities and specificities for the five causes of death studied in Namibia by Mobley and co-workers were as follows: malnutrition 73% and 76%, cerebral malaria 72% and 85%, malaria 45% and 87%, diarrhea 89% and 61%, cough with dyspnea 72% and 64%, measles 71% and 85% [37]. In Kenya, PCVA detected the common cause of death with specificities >80%, but its sensitivity for measles, neonatal tetanus, malnutrition trauma-related deaths was >75%, while it was <50% for malaria, anemia, acute respiratory tract infection, gastroenteritis and meningitis.

 

From the forgoing, validation studies comparing PCVA to hospital records on child mortality in sub-Saharan Africa clearly demonstrate mixed and highly varied outcomes. This is consistent with results obtained from similar validity investigations conducted in other parts of the world in which the fraction of deaths where the true cause is accurately predicted varied from 0% to 95% for different causes [12]. Another major factor that compounds validation studies comparing PCVA to hospital records is that in most low-income settings the underlying quality of clinical diagnosis by physicians, hospital record keeping, laboratory and pathological services and medical imaging can be extremely poor [4, 11, 12]. Murray and colleagues have therefore described many of the validation studies as actually comparisons of two imperfect cause of death assignment approaches: low-quality hospital-assigned cause of death and the verbal autopsy [11]. Nevertheless, the validation of PCVA using hospital records should be viewed as a context-specific issue of which the accuracy and reliability depend on the level of healthcare the respondents have access to and the competence and knowledge of the physicians on the local disease aetiology and epidemiology [12, 29, 31]. Until newer, accurate, cheaper and more feasible methods of determining COD among children is developed, the sub-Saharan Africa will continue to rely on PCVA for establishing COD from VA questionnaire data, since most countries in the region are yet to make their CRVS very functional.

 

Validation of physician-certified verbal autopsy with expert algorithm and InterVA

 

In recent years, efforts have been made to develop novel strategies for the interpretation of VA data that will serve as alternative to or replacement of PCVA for use in resource poor settings. Within the last decade a number of Computerized Coding of Verbal Autopsy (CCVA) methods for interpreting VA data have been investigated and validated and these include the algorithmic and InterVA methods [5, 14, 16, 21]. There are two types of algorithms, the expert-derived and the data-derived. According to Fottrell and Byass [45], expert algorithms consist of a set of predefined diagnostic criteria outlined by a panel of physicians, based on their experience or review of relevant literature, while data-driven algorithms are derived from existing data using standard statistical techniques. In the present review, a total of seven [16, 27-30, 34, 36] studies investigated the validation of VA by comparing either PCVA and algorithm or PCVA and InterVA (Table 1, Table 2 and Table 3).

 

In Nigeria, Adewemimo et al. [27] reported the outcome of the assessment of level of agreement between the Expert Algorithm Verbal Autopsy (EAVA) and PCVA and noted that both methods are very useful in providing direct estimates of the causes and determinants of neonatal and 1-59 month mortality. In Uganda, Mpimbaza and colleagues [28] compared the accuracy of PCVA, non-hierarchical and hierarchical algorithm in relation to the hospital diagnosis and observed high intra-rater reliability (83% agreement), the PCVA had high specificity for neonatal (>67%) and child death (>83%) while the hierarchical was better than the non-hierarchical algorithm. Comparing PCVA with expert algorithm in Niger, Kalter et al. [29], noted that cause distributions of neonatal deaths as determined by expert algorithms and the PCVA were similar, with the same top three causes by both methods and all but two other causes within one rank of each other. In Kenya, the COD was assessed by expert and data-derived algorithms and both were compared with hospital diagnosis, result showed that data-derived diagnostic algorithm gave comparable or better diagnostic accuracy than expert algorithm, but both gave high specificity [36]. These findings suggest that the algorithmic method performed well and could be described as an alternative to PCVA and its possible replacement. However, this outcome should be considered with some caution because a number of studies on expert opinion algorithm and data derived algorithm have shown conflicting levels of accuracy and sometimes even lower than that of PCVA [28, 31].

 

The use of the InterVA model to interpret VA data is increasingly gaining recognition and has been studied and validated in different resource scarce settings with encouraging outcomes [8, 16, 34]. In Kenya, PCVA and InterVA-4 were validated against hospital diagnosis of Sickle Cell Disease (SCD) among children aged 28 days to 14 years and result showed that 6% and 5% of deaths were attributed to SCD using PCVA and InterVA respectively [30]. Furthermore, agreement coefficient for SCD between PCVA and hospital diagnosis was 95.5% while that for InterVA was 96.9% [30]. In Ethiopia, the InterVA model was observed to be much less labour intensive than PCVA in determining COD among children and infants and offered 100% consistency [34]. The InterVA method is derived from Bayes' probability theorem and has been shown to be capable of achieving maximum consistency in interpreting VA data, requires relatively minimal time and labor resources, and freely available in the internet [16, 34, 46]. As with the algorithm method, InterVA has also been proposed as an alternative and possible replacement of the PCVA [15, 47-49]. However, Oti and Kyobutungi [16] argued that there is need to refine the InterVA model, continue its validation with more extensive data from different settings and give further thought to the interpretation and analysis of multiple causes of death for individual cases.

 

Policy implication of VA validation

 

The need for sub-Saharan Africa to accelerate progress towards achieving the SDGs targets in child survival cannot be overstated. This is very important because if current trends continue, more than three quarters of all countries in sub-Saharan Africa will miss the under-five mortality target, and 13 countries in the region will not reach the target until after 2050 [1]. It is imperative therefore for the health systems on which child health interventions are deployed to be strengthened in the sub-region [50]. Accurate and reliable information on COD among children will continue to serve as one of the bedrocks for planning and health systems strengthening for improved child health outcomes in sub-Saharan Africa. As the countries in sub-Saharan Africa continues to battle with dysfunctional CRVS system, VA will remain the only viable option for the supply of child mortality data necessary for policy making [51].

 

Another important reason why VA will not only remain viable but also acceptable in sub-Saharan Africa is because of the myriad of concerns associated with incisional autopsy. Available reports from sub-Saharan Africa indicate that majority of the people will not normally give consent to performing incisional autopsy on their dead relatives. Some of the main reasons included: administrative challenges in requesting autopsy, delay in obtaining autopsy report, relatives of the deceased citing fear of mutilation, permission required from other family members, concerns about delaying the funeral, objection by the patient before death, and objection based on religious and cultural beliefs [52-55]. According to Oluwasola et al. [52], socio-demographic factors such as age, occupation, religion, ethnicity, and level of education significantly influenced willingness to give consent for autopsy. Because VA is generally the more acceptable autopsy in sub-Saharan Africa, the need for its standardization cannot be over emphasized.

 

The WHO is actively promoting the standardization of the VA tools to permit certification and coding commensurate with International Statistical Classification of Diseases and Related Health Problems, 10^th Revision (ICD-10) and ascertain all causes of death with reasonable accuracy drawing on well-administered VA interviews [3]. Since WHO published the first three standard VA questionnaires in 2007, the tools have been revised and updated thrice (in 2012, 2014 and 2016) to improve cause-specific mortality data and to ensure consistency and comparability between countries [56]. However, because communities, healthcare planners, managers, researchers, global decision-makers and donors will continue to require VA data to inform burden of disease estimation and program evaluation, implies that COD estimates must meet high accuracy standards and be comparable over time and across countries [51].

 

 

The outcome of this review clearly showed that none of the VA methods are a one-size-fits-all approach and like Leitao et al. [42] reported, none of the methods consistently outperformed the others across selected CODs, for both individuals and population-level COD assignment. There is a dire need to enhance the usefulness of VA data in facilitating evidence-informed policymaking and implementation process that will help to improve child health outcomes in sub-Saharan Africa. To achieve this, we subscribe to the suggestion of Thomas and colleagues [4] who noted that active collaboration of various stakeholders: governments, health authorities, communities and research groups, is highly required because this can foster engagement in, and coverage of, data collection and enhance the validity and utility of the process. This review was not without limitations. First, our use of only PubMed and the inability to use other databases for our search may have limited the number of potential publications for this review. However, we have confidence in the quality of publications from PubMed because it is a globally acclaimed data base for health research. Second, the variation in study design and reference standards employed in the various studies reviewed along with the differences in context of study areas could not permit adequate comparison across studies in order to make a generalized conclusion regarding the use of the VA methods to determine child mortality. More research is clearly needed to develop VA methods that are accurate and affordable for use in resource scarce setting like sub-Saharan Africa.

 

 

The authors declare no competing interests.

 

 

All authors participated in the design and development of the study. CJU drafted the manuscript, all other authors made inputs to the final manuscript.

 

 

This study was supported by Internal Research Project Fund 2018 of the African Institute for Health Policy & Health Systems, Ebonyi State University Abakaliki Nigeria.

 

 

Table 1: profile of scientific publications in 2010-2018 that investigated verbal autopsy validation in sub-Saharan Africa

Table 2: profile of scientific publications in 2000-2009 that investigated verbal autopsy validation in sub-Saharan Africa

Table 3: profile of scientific publications in 1990-1999 that investigated verbal autopsy validation in sub-Saharan Africa

 

 

1. UNICEF, WHO, World Bank, UN-DESA Population Division. Levels and trends in child mortality: estimates developed by the UN Inter-agency Group for Child Mortality Estimation (2017 Report). Accessed 18 June 2018.


2. World Health Organization. Global Health Observatory data repository: probability of dying per 1000 live births-data by WHO region (2016) . Accessed 18 June 2018.


3. World Health Organization. Verbal autopsy standards: the 2016 WHO verbal autopsy instrument. Geneva: World Health Organization. 2016.


4. Thomas LM, D'Ambruoso L, Balabanova D. Verbal autopsy in health policy and systems: a literature review. BMJ Glob Health. 2018;3(2):e000639. PubMed | Google Scholar


5. Leitao J, Chandramohan D, Byass P, Jakob R, Bundhamcharoen K, Choprapawon C et al. Revising the WHO verbal autopsy instrument to facilitate routine cause-of-death monitoring. Glob Health Action. 2013 Sep 13;6:21518. PubMed | Google Scholar


6. Murray CJ, Lozano R, Flaxman AD, Serina P, Phillips D, Stewart A et al. Using verbal autopsy to measure causes of death: the comparative performance of existing methods. BMC Med. 2014 Jan 9;12:5. PubMed | Google Scholar


7. Desai N, Aleksandrowicz L, Miasnikof P, Lu Y, Leitao J, Byass P et al. Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries. BMC Med. 2014 Feb 4;12:20 PubMed | Google Scholar


8. Nichols EK, Byass P, Chandramohan D, Clark SJ, Flaxman AD, Jakob R et al. WHO Verbal Autopsy Working Group,the WHO 2016 verbal autopsy instrument: An international standard suitable for automated analysis by InterVA, InSilicoVA, and Tariff 20. PLoS Med. 2018;15(1):e1002486. PubMed | Google Scholar


9. Joubert J, Bradshaw D, Kabudula C, Rao C, Kahn K, Mee P et al. Record-linkage comparison of verbal autopsy and routine civil registration death certification in rural north-east South Africa: 2006-09. Int J Epidemiol. 2014;43(6):1945-1958. PubMed | Google Scholar


10. de Savigny D, Riley I, Chandramohan D, Odhiambo F, Nichols E, Notzon S et al. Integrating community-based verbal autopsy into civil registration and vital statistics (CRVS): system-level considerations. Glob Health Action. 2017;10(1):1272882. PubMed | Google Scholar


11. Murray CJ, Lopez AD, Black R, Ahuja R, Ali SM, Baqui A et al. Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets. Popul Health Metr. 2011;9:27. PubMed | Google Scholar


12. Lozano R, Lopez AD, Atkinson C, Naghavi M, Flaxman AD, Murray CJL et al. Performance of physician-certified verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:32. PubMed | Google Scholar


13. Boulle A, Chandramohan D, Weller P. A case study of using artificial neural networks for classifying cause of death from verbal autopsy. Int J Epidemiol. 2001 Jun;30(3):515-20. PubMed | Google Scholar


14. Quigley MA, Chandramohan D, Setel P, Binka F, Rodrigues LC. Validity of data-derived algorithms for ascertaining causes of adult death in two African sites using verbal autopsy. Trop Med Int Health. 2000 Jan;5(1):33-9. PubMed | Google Scholar


15. Byass P, Huong DL, Minh HV. A probabilistic approach to interpreting verbal autopsies: methodology and preliminary validation in Vietnam. Scand J Public Health Suppl. 2003;62:32-7. PubMed | Google Scholar


16. Oti SO, Kyobutungi C. Verbal autopsy interpretation: a comparative analysis of the InterVA model versus physician review in determining causes of death in the Nairobi DSS. Popul Health Metr. 2010 Jun 29;8:21. PubMed | Google Scholar


17. Murray CJL, Lopez AD, Feehan DM, Peter ST, Yang G. Validation of the Symptom Pattern Method for Analyzing Verbal Autopsy Data. PLoS Med. 2007 Nov 20;4(11):e327. PubMed | Google Scholar


18. King G, Lu Y. Verbal Autopsy Methods with Multiple Causes of Death. Statistical Science. 2008;23(1):78-91. Google Scholar


19. Murray CJL, Lozano R, Flaxman AD, Vahdatpour A, Lopez AD. Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies. Popul Health Metr. 2011 Aug 4;9:28. PubMed | Google Scholar


20. Flaxman AD, Vahdatpour A, James SL, Birnbaum JK, Murray CJL, The Population Health Metrics Research Consortium (PHMRC). Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011 Aug 4;9:35. PubMed | Google Scholar


21. James SL, Flaxman AD, Murray CJL, The Population Health Metrics Research Consortium (PHMRC). Performance of the Tariff Method: validation of a simple additive algorithm for analysis of verbal autopsies. Popul Health Metr. 2011 Aug 4;9:31. PubMed | Google Scholar


22. Flaxman AD, Vahdatpour A, Green S, James SL, Murray CJL, The Population Health Metrics Research Consortium (PHMRC). Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2 2011 Aug 4;9:29. PubMed | Google Scholar


23. Menendez C, Castillo P, Martínez MJ, Jordao D, Lovane L, Ismail MR et al. Validity of a minimally invasive autopsy for cause of death determination in stillborn babies and neonates in Mozambique: An observational study. PLoS Med. 2017 Jun 20;14(6):e1002318. PubMed | Google Scholar


24. Byass P, Herbst K, Fottrell E, Ali MM, Odhiambo F, Amek N et al. Comparing verbal autopsy cause of death findings as determined by physician coding and probabilistic modelling: a public health analysis of 54000 deaths in Africa and Asia. J Glob Health. 2015;5(1):010402. PubMed | Google Scholar


25. Tran HT, Nguyen HP, Walker SM, Hill PS, Rao C. Validation of verbal autopsy methods using hospital medical records: a case study in Vietnam. BMC Med Res Methodol. 2018;18(1):43. PubMed | Google Scholar


26. Yang G, Rao C, Ma J, Wang L, Wan X, Dubrovsky G, Lopez AD. Validation of verbal autopsy procedures for adult deaths in China. Int J Epidemiol. 2006;35(3):741-8. PubMed | Google Scholar


27. Adewemimo A, Kalter HD, Perin J, Koffi AK, Quinley J, Black RE. Direct estimates of cause-specific mortality fractions and rates of under-five deaths in the northern and southern regions of Nigeria by verbal autopsy interview. PLoS One. 2017;12(5):e0178129. PubMed | Google Scholar


28. Mpimbaza A, Filler S, Katureebe A, Quick L, Chandramohan D, Staedke SG. Verbal Autopsy: Evaluation of Methods to Certify Causes of Death in Uganda. PLoS One. 2015;10(6):e0128801. PubMed | Google Scholar


29. Kalter HD, Roubanatou AM, Koffi A, Black RE. Direct estimates of national neonatal and child cause-specific mortality proportions in Niger by expert algorithm and physician-coded analysis of verbal autopsy interviews. J Glob Health. 2015;5(1):010415. PubMed | Google Scholar


30. Ndila C, Bauni E, Nyirongo V, Mochamah G, Makazi A, Kosgei P et al. Verbal autopsy as a tool for identifying children dying of sickle cell disease: a validation study conducted in Kilifi district, Kenya. BMC Med. 2014;12:65. PubMed | Google Scholar


31. Mpimbaza A, Filler S, Katureebe A, Kinara SO, Nzabandora E, Quick L et al. Validity of verbal autopsy procedures for determining malaria deaths in different epidemiological settings in Uganda. PLoS One. 2011;6(10):e26892. PubMed | Google Scholar


32. Edmond KM, Quigley MA, Zandoh C, Danso S, Hurt C, Owusu Agyei S et al. Diagnostic accuracy of verbal autopsies in ascertaining the causes of stillbirths and neonatal deaths in rural Ghana.Paediatr Perinat Epidemiol. 2008;22(5):417-29. PubMed | Google Scholar


33. Setel PW, Whiting DR, Hemed Y, Chandramohan D, Wolfson LJ, Alberti KG et al. Validity of verbal autopsy procedures for determining cause of death in Tanzania. Trop Med Int Health. 2006;11(5):681-96. PubMed | Google Scholar


34. Fantahun M, Fottrell E, Berhane Y, Wall S, Högberg U, Byass P. Assessing a new approach to verbal autopsy interpretation in a ruralEthiopian community: the InterVA model. Bull World Health Organ. 2006;84(3):204-10. PubMed | Google Scholar


35. Kahn K, Tollman SM, Garenne M, Gear JS. Validation and application of verbal autopsies in a rural area of South Africa.Trop Med Int Health. 2000;5(11):824-31. PubMed | Google Scholar


36. Quigley MA, Armstrong Schellenberg JR, Snow RW. Algorithms for verbal autopsies: validation study in Kenyan children. Bull World Health Organ. 1996;74(2):147-54. PubMed | Google Scholar


37. Mobley CC, Boerma JT, Titus S, Lohrke B, Shangula K, Black RE. Validation study of a verbal autopsy method for causes of childhood mortality in Namibia. J Trop Pediatr. 1996;42(6):365-9. PubMed | Google Scholar


38. Nykanen M, Tamaona W, Cullinan T, Van Oosterzee V, Ashorn P. Verbal autopsy as a technique to establish causes of infant and child mortality. East Afr Med J. 1995;72(11):731-4. PubMed | Google Scholar


39. Todd JE, De Francisco A, O'Dempsey TJ, Greenwood BM. The limitations of verbal autopsy in a malaria-endemic region. Ann Trop Paediatr. 1994;14(1):31-6. PubMed | Google Scholar


40. Snow RW, Armstrong JR, Forster D, Winstanley MT, Marsh VM, Newton CR et al. Childhood deaths in Africa: uses and limitations of verbal autopsies. Lancet. 1992;340(8815):351-5. PubMed | Google Scholar


41. Yé M, Diboulo E, Niamba L, Sié A, Coulibaly B, Bagagnan C et al. An improved method for physician-certified verbal autopsy reduces the rate of discrepancy: experiences in the Nouna Health and Demographic Surveillance Site (NHDSS), Burkina Faso. Popul Health Metr. 2011;9:34. PubMed | Google Scholar


42. Leitao J, Desai N, Aleksandrowicz L, Byass P, Miasnikof P, Tollman S et al. Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review. BMC Med. 2014;12:22. PubMed | Google Scholar


43. Morris SK, Bassani DG, Kumar R, Awasthi S, Paul VK, Jha P. Factors associated with physician agreement on verbal autopsy of over 27000 childhood deaths in India. PLoS ONE. 2010 Mar 8;5(3):e9583. PubMed | Google Scholar


44. Engmann C, Jehan I, Ditekemena J, Garces A, Phiri M, Mazariegos M et al. An alternative strategy for perinatal verbal autopsy coding: single versus multiple coders. Trop Med Int Health. 2011;16(1):18-29. PubMed | Google Scholar


45. Fottrell E, Byass P. Verbal autopsy: methods in transition. Epidemiol Rev. 2010;32:38-55. PubMed | Google Scholar


46. Byass P, Hussain-Alkhateeb L, D'Ambruoso L, Clark S, Davies J, Fottrell E et al. An integrated approach to processing WHO-2016 verbal autopsy data: the InterVA-5 model. BMC Med. 2019;17(1):10. PubMed | Google Scholar


47. Lozano R, Freeman MK, James SL, Campbell B, Lopez AD, Flaxman AD et al. Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:50. PubMed | Google Scholar


48. Bauni E, Ndila C, Mochamah G, Nyutu G, Matata L, Ondieki C et al. Validating physician-certified verbal autopsy and probabilistic modeling (InterVA) approaches to verbal autopsy interpretation using hospital causes of adult deaths. Popul Health Metr. 2011;9:49. PubMed | Google Scholar


49. Byass P, Fottrell E, Dao LH, Berhane Y, Corrah T, Kahn K et al. Refining a probabilistic model for interpreting verbal autopsy data. Scand J Public Health. 2006;34(1):26-31. PubMed | Google Scholar


50. World Health Organization. The World health report 2000: health systems: improving performance. Geneva, Switzerland. 2000. Google Scholar


51. Abouzahr C. Verbal autopsy: who needs it?. Popul Health Metr. 2011;9:19. PubMed | Google Scholar


52. Oluwasola OA, Fawole OI, Otegbayo AJ, Ogun GO, Adebamowo CA, Bamigboye AE. The autopsy: knowledge, attitude, and perceptions of doctors and relatives of the deceased. Arch Pathol Lab Med. 2009;133(1):78-82. PubMed | Google Scholar


53. Lishimpi K, Chintu C, Lucas S, Mudenda V, Kaluwaji J, Story A et al. Necropsies in African children: consent dilemmas for parents and guardians. Arch Dis Child. 2001;84(6):463-7. PubMed | Google Scholar


54. Banyini AV, Rees D, Gilbert L. "Even if I were to consent, my family will never agree": exploring autopsy services for posthumous occupational lung disease compensation among mineworkers in South Africa. Glob Health Action. 2013;6:19518. PubMed | Google Scholar


55. Cox JA, Lukande RL, Kateregga A, Mayanja-Kizza H, Manabe YC, Colebunders R. Autopsy acceptance rate and reasons for decline in Mulago Hospital, Kampala, Uganda. Trop Med Int Health. 2011;16(8):1015-8. PubMed | Google Scholar


56. World Health Organization, Health statistics and information systems. Verbal autopsy standards: ascertaining and attributing causes of death (The 2016 WHO verbal autopsy instrument). Accessed 18 June 2018.