Triage capabilities of medical trainees in Ghana using the South African triage scale: an opportunity to improve emergency care

Triage capabilities of medical trainees in Ghana using the South African triage scale: an opportunity to improve emergency care

Adam Gyedu^1,2,&, Kwabena Agbedinu², Mohammed Dalwai³, Maxwell Osei-Ampofo⁴, Emmanuel Kweku Nakua⁵, Rockefeller Oteng^4,6, Barclay Stewart^1,7,8

 

¹Department of Surgery, School of Medical Sciences, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana, ²Directorate of Surgery, Komfo Anokye Teaching Hospital, Kumasi, Ghana, ³Division of Emergency Medicine, Faculty of Health Sciences, University of Cape Town, South Africa, ⁴Emergency Medicine Directorate, Komfo Anokye Teaching Hospital, Kumasi, Ghana, ⁵Department of Population, Family and Reproductive Health, School of Public Health, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana, ⁶Department of Emergency Medicine, University of Michigan Health System, Ann Arbor, MI, USA, ⁷Department of Surgery, University of Washington, Seattle, WA, USA, ⁸Department of Interdisciplinary Health Sciences, Stellenbosch University, Cape Town, South Africa

 

 

^&Corresponding author
Adam Gyedu, Department of Surgery, School of Medical Sciences, KNUST Kumasi, Ghana

 

 

 

 

The incidence of emergency conditions is increasing worldwide due to aging populations, urbanization and a lack of preventative care capacity [1-3]. This increasing burden falls disproportionately on low and middle-income countries (LMICs), which are ill equipped to provide emergency care [4, 5]. Opportunely, sound planning for and organization of emergency care can prevent deaths and reduce disability without the addition of significant costs [6-8]. An important part of emergency care organization is triage [9]. Triage improves emergency care service delivery by optimizing the utilization of existing and often scarce resources among patients depending on their acuity [10]. This process is particularly important in LMICs, which often have limited human and physical emergency resource capacity [11]. However, in many LMICs, triage is under-utilized, under-resourced, and poorly researched [12]. The South African Triage Scale (SATS) was developed in 2004 as a simple triage tool for South Africa [9]. The SATS has proven reliability and validity among providers who have undergone dedicated triage training, in emergency centers with limited resources both in and out of South Africa [12]. It has become widely used in LMICs, including first-level hospitals, which are typically the first point of care for patients with emergency conditions [12, 13]. Given these successes, the Ministry of Health (MoH) of Ghana recommended that the SATS become the triage method of choice for all hospitals in the country in 2011 [14, 15]. Although it is endorsed by MoH in Ghana and is expected to be in practice, triage training for current medical school trainees and existing non-emergency medicine providers has not been delivered. Although the SATS has proven construct validity and is effective in most LMIC settings, it has not been formally assessed among providers without dedicated SATS training. In Ghana, as in many first-level hospitals in LMICs, the healthcare provider caring for patients with emergency conditions is a general practitioner (GP) or a non-physician [4, 16]. Medical students graduate, spend two years as a house-officer to become a GP, and then are deployed to first-level hospitals countrywide for at least one year. These providers are often responsible for planning and organizing triage and emergency care at their facility [17, 18]. However, emergency care training of medical students and house officers has not been prioritized [19]; thus, many house officers who are becoming GPs and preparing for their national service in first-level hospitals may lack the knowledge necessary for effective triage. To address these gaps, we aimed to: i. assess the reliability and validity of the SATS when used by providers not specifically trained in SATS use; and ii. compare triage capabilities between senior medical students and senior house officers to examine the effectiveness of our house officer training curriculum with regards to triage. The findings might inform decisions around formal triage training for current and future medical trainees in Ghana, as well as LMICs more broadly.

 

 

Setting

 

Ghana is a heavily indebted, lower-middle income country in West Africa with a population of 26 million people and an annual per capita income of US$ 1,760 [20]. This study focused on medical students and house officers at Komfo Anokye Teaching Hospital (KATH), which is a 1,200-bed tertiary facility that serves around 8.6 million Ghanaians [21]. After graduating medical school, students become house officers. House officers spend six months each in internal medicine, general surgery, obstetrics and gynecology, and pediatrics. Additionally, house officers care for patients in the Accident and Emergency Unit (A&E) at KATH who have an emergency condition that falls under the auspices of each of the specialties. Some of the concepts surrounding triage are taught as part of a broader medical and surgical curriculum for both students and house officers (e.g. shock, injury management); however, neither curriculum includes modules specifically on triage theory or methods or SATS.

 

South African Triage Scale and its use in Ghana

 

The SATS is a physiologically based composite scoring system that includes the Triage Early Warning Score and a list of discriminators to triage patients into one of five color-coded acuity groups: Red – emergency (i.e. patient to be seen by a provider immediately); Orange - very urgent (i.e. patient to be seen by a provider within 10 minutes); Yellow - urgent (i.e. patient to be seen by a provider within 60 minutes); Green - routine (i.e. patient to be seen by a provider within 240 minutes); or Blue - dead. It was designed to be simple and used by nurses to alleviate the pressures of both human and physical resource limitations at hospitals in LMICs [12]. Given its simplicity and effectiveness in similar contexts, the MoH in Ghana designated SATS as the triage method of choice for casualty units across the country in 2011 [15]. It was incorporated into A&E care at KATH in 2010 [14]. At KATH, the nurses perform triage and patients are transported to and cared for in the respective color-coded ward. However, at the 187 first-level/district hospitals across the country, both nurses and GPs perform triage; the latter are responsible for organizing emergency care and deciding the order in which patients are resuscitated, treated, and/or referred to a higher level of care..

 

Survey

 

We created an online survey with 25 questions via SurveyMonkey (SurveyMonkey, CA, USA) using triage vignettes validated in South Africa and Pakistan [12, 22]. Each question described a patient presentation in the form of a vignette. The type and spectrum of emergencies presented in the vignettes were similar to situations encountered in Ghana. The vignettes included information on gender, age, presenting complaint, mode of arrival to the casualty unit, and vital signs. Some vignettes also included information from investigations that are often performed at the time of triage (e.g. blood glucose, hemoglobin). Respondents were asked to assign an acuity level to each vignette that corresponded to the SATS acuity groups (e.g. emergency, very urgent, urgent, or routine); vignettes that described a patient that should have been triaged blue were not included. Each question required a response (i.e. the participant could not proceed without answering the question). To avoid testing transgressions, questions were timed and question order was randomized for each respondent. The survey was performed in English, which is the language used to teach in Ghana.

 

Strategy and sample

 

To evaluate the survey acceptability, estimate sample size and set question times, 10 medical students and 10 house officers piloted the survey. The median medical student score was 40% and the median house officer score was 70%. Sample size was estimated to be 80 participants (i.e. 40 medical students and 40 senior house officers) using these pilot scores and:

Sample =

Z_α/2 + Z_ß)² x [(p₁(1- µ ₂) + p₂(1- 1 - µ ₂)] /(p₁ - p₂ )²

Where Z_α/2 is the critical value of the normal distribution at α/2 (i.e. 1.96 for a confidence level of 95%), Z_ß is the critical value of the normal distribution at ß (i.e. 0.84 for a power of 80%), and µ ₁ and µ ₂ are the expected proportions of correct triage responses in medical students and house officers (i.e. 40 and 70%, respectively).

 

A list of current senior medical students and senior house officers was obtained. Sixty randomly selected potential participants were approached from each group via both email and WhatsApp (WhatsApp, CA, USA). A link to an introduction and informed consent page was included in the messages. After providing informed consent, the survey continued to an instructions page and then the emergency vignettes.

 

Reliability and validity

 

To assess the extent to which the triage scale yields the same assessment between different respondents rating the same patient, we measured inter-rater reliability. In accordance with the Guidelines for Reporting Reliability and Agreement Studies (GRRAS), inter-rater reliability was assessed using the unweighted, linearly weighted and quadratically weighted κ (QWK) statistic [23]. The QWK considers the degree of disagreement between responses. A weighted κ places maximum weight at the two opposite ends of the triage scale (e.g. emergency and routine); therefore, it is identical to the intra-class correlation coefficient (ICC) [10]. Thus, the ICC was not reported given its equivalence with the QWK statistic [24]. Standard errors and confidence intervals were calculated using jackknife simulation. To allow apposite comparison with other SATS assessments, point estimates for each measure of inter-rater reliability were graded using the Landis and Koch classification system as follows: 0.0 - 0.20 - slight agreement; 0.21 - 0.40 – fair agreement; 0.41 - 0.60 – moderate agreement; 0.61 - 0.80 - substantial agreement; and 0.81 - 1.00 - almost perfect agreement [25]. Ten randomly selected respondents in each of the two groups re-triaged 10 randomly selected vignettes after one week. The results of this re-assessment were used to estimate intra-rater reliability by calculating the percentage of exact agreement, as well as the percentage of agreement allowing for one level of discrepancy in the triage assignments. The triage assignment accuracy of both groups was assessed by calculating the sensitivity, specificity, and over- or under-triage relative to the triage assignment for each of the vignettes suggested by an expert triage panel. The characteristics of and methods used by the expert panel for determining the triage assignment for each of the vignettes have been previously published [22]. Briefly, a panel of 18 emergency medicine physicians and emergency nurses from both developing and developed countries independently reviewed the vignettes. Using the Delphi technique, the panel reached consensus on the ‘true’ acuity of the patient in each vignette. They assigned an acuity based on their expert opinion rather than through the direct application of SATS; however, the acuities assigned mirrored SATS colors (i.e. emergency, very urgent, urgent, and routine). By using this method, triage assignment for each vignette follows emergency care acumen, and not exclusively the application of SATS itself. Over- and under-triage were interpreted using an accepted range for average under-triage of not more than 5 - 10% and an associated average over-triage rate of 30 - 50%. These ranges are considered acceptable by the American College of Surgeons Committee on Trauma.[26] Data were analyzed using STATA v13.1 (StataCorp, TX, USA).

 

Ethics

 

Ethical approval was obtained from the Kwame Nkrumah University of Science and Technology Committee on Human Research and Publication Ethics.

 

 

Sixty senior medical students and sixty senior house officers were approached to participate in the survey. Fifty-nine medical students (98% response rate) and 43 house officers (72% response rate) completed the survey (84% response rate overall).

 

Reliability

 

A total of 2,550 triage assignments were included in the analysis, which consisted of 59 senior medical student and 43 senior house officer triage assignments for 25 vignettes each (1,475 and 1,075 triage assignments, respectively). Inter-rater reliability was fair in both groups, as measured by unweighted κ (Table 1). When measured by linearly and quadratically weighted κ, agreement was moderate. The level of exact intra-rater agreement among respondents was substantial (74%, 95% CI 67 – 80). When allowing for a one level of discrepancy between triage assignments, the level of intra-rater agreement increased to almost perfect (97%, 95% CI 95 - 99).

 

Validity

 

Table 2 and Table 3 present the accuracy of senior medical student and senior house officer triage assignment compared with those assigned by the expert panel, respectively. Overall, the triage using SATS performed by these groups had low sensitivity (medical students: 54%, 95% CI 49 – 59; house officers: 55%, 95% CI 48 – 60) and moderate specificity (medical students: 84%, 95% CI 82 - 89; house officers: 84%, 95% CI 82 - 97). Sensitivity was lower for "emergency" and "routine" triage assignments in both groups; however, these assignments had the highest specificity. Both groups over-triaged vignette patients whose triage assignment was "routine" (medical students: 77%, 95% CI 75 – 79; house officers: 79%, 95% CI 76 – 82), and nearly half of vignette patients who were "urgent" (Table 2 and Table 3). As reference, over-triage rates should be less than 50% [26]. Alarmingly, both groups under-triaged most of the "emergency" vignette patients (80 and 82% for medical students and house officers, respectively). "Very urgent" vignette patients were under-triaged in around half of triage assignments (53 and 52% for medical students and house officers, respectively). The published and acceptable under-triage rates should be less than 10% [26].

 

Difference between medical student and house officer triage

 

There was no evidence to suggest a difference between senior medical students and senior house officers with regards to inter-rater reliability (Table 1), triage assignment accuracy, or over- or under-triage (Table 2 and Table 3).

 

 

This study aimed to demonstrate the reliability and validity of the SATS among providers not specifically trained in triage or SATS use and to compare medical student and house officer triage capabilities. Among untrained users, SATS demonstrated fair to moderate inter-rater reliability, low sensitivity, and moderate to high specificity. Most importantly, both groups over- and under-triaged vignette patients too frequently compared to accepted standards. Lastly, there was no difference between the capabilities of medical students and house officers to triage vignette patients correctly. These findings highlight two issues: i. while the SATS is effective in many settings, it requires some degree of specific training to be used effectively; and ii. the house officer training curriculum at our teaching facility with regards to triage is inadequate. The SATS has been used successfully in each hospital level (i.e. first-level, referral hospitals, tertiary hospitals) and numerous LMICs [12-14]. In reports from these settings, training was uniformly included as part of the SATS implementation process. For example, the SATS was successfully implemented at the Médecins Sans Frontières (MSF) hospital in Timergara, Pakistan in 2011 [12]. Like government hospitals in LMICs, this hospital also faced human and physical resource limitations. Nurses performed triage at this hospital. Prior to SATS implementation, each nurse participated in a 1-hour training about triage and SATS use specifically. After nurses had been using the SATS for at least a month, the scale was assessed in detail with methods similar to those presented here. Inter-rater reliability in that setting was moderate to substantial and intra-rater agreement was high (exact agreement 87%, 95% CI 67 – 100). Further, the sensitivity and specificity compared to the same expert panel triage assignments as we used here were significantly higher than that achieved by the medical students and house officers. In general, the nurses in Pakistan over- and under-triaged patients less often than our respondents. However, the nurses, medical students, and house officers most often under-triaged ‘emergency’ vignette patients. The differences discussed above demonstrate the importance of ensuring that staff are trained in triage theory and methods, as well as the tool they are expected to use. Despite having completed two years as a house officer and caring for patients with emergency conditions during the same period, house officers were not able to triage vignette patients more effectively than medical students. At our facility, there is no structured curriculum for triage or emergency care teaching for a house officer. House officers learn on-the-job, such as on rounds, in the clinic or operating theater, or in departmental meetings. Given that there was no difference in the triage capabilities between medical students and house officers, this approach appears to be inadequate for teaching triage fundamentals. Other LMICs have built house officer training programs that include structured modules that cover a number of topics essential to providing care in LMIC as a GP in a first-level hospital after graduation, such as triage [27]. Further, some house officer programs evaluate their trainees based upon their knowledge of the topics covered in the modules [28]; emergency care vignettes such as these might be used to assess training quality and progress over time [22, 29]. Given that other studies have reported similar triage inabilities of providers graduating to first-level hospitals in other LMICs, this is not a problem that is unique to our teaching facility [30, 31]. Thus, teaching facilities in LMICs globally should consider assessing their curricula regarding triage, particularly for providers who will soon be transferred to first-level hospitals, where they will be without senior provider supervision.

 

A systematic review of emergency care in 59 LMICs reported that only 17 of the countries had an emergency physician; 27 countries had non-emergency consultants providing emergency care; and 15 countries had non-physicians or medical students providing emergency care [32]. In Ghana, we have a unique opportunity to improve emergency care training by including trainees in the operations of two newly developed emergency medicine residency programs [19]. Structured emergency medicine training at these facilities is currently available to residents. At Komfo Anokye Teaching Hospital there is a two-week rotation for medical students as they rotate through surgery. However, with local and national support, this two-week “exposure” could be expanded in length and in availability. It could also be easily adjusted to include house officers. Additionally, these facilities might also serve as resources for existing providers from surrounding hospitals who might be insufficiently trained in triage or emergency care (e.g. supporting continuing education experiences). Further, advanced trainees or emergency medicine faculty might serve as trainers for existing providers in first-level and referral hospitals; such models have been effective in Papua New Guinea [33, 34]. While this study clearly demonstrates the importance of triage training and highlights current training deficiencies in our house officer curriculum, there are several limitations that should be considered when interpreting the results. First, the use of online vignettes lack patient cues and other important contextual information that can be used to improve triage accuracy. However, the vignettes and survey methods we used have been well validated in LMIC settings [12, 35]. Additionally, a published report compared the use of vignettes to live emergency patients for assessing inter-rater reliability and demonstrated an acceptable level of agreement between the two methods [36]. Second, the vignettes were developed in South Africa, which is more developed and has slightly different emergency condition epidemiology than Ghana [22, 37]. However, the vignettes were screened for contextual relevance and deemed appropriate by pilot groups of medical students and house officers. Next, the response rate of house officers was not high (72%). However, the sample frame of house officers was randomly selected, and they are all exposed to the same curriculum, or lack thereof. Lastly, some of the experts in the panel that developed consensus on the triage assignments for the vignettes were from high-income countries. These individuals might have over-assigned acuity (i.e. over-triaged vignette patients), which might partially explain why our groups under-triaged vignette patients. However, the triage assignments have been evaluated in South Africa and at a first-level MSF hospital in Pakistan with markedly lower under-triaged rates. Despite these limitations, these findings allow reasonable conclusions to be made about the importance of specific triage training for those who are to follow the SATS, as well as potential deficiencies in house officer curricula in Ghana and potentially LMICs more broadly.

 

 

Although the SATS has proven utility in a number of different settings and LMICs, its success relies on its use by trained providers. Given the large and growing burden of emergency conditions, training current students and house officers in triage is imperative. A number of triage and emergency care training models exist in high-income countries, and several LMICs have formed accredited training programs. However, ways in which LMIC emergency medicine training programs can have broader benefits for future and current first-level hospital providers is yet to be seen. In the meantime, the basics of triage must be taught to medical students and house officers. Otherwise, the cadre of providers at first-level hospitals in LMICs will be unable to quickly prioritize the care of critically ill and injured patients they will certainly encounter; inadequate triage capabilities at this level may waste scarce resources and lead to avertable death and disability.

 

 

The authors declare no competing interest.

 

 

AG, KA, MD and BTS designed the study. AG, KA, BTS collected data or supervised its collection. AG, BTS and EN analyzed the data. AG, MO, RO & BTS prepared the manuscript. All authors have read and agreed to the final version of this manuscript and have equally contributed to its content and to the management of the case.

 

 

Table 1: measures of inter- and intra-rater reliability of senior medical student and senior house officer triage assignments of emergency vignettes using the South African Triage Scale in Ghana

Table 2: senior medical student triage assignments of emergency vignettes using the South African Triage Scale compared to those of an expert panel [22]

Table 3: senior house officer triage assignments of emergency vignettes using the South African Triage Scale compared to those of an expert panel [22]

 

 

1. Stewart B, Khanduri P, McCord C, Ohene-Yeboah M, Uranues S, Vega Rivera F et al. Global disease burden of conditions requiring emergency surgery. British Journal of Surgery. 2014 Jan; 101(1): e9-22. PubMed | Google Scholar


2. Gyedu A, Abantanga F, Kyei I, Boakye G, Stewart BT. Changing Epidemiology of Intestinal Obstruction in Ghana: Signs of Increasing Surgical Capacity and an Aging Population. Digestive surgery. 2015; 32(5): 389-96. PubMed | Google Scholar


3. Stewart BT, Wong E, Gupta S, Bastola S, Shrestha S, Kushner AL et al. Surgical need in an ageing population: a cluster-based household survey in Nepal. Lancet. 2015 Apr 27; 385(Suppl 2): S5. PubMed | Google Scholar


4. Stewart BT, Quansah R, Gyedu A, Ankomah J, Donkor P, Mock C. Strategic Assessment of Trauma Care Capacity in Ghana. World J Surg. 2015 Oct; 39(10): 2428-40. PubMed | Google Scholar


5. Japiong KB, Asiamah G, Owusu-Dabo E, Donkor P, Stewart B, Ebel BE et al. Availability of resources for emergency care at a second-level hospital in Ghana: a mixed methods assessment. African Journal of Emergency Medicine. 2015. Google Scholar


6. Mock C, Julliard C, Brundage S, Goosen J, Joshipura M. Guidelines for trauma quality improvement programmes. Geneva: World Health Organization. 2009. Google Scholar


7. Son NT, Mock C. Improvements in trauma care capabilities in Vietnam through use of the WHO-IATSIC Guidelines for Essential Trauma Care. International journal of injury control and safety promotion. 2006 Jun; 13(2): 125-7. PubMed | Google Scholar


8. Stewart BT, Gyedu A, Quansah R, Addo WL, Afoko A, Agbenorku P et al. District-level hospital trauma care audit filters: Delphi technique for defining context-appropriate indicators for quality improvement initiative evaluation in developing countries. Injury. 2016 Jan; 47(1): 211-9. PubMed | Google Scholar


9. Gottschalk SB, Wood D, DeVries S, Wallis LA, Bruijns S, Cape Triage G. The Cape Triage Score: a new triage system South Africa. Proposal from the Cape Triage Group. Emerg Med J. 2006 Feb; 23(2): 149-53. PubMed | Google Scholar


10. Tamburlini G, Di Mario S, Maggi RS, Vilarim JN, Gove S. Evaluation of guidelines for emergency triage assessment and treatment in developing countries. Archives of disease in childhood. 1999 Dec; 81(6): 478-82. PubMed | Google Scholar


11. Gove S, Tamburlini G, Molyneux E, Whitesell P, Campbell H. Development and technical basis of simplified guidelines for emergency triage assessment and treatment in developing countries. WHO Integrated Management of Childhood Illness Referral Care Project. Archives of disease in childhood. 1999 Dec; 81(6): 473-7. PubMed | Google Scholar


12. Dalwai MK, Twomey M, Maikere J, Said S, Wakeel M, Jemmy JP et al. Reliability and accuracy of the South African Triage Scale when used by nurses in the emergency department of Timergara Hospital, Pakistan. S Afr Med J. 2014 Mar 26; 104(5): 372-5 PubMed | Google Scholar


13. Rosedale K, Smith Z, Davies H, Wood D. The effectiveness of the South African Triage Score (SATS) in a rural emergency department. South African Medical Journal. 2011; 101(8): 537-40. PubMed | Google Scholar


14. Rominski S, Bell SA, Oduro G, Ampong P, Oteng R, Donkor P. The implementation of the South African Triage Score (SATS) in an urban teaching hospital, Ghana. African Journal of Emergency Medicine. 2014; 4(2): 71-5. PubMed | Google Scholar


15. Amoah AGB, Bannerman C, Zacharia A, Baah-Odoom D, Acquaye G, Addo WL et al. Policy and guidelines for hospital accident and emergency services in Ghana. Accra, Ghana: Ministry of Health, 2011. Google Scholar


16. Mock CN, Donkor P, Gawande A, Jamison DT, Kruk ME, Debas HT et al. Essential surgery: key messages from Disease Control Priorities, 3rd edition. Lancet. 2015 Feb 4. Google Scholar


17. Afari H, Hirschhorn LR, Michaelis A, Barker P, Sodzi-Tettey S. Quality improvement in emergency obstetric referrals: qualitative study of provider perspectives in Assin North district, Ghana. BMJ open. 2014; 4(5): e005052. PubMed | Google Scholar


18. Agyepong IA. Reforming health service delivery at district level in Ghana: the perspective of a district medical officer. Health Policy Plan. 1999 Mar; 14(1): 59-69. PubMed | Google Scholar


19. Martel J, Oteng R, Mould-Millman NK, Bell SA, Zakariah A, Oduro G et al. The development of sustainable emergency care in ghana: physician, nursing and prehospital care training initiatives. Journal of Emergency Medicine. 2014 Oct; 47(4): 462-8. PubMed | Google Scholar


20. CIA World Factbook Washington, D.C.: United States Central Intellegence Agency; 2014 [cited 2015 April 12th]. Available from: http://www.ciaworldfactbook.us/africa/ghana.html. Google Scholar


21. Gyedu A, Baah EG, Boakye G, Ohene-Yeboah M, Otupiri E, Stewart BT. Quality of referrals for elective surgery at a tertiary care hospital in a developing country: An opportunity for improving timely access to and cost-effectiveness of surgical care. Int J Surg. 2015 Mar; 15: 74-8. PubMed | Google Scholar


22. Twomey M, Wallis LA, Myers JE. Evaluating the construct of triage acuity against a set of reference vignettes developed via modified Delphi method. Emerg Med J. 2013 Apr 24. PubMed | Google Scholar


23. Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011 Jan; 64(1): 96-106. PubMed | Google Scholar


24. Brenner H, Kliebsch U. Dependence of weighted kappa coefficients on the number of categories. Epidemiology. 1996 Mar; 7(2): 199-202. PubMed | Google Scholar


25. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar; 33(1): 159-74. PubMed | Google Scholar


26. Mohan D, Rosengart MR, Farris C, Cohen E, Angus DC, Barnato AE. Assessing the feasibility of the American College of Surgeons' benchmarks for the triage of trauma patients. Archives of surgery. 2011 Jul; 146(7): 786-92. PubMed | Google Scholar


27. Cameron BH, Rambaran M, Sharma DP, Taylor RH. International surgery: the development of postgraduate surgical training in Guyana. Can J Surg. 2010 Feb; 53(1): 11-6. PubMed | Google Scholar


28. Wen LS, Geduld HI, Nagurney JT, Wallis LA. Africa's first emergency medicine training program at the University of Cape Town/Stellenbosch University: history, progress, and lessons learned. Acad Emerg Med. 2011 Aug; 18(8): 868-71. PubMed | Google Scholar


29. Carr SJ. Assessing clinical competency in medical senior house officers: how and why should we do it? Postgrad Med J. 2004 Feb; 80(940): 63-6. PubMed | Google Scholar


30. Muthaura PN, Khamis T, Ahmed M, Hussain SR. Perceptions of the preparedness of medical graduates for internship responsibilities in district hospitals in Kenya: a qualitative study. BMC Med Educ. 2015; 15(1): 178. PubMed | Google Scholar


31. Hexom B, Calvello EJB, Babcock CA, Fowler R, Theodosis C, Sirleaf JA et al. A model for emergency medicine education in post-conflict Liberia. African Journal of Emergency Medicine. 2012; 2(1): 143-50. PubMed | Google Scholar


32. Obermeyer Z, Abujaber S, Makar M, Stoll S, Kayden SR, Wallis LA et al. Emergency care in 59 low- and middle-income countries: a systematic review. Bulletin of the World Health Organization. 2015 Aug 1; 93(8): 577-86G. PubMed | Google Scholar


33. Curry C. A perspective on developing emergency medicine as a specialty. Int J Emerg Med. 2008 Sep; 1(3): 163-7. PubMed | Google Scholar


34. Symmons D, Curry C. Rural hospital generalist and emergency medicine training in Papua New Guinea. Emerg Med Australas. 2007 Apr; 19(2): 151-4. PubMed | Google Scholar


35. Stewart BT, Gyedu A, Quansah R, Addo WL, Afoko A, Agbenorku P et al. District-level hospital trauma care audit filters: Delphi technique for defining context-appropriate indicators for quality improvement initiative evaluation in developing countries. Injury. 2016 Jan;47(1):211-9. PubMed | Google Scholar


36. Worster A, Sardo A, Eva K, Fernandes CM, Upadhye S. Triage tool inter-rater reliability: a comparison of live versus paper case scenarios. J Emerg Nurs. 2007 Aug; 33(4): 319-23. PubMed | Google Scholar


37. Data Visualizations Seattle, WA: Institute of Health Metrics and Evaluation; 2013 [cited 2015 May 27]. Available from: http://vizhub.healthdata.org/gbd-cause-patterns/. Google Scholar