Enhancing capacity for clinical practice guidelines in South Africa

Enhancing capacity for clinical practice guidelines in South Africa

Taryn Young^1,&, Janine Dizon², Tamara Kredo^3,4, Michael McCaul¹, Eleanor Ochodo¹, Karen Grimmer^5,6, Quinette Louw⁵

¹Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Faculty of Medicine and Health Sciences, Stellenbosch University, PO Box 241, Cape Town, 8000, South Africa, ²International Centre for Allied Health Evidence (iCAHE), City East Campus, P4-18 North Terrace, University of South Australia, Adelaide 5000, Australia, ³Cochrane South Africa, South African Medical Research Council, P.O. Box 19070, Tygerberg, 7505, South Africa, ⁴Division of Clinical Pharmacology, Department of Medicine, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa, ⁵Physiotherapy Division, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa,

^&Corresponding author
Taryn Young, Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Faculty of Medicine and Health Sciences, Stellenbosch University, PO Box 241, Cape Town, 8000, South Africa

Introduction    

High quality, evidence-informed clinical practice guidelines (CPGs) offer a way of bridging the gap between policy, best-practice, local contexts and patient choice [1]. Good quality CPGs offer a ‘one stop shop’ for end-users [2]. They present current best evidence (primary, secondary or opinion-based) to answer questions pertaining to care for a health condition [3]. The development of CPGs has traditionally been motivated by the need to improve efficiency and cost-effectiveness of health system utilization, and to decrease costly and preventable mistakes and adverse events [4]. CPGs are intended to provide transparent synthesis of the evidence on which sound, ‘on balance’ judgments can be made by clinicians, administrators, policy-makers and patients. Evidence-informed decisions should minimise over-, under- or mis-use of services [5]. Constructing a good quality CPG and presenting recommendations in a user-friendly format is a more complex process, compared to constructing other forms of secondary evidence (such as systematic literature reviews) [6]. CPG construction usually involves multiple stakeholders including content experts, methodologists, systematic reviewers and guideline users. CPG methodologists have knowledge and experience to conduct evidence syntheses of multiple clinical questions for a single guideline, and guide the transparent and justified approach to assess the level of evidence to inform the final recommendation. Furthermore, the ability to formulate real-world recommendations is not only based on the evidence but also considers other factors such as applicability and acceptability to a local context. Consequently, specific training is required to understand the nuances of CPG writing, and to assist in efficient task completion.

South Africa has a long history of developing various CPGs. Key role players include the National Department of Health, professional societies, private sector and non-governmental organisations producing guidance for their respective constituents. Our team came together in 2014 to conduct the South African Guidelines Excellence (SAGE) Project [7]. We undertook a range of projects to understand the context, processes, and the need for, South African CPGs. Project SAGE explored perspectives of over 100 CPG stakeholders, regarding CPG development and use, how they assessed CPG believability and impact, how they implemented CPGs and what implementation barriers they faced [8-10]. Stakeholder interviews, particularly those involved with CPG development activities, consistently expressed a need for greater technical capacity to develop and implement CPGs. This was perceived to be important to ensure efficient and competent CPG construction, interpretation, updating and implementation. This finding was supported by the quantitative evaluation of 16 South African primary care clinical CPGs which were critically appraised for methodological quality. This activity found these CPGs to be substantially lacking in their reporting on methodological rigour, applicability and editorial independence [11, 12]. Project SAGE findings led to an innovative model of CPG transferability and implementation (adopt, contextualise, adapt) [8, 13]. In response to the gaps we identified in CPG reporting, and the expressed need for capacity building in evidence-informed CPG development and implementation, this paper describes the development, implementation and evaluation of a dedicated CPG training course to address the training needs of CPG stakeholders in South Africa.

Methods     

We first reviewed the content and teaching strategies of existing CPG courses using a systematic review of teaching and learning strategies for guideline teams and a document review of existing courses offered by international guideline groups, universities and professional groups. We then developed, implemented and evaluated a CPG course targeted to the needs of end-users in resource constrained environments.

Systematic review: to evaluate the effectiveness of teaching or learning strategies for CPGs, we searched for randomised and non-randomised trials, pre-post experimental studies or primary observational studies. We excluded studies that focussed only on methods for teaching evidence-based practice, and did not include CPGs. Our target participants included CPG writers, methodologists, implementers and evaluators. We developed a comprehensive strategy (Supplementary file 1) to search for all eligible studies available up to 29^th November 2017, regardless of language or publication status. For published literature we searched Medline, Cochrane Database of Systematic Reviews (Effective Practice and Organization Group), Database of Abstracts of Reviews of Effects (DARE), Cumulative Index to Nursing and Allied Health Literature (CINAHL), current controlled trials, Educational Resources Information Center (ERIC) and Turning Research into Practice (Trip) database. We used the following search terms and tailored them appropriately to the different databases: ‘guide*’, ‘clinical practice guideline’, ‘teach*’, ‘learn*’. Three reviewers (EO, JD and VL) independently screened titles, abstracts and full texts of potentially eligible articles. To access unpublished literature, we searched websites of 17 guideline groups using a Google scoping search. We also contacted relevant experts in the field of CPGs. We aimed to conduct independent data extraction and risk of bias assessments and then to look at within group differences focussing on outcomes measured between baseline and post training. These outcomes could include short-term measures on teaching methods or programs (i.e. educational outcomes including feedback, evaluation, examination or self-report scores) and longer-term measures on the quality of the guideline process from development to evaluation.

Document review of existing CPG courses: we searched the internet for existing CPG courses using search term “clinical practice guidelines courses” OR “guideline courses”. We searched guideline sites and professional associations/organizations. We contacted 21 content experts via email, professional groups and the networks of Cochrane South Africa, Centre for Evidence Based Health Care and International Centre for Allied Health Evidence (iCAHE) to seek information regarding any CPG course training being offered, and for any evaluation of CPG courses delivered. Eighteen groups responded and we obtained the information from the websites of three groups who did not respond. We collected all possible information about CPG courses and extracted relevant information: course description, objectives, content, teaching and learning strategies, mode of delivery, format, duration, outcomes and evaluation/assessments. We used a descriptive qualitative framework to guide our data collection. Qualitative descriptive approaches are particularly useful in program assessment and development. We used content analysis to synthesize our data [14] regarding CPG course information based from the available data. Content analysis is an approach to analyzing findings from the document search by categorizing and classifying information to summarize the findings. Content analysis is the recommended synthesis approach for descriptive qualitative information. Findings were summarized on the basis of the data extracted.

CPG course development and implementation: we built on an existing master's level elective course on CPGs developed at Stellenbosch University, South Africa, some five years earlier. The original course was initially offered in 2009 as part of the MSc in Clinical Epidemiology. Between its inception and 2014 it had already shifted from being a mostly theory-based half-semester course to a practical full semester (4-month) course. However, with funding support from Project SAGE, together with reflection on elements of research and feedback from the project, we recognised that the needs of students working in resource constrained settings differ from those in other contexts. Thus, the course needed to be tailored to their needs. In 2015, the course was redesigned. Course characteristics are presented in Table 1. The course was offered in 2016 and a formal independent evaluation was conducted.

CPG course evaluation: a document review, observation of training delivered during the contact sessions and interviews with course participants were done. It evaluated whether the course material addressed the aim of the course and whether the predicted learning outcomes would be met; whether resources were appropriate to assist students to achieve the learning objectives; how assessments mapped to the learning outcomes; how well the teaching and learning approach mapped to best practice pedagogy; and determined the level of satisfaction of students with the teaching and learning processes of the course. All participants of 2016 were invited to participate. All course outlines and teaching material were provided to the independent evaluation team. They ascertained whether the course material addressed the course objectives, if the presentation of the course material was appropriate for the time-frame for the intensive, five day teaching mode and whether assessments mapped to the learning outcomes. In addition, the content expert ascertained whether the predicted learning outcomes were met by the teaching material. A pre-course demographics questionnaire was designed and emailed to all course participants one week prior to commencement of the face-to-face contact session in March (Supplementary file 2). This questionnaire was used to obtain: demographics (gender, name, age and current employment, contact details, citizenship); academic qualifications (and institution where the training was obtained); CPG training (whether students had received any prior CPG training, and if so, details about the training including where the training was obtained, the duration of the training program and brief description of the course content); and experience with CPG activity (any involvement with the CPG development, implementation of CPGs to the local context, or training related to CPGs). The workshop evaluation form was informed by the training evaluation framework. It focused on generic training issues and also explored the three key learning areas from each of the five days of face-to-face training.

The evaluation team observed training sessions and conducted post-course telephonic interviews one month after the March face-to-face contact sessions. The semi-structured telephonic interviews (in English, Supplementary file 3) focused on perceived change in knowledge or attitudes towards CPGs, barriers that participants anticipated in putting their learning outcomes into action, support for students post-completion so that barriers can be addressed, whether the course content information was considered comprehensive and value for money or time, and the level of satisfaction with the teaching and learning process. The evaluation team also interviewed an international content expert in the field (Supplementary file 4 for interview guide). A digital voice recorder was used to record interviews. Interviews lasted between thirty and sixty minutes. Interviews were transcribed verbatim. Names of participants did not appear on the transcriptions. The team used triangulation of data sources (interviews and workshop evaluations) and carried out inter-rater reliability checks on at least two of the observation days. Qualitative data from telephonic interviews, the observation schedule and the text generated from workshop evaluation were stored and managed using Atlas.ti. Inductive thematic content analysis was undertaken to analyse the interviews and identify key themes and sub-themes from the interview transcripts. Ethical clearance was obtained from the Health Research Ethics Committee (HREC), Stellenbosch University (ethics number N14/02/008A). All participants provided informed written consent.

Results     

Systematic review: our search for teaching and learning strategies yielded 6224 hits. Upon screening the titles and abstracts, we found nine potentially-eligible studies and excluded all upon reading the full texts, three articles were primary guidelines [15-17], two focused on guideline methodology [18, 19], two were on guideline implementation [20, 21] and two were surveys on methodological process [22, 23] (Figure 1). Thus, we found no eligible studies from our systematic literature review.

Document review of existing CPG courses: we found six (6) CPG courses, four (4) from universities (University of Sydney, McMaster University, Stellenbosch University, University of Toronto) and two (2) from professional groups (American Academy of Orthopaedic Surgeons, Kidney Health Australia) (Table 2). These were either offered by universities as a full course (part of a postgraduate degree program) or as independent training for continuing professional education by professional groups. The CPG courses focused on covering one or more components of the guideline development process (e.g. critical appraisal of guidelines, developing recommendations) as well as developing implementation plans (Table 2). One university-based CPG course included involvement of consumers in guidelines and examination of hospital-and community based guidelines (University of Sydney). Another university-based CPG course included other methods of guideline approaches such as contextualization and adaptation (Stellenbosch University) (Table 2). The format and delivery of CPG courses varied from face-to-face sessions, to online sessions, group exercises and discussions, seminar format and project based activities (Table 2). Courses ranged from two hours (short course) to 120 hours (full course program) depending on the content being covered. Only CPG courses offered by universities reported evaluations of their courses. Evaluation procedures included appraisal and barriers assessment, summative assessments, quizzes, examinations and online discussions (Table 2). In summary, it appeared from our comprehensive systematic search that CPG courses were offered by only a few universities and health professional groups. The focus of the courses was mainly on developing de novo CPGs or on one or more components of the process of developing de novo CPGs.

CPG course implementation and evaluation: the CPG four month long course was delivered at Stellenbosch University in 2016. The course was designed to be pragmatic, reflective and contextually relevant, using clinically relevant issues and examples, authentic tasks, online discussions and existing online resources. Students attended a 5-day teaching block with the remaining time spent in online engagement, self-study, readings and assessments. The course covered principles of both de novo CPG development and alternative methods of CPG writing (adopting, contextualising, adapting and updating CPGs). Students were assessed using online activities and written assignments followed by a summative portfolio of evidence which included reflective writing. The evaluation identified strengths and areas for improvement. Eighteen of the twenty one students who attended the course completed the demographic survey (Table 3). Most students were from South Africa, the majority had no previous training in writing CPGs and about a third had previous experience with using CPGs. Students rated the course highly, expressed that for most the course met their expectations, and that the content was relevant and they had learnt new things.

‘I learnt about new sources of guidelines new work platforms to find guidelines. I also learnt about the other types of guidelines that are out there. The type of guideline I use are WHO guidelines so it's nice to learn about the other types of guidelines that are out there.’ ‘Each one [module] had unique aspects that is going to help me in my job. We had modules on phrasing questions we had modules on searching and finding guidelines which is what I do every day. We had modules on evaluating guidelines, we had modules on implementation the implementation part, maybe not so much, but it is something I am actually getting into it. They were all very helpful, and I'm going to be using the knowledge I have learnt in my everyday work actually.’

Students appreciated the various forms of engagement, level of expertise and preparation by facilitators. They liked the use of practical exercises alongside didactic teaching, and rated the course as providing high value to their work. Students found the assessment process relevant, fair and appropriate. Some of the students gave recommendations for areas of improvement for future contact sessions-availability of materials before the block contact session, allotting more time for some sessions, early briefing on the assessment structure as well as provision of in-class notes. They also indicated some concern about having too much information covered during the 5-day teaching block.

‘I wish they could have structured it in a way that you speak for an hour, somebody else comes for an hour, and you come back so that you have different people speaking. And they have more interactive slides and you ask a few questions, somebody answers. Now some of the people all they do is sit there in front of us and go through it so fast and you are more confused by the end of the hour than you were before.’

The CPG content expert reviewer emphasized the importance of conveying concepts simply and ensuring adequate time for practical examples and reflection. He suggested additional relevant content such as the project management and chairing skills for CPG development as he described that ‘at the heart of this process is a very mucky set of human interactions which can be managed well and can be managed badly, and can make a huge difference to the outcome.’

Discussion     

With the growth in evidence-informed practices in the African region [24, 25] and the recognised role of evidence-informed CPGs in shaping healthcare practices, there is an increasing need, and demand for, building capacity for evidence-informed CPG development and implementation [9, 10]. This paper described the development, implementation and evaluation of a dedicated CPG training course designed to meet specific development needs in South Africa and other similar resource constrained settings. The paucity of courses may be that CPG activities are traditionally learned on the job with panels often consisting of practicing clinical experts. However, this approach has long been recognised as insufficient [26]. Evidence of poor CPG quality, globally and in our region, provide a sound case for the need for additional training [12, 27-29]. In response to these needs, we used an already available CPG course offered as part of the Masters in Clinical Epidemiology degree at Stellenbosch University [30], we transformed the content of this course to be more contextually relevant, pragmatic and student-focused. It covered de novo CPG development, alternative methods of writing (adopting, contextualising, adapting) as well as implementing CPGs. It used a blended learning approach, with both face-to-face and online sessions, coupled with assessment. It used local CPG examples, and authentic tasks. Course evaluation identified strengths and areas for improvement.

The course fills a specific gap for decision-makers from resource constrained settings as we share best practice knowledge for alternative development of CPGs. De novo CPG development is time and resource intensive requiring technical skills for conducting systematic reviews, funding to host meetings and ensure participation and consultation as broadly as possible. For this reason, alternative methods have been developed and our course is unique in generating awareness and discussion of these approaches [13, 31]. We are already seeing the benefits of skills development as students exposed to CPG training have lead CPG processes using rigorous reportable methods [32, 33]. Despite the demand for improving CPG-related skills, particularly in countries with scarce healthcare and training resources, there are limited robust evaluations of CPG capacity-development initiatives. The findings of our systematic assessment of existing international CPG training courses underpinned a targeted comprehensive CPG training programme. The course evaluation followed good practice methods for teaching evaluation including a review of documents, observation, interviews with faculty, and a survey of students. The inclusion of independent evaluation team, enhanced the objective collection of data from students. The evaluation had limitations as it did not assess the longer term effectiveness of learning to determine whether learning outcomes were achieved. Despite these limitations, this process could inform future capacity development initiatives in evidence production and uptake.

Conclusion     

We used an evidence-informed multifaceted approach to enhance a master's level CPG course for the needs of CPG developers and implementers in resource constrained settings. Dedicated, evidence-informed capacity development for CPG activities may empower current and future CPG developers and end-users and has the potential to positively influence CPG development and implementation.

Competing interests     

The authors declare no competing interests.

Authors' contributions     

Young T lead the capacity development aspect of the SAGE project, lead the writing of this overarching manuscript, contributed to the planning and implementation of the CPG course. Dizon JMR lead the assessment of existing CPG courses, contributed to the systematic review and implementation of the CPG course. Kredo T played a key role in the CPG course planning and implementation. McCaul M played a key role in the CPG course planning and implementation. Ochodo E lead the systematic review and contributed to the implementation of the CPG course. Grimmer K contributed to the planning and implementation of the CPG course. Louw Q co-lead the capacity development aspect of the SAGE project, played a key role in the planning, implementation and evaluation of the CPG course. All authors contributed to manuscript writing and approved the final version.

Acknowledgments     

This research was supported through the Flagships Awards Project by the South African Medical Research Council (SAMRC-RFA-IFSP-01-2013/ SAGE) which has enabled the project and supported activities. Vittoria Lutje who helped with the searching and screening process of the systematic review. External evaluation team from Evaluation Research Agency.

Tables and figure     

Table 1: CPG module characteristics

Table 2: document review: summary of CPG courses offered by universities and professional organizations

Table 3: evaluating student's demographic and past CPG exposure

Figure 1: flow chart of systematic review search results

References     

1. Shaun Treweek, Andrew D Oxman, Philip Alderson, Patrick M Bossuyt, Linn Brandt, Jan Brożek et al. Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013; 8:6. PubMed | Google Scholar


2. IOM 2011, GReae. Clinical Practice Guidelines We Can Trust. Institute of Medicine, Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. The National Academies Press, Washington, DC. 2011;:15. PubMed


3. Kredo T, Bernhardsson S, Machingaidze S, Young T, Louw Q, Ochodo E et al. Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care. 2016;28(1):122-8. PubMed | Google Scholar


4. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525-31. PubMed | Google Scholar


5. Runciman WB, Williamson JA, Deakin A, Benveniste KA, Bannon K, Hibbert PD. An integrated framework for safety, quality and risk management: an information and incident management system based on a universal patient safety classification. Qual Saf Health Care. 2006 Dec;15 Suppl 1:i82-90. PubMed | Google Scholar


6. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014 Feb 18;186(3):E123-42. PubMed | Google Scholar


7. Kredo T1, Machingaidze S, Louw Q, Young T, Grimmer K. South African Guideline Excellence (SAGE): What's in a name? S Afr Med J. 2015;106(1):18-20. PubMed | Google Scholar


8. Dizon JM, Grimmer K, Louw Q, Kredo T, Young T, Machingaidze S. South African Guidelines Excellence (SAGE): Adopt, adapt, or contextualise? S Afr Med J. 2016;106(12):1177-1178. PubMed | Google Scholar


9. Kredo T, Abrams A, Young T, Louw Q, Volmink J, Daniels K. Primary care clinical practice guidelines in South Africa: qualitative study exploring perspectives of national stakeholders. BMC Health Serv Res. 2017 Aug 29;17(1):608. PubMed | Google Scholar


10. JM Dizon, K Grimmer, Q Louw, S Machingaidze, H Parker, H Pillen. Barriers and enablers for the development and implementation of allied health clinical practice guidelines in South African primary healthcare settings: a qualitative study. Health Res Policy Syst. 2017;15(1):79. PubMed | Google Scholar


11. Karen Grimmer, Shingai Machingaidze, Janine Dizon, Tamara Kredo, Quinette Louw, Taryn Young. South African clinical practice guidelines quality measured with complex and rapid appraisal instruments. BMC Res Notes. 2016;9:244. PubMed | Google Scholar


12. Machingaidze S, Zani B, Abrams A, Durao S, Louw Q, Kredo T et al. Series: Clinical Epidemiology in South Africa, Paper 2: Quality and reporting standards of South African primary care clinical practice guidelines. J Clin Epidemiol. 2017;83:31-36. PubMed | Google Scholar


13. Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442. PubMed | Google Scholar


14. Mays N, C Pope. Qualitative research: Observational methods in health care settings. BMJ. 1995;311(6998):182-4. PubMed | Google Scholar


15. Kingson JF. Training child life students: guidelines for hospital supervisors. Child Health Care. 1984;12(3):137-42. PubMed | Google Scholar


16. Perraton H, C Creed, Bea Robinson. Teacher Education Guidelines: Using Open and Distance Learning. Technology, Curriculum, Cost, Evaluation. ERIC. 2002. Google Scholar


17. Trogdon JG, Allaire BT, Egan BM, Lackland DT, Masters D. Training providers in hypertension guidelines: cost-effectiveness evaluation of a continuing medical education program in South Carolina. Am Heart J. 2011;162(4):786-793 e1. PubMed | Google Scholar


18. Lappia JH. Towards Design Guidelines for Work Related Learning Arrangements. Journal of European Industrial Training. 2011;35(6):573-88. Google Scholar


19. Love RR. Clinical trials and practice guidelines as educational methods in developing countries. Journal of cancer education. 1994;9(4):200-1. Google Scholar


20. Grimshaw J, M Eccles, J Tetroe. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004; 24 Suppl 1:S31-7. PubMed | Google Scholar


21. Brouwers MC, Makarski J, Durocher LD, Levinson AJ. E-learning interventions are comparable to user's manual in a randomized trial of training strategies for the AGREE II. Implement Sci. 2011 Jul 26;6:81. PubMed | Google Scholar


22. Haller MC, van der Veer SN, Nagler EV, Tomson C, Lewington A, Hemmelgarn BR et al. A survey on the methodological processes and policies of renal guideline groups as a first step to harmonize renal guidelines. Nephrol Dial Transplant. 2015;30(7):1066-74. PubMed | Google Scholar


23. Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J. AGREE Collaboration. Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003;15(1):31-45. PubMed | Google Scholar


24. Young T, Garner P, Clarke M, Volmink J. Series: Clinical Epidemiology in South Africa, Paper 1: Evidence-based health care and policy in Africa: past, present, and future. J Clin Epidemiol. 2017 Mar;83:24-30. PubMed | Google Scholar


25. Gulland A. Doctors pledge to spread evidence based healthcare in Africa. BMJ. 2013;346:f356. PubMed | Google Scholar


26. Paul G Shekelle, Steven H Woolf, Jeremy Grimshaw. Clinical guidelines: developing guidelines. BMJ. 1999;318(7183):593-6. PubMed | Google Scholar


27. Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58. PubMed | Google Scholar


28. Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T et al. International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. J Clin Oncol. 2004;22(10):2000-7. PubMed | Google Scholar


29. Kredo T, Gerritsen A, van Heerden J, Conway S, Siegfried N. Clinical practice guidelines within the Southern African Development Community: a descriptive study of the quality of guideline development and concordance with best evidence for five priority diseases. Health Res Policy Syst. 2012 Jan 5;10:1. PubMed | Google Scholar


30. Taryn Young, Celeste Naude, Tania Brodovcky, Tonya Esterhuizen. Building capacity in Clinical Epidemiology in Africa: experiences from Masters programmes. BMC Med Educ. 2017;17(1):46. PubMed | Google Scholar


31. Schünemann HJ, Wiercioch W, Brozek J, Etxeandia-Ikobaltzeta I, Mustafa RA, Manja V et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol. 2017;81:101-110. PubMed | Google Scholar


32. McCaul M, de Waal B, Hodkinson P, Pigoga JL, Young T, Wallis LA. Developing prehospital clinical practice guidelines for resource limited settings: why re-invent the wheel? BMC Res Notes. 2018;11(1):97. PubMed | Google Scholar


33. Michael McCaul, Dawn Ernstzen, Henk Temmingh, Beverly Draper, Michelle Galloway, Tamara Kredo. Adaptive clinical practice guideline development methods in resource-constrained settings: four case studies from South Africa. EBM Analysis. 2018. Google Scholar