Assessment of quality and readability of internet-based health information related to commonly prescribed angiotensin receptor blockers

Assessment of quality and readability of internet-based health information related to commonly prescribed angiotensin receptor blockers

Anuoluwapo Oloidi^1,&, Sabina Onyinye Nduaguba², Kehinde Obamiro³

 

¹Medplus Health and Beauty, Saka Tinubu, Lagos State, Nigeria, ²Health Outcomes and Pharmacy Practice 'Division' College of Pharmacy, University of Texas at Austin, USA, ³Centre for Rural Health, College of Health and Medicine, University of Tasmania, Australia

 

 

^&Corresponding author
Anuoluwapo Oloidi, Medplus Health and Beauty, Saka Tinubu, Lagos State, Nigeria

 

 

 

 

The internet has rapidly become the largest source of information worldwide [1] and an important source of health information both for patients and health professionals [2]. With the advent of the internet, medical information is easily assessed with the click of a mouse for several disease conditions requiring short-term or long-term therapy [3]. Studies have showed that in Canada and the United States, more than 80% of the population have access to the internet [4], and about 72% of United State users seek information over the internet [5]. Studies have also showed that health information are among the most sought after topics on the internet [6], not necessarily as a means of replacing advice given by health professionals, but for validation of information given and to gather additional information [7]. Increasingly, patients and health professionals are turning to the internet for information pertaining health challenges and complicated therapies [8]. One of such health challenges is hypertension which is a major risk factor for Cardiovascular diseases (CVD) [9]. In a study conducted in 2013 at the Pew Research Center, 45% of adults in the United States reported that they live with one or more chronic conditions including hypertension, lung conditions, diabetes and heart disease [10]. Sixty-seven percent of those living with hypertension were internet users, and 58% of them accessed websites that provided information about a specific medical condition or problem [10]. Hypertension is a global public health burden that contributes to morbidity, mortality and healthcare cost in both developing and developed countries [11]. Approximately 25% of hypertensive patients worldwide are taking ARBs, and about 20 million people worldwide take ARBs for both hypertension and other cardiovascular conditions [12]. The effectiveness and safety of ARBs are well established [13]. However, its use has been associated with a number of adverse drug events, especially in elderly patients. Providing quality patient information and education is therefore important for safe and effective hypertension management. Despite its potential as a significant patient information resource, the internet's usefulness is often limited by the challenges associated with finding good quality information that comes from reliable and authentic sources [14]. Previous studies have reported that more than half of websites provide poor quality health information [15]. Health professionals are therefore constantly faced with patients who have been informed or misinformed by the internet [16]. As a result, clinicians, researchers, and consumers are concerned about the quality and accuracy of online health information and it is essential to assess the validity [17]. As much as disseminating health information on the internet can help improve knowledge transfer from health professionals to the population and help individuals maintain and improve their health, the rapid development of medical information on the Internet has its shortcomings which include: 1) uneven quality of medical information available on the internet [18]; 2) difficulty in reading and understanding this information due to use of technical language [19]; 3) potential dangers related to its erroneous and unsuitable use [20]. In addition to addressing the issue of quality, accuracy and reliability, it is also imperative to systematically assess the presentation of online health information using readability algorithms to ensure that such information is easily read and understood. We therefore aimed to assess the quality and readability of Internet-based health information related to select Angiotensin Receptor Blockers (ARBs).

 

 

Search strategy

 

Identification and selection of websites

 

We identified the three most widely used Angiotensin Receptor Blockers from published literature: losartan, irbesartan and valsartan [21]. Each of these key terms ‘losartan', ‘irbesartan', ‘valsartan' and ‘patient education' were entered into three different search engines (Google, Yahoo, Bing) chosen based on their widespread use [22], in March 2017. Only the first 50 links reported by each search engine were identified, as studies have showed that web users only visit the top 10 websites listed in search results [23].

 

Websites were included in the research if they were in English Language and free to access, and if they provided patient information relating to any of the search terms. Websites designed for marketing purposes evident by presence of diverse advertisements were excluded [24]. News feeds, video feeds, abstract listings and duplicate websites were also excluded. Additionally, websites for co-formulated ARB products were excluded. Quality of identified sites was evaluated using the DISCERN questionnaire, while readability was assessed using the SMOG calculator and Flesh-kincaid algorithm.

 

Assessment of quality of information using DISCERN instrument

 

The DISCERN questionnaire is a valid instrument that was designed to enable patients (or health consumers) and health care providers to assess the quality of health information. This questionnaire was developed based on the input of an expert panel, health information providers and representative of patient population [25]. The DISCERN tool is freely accessible online and a downloadable instruction handbook is available from the DISCERN handbook [26]. It is suitable for anyone who utilizes or produces information about treatment choices. Its uses include an aid for individual consumers who are making decisions about treatment, a screening tool for health information providers, a checklist for authors and producers of written consumer health information, a training tool for health professionals to improve communication and shared decision-making skills. The DISCERN questionnaire, consisting of 16 questions, was used to evaluate the quality of information on the selected websites. These questions are organized into three sections [26]. Section 1 (questions 1 to 8) addressed the reliability of the publication and helps in considering whether it can be trusted as a source of information. Section 2 (questions 9 to 15) focused on the specific details of the information about treatment choices while Section 3 (question 16) assessed the overall quality rating. Each question was rated on a 5-point scale ranging from "no" to "yes". A score of "5" indicates a definite yes; a score of "2-4" indicates that the publication meets the criterion in the question to an extent; while a score of "1" indicates a definite no.

 

Readability assessment

 

It is generally accepted that in evaluating the readability scores of written information, the use of more than one readability formula improves the reliability of the readability scores [27], hence two readability formulae; SMOG formula [28], and Flesch-Kincaid algorithm [29] were used in this study. The Smog readability formula was created by G Harry McLaughlin in 1969 and it estimates the years of education a person needs to understand a piece of writing [28]. SMOG readability grades were measured using the manual SMOG formula [28]. The SMOG was then calculated as described by McLaughlin et al. [28]. The readability scores were calculated using Microsoft Office Word. The Flesch Reading Ease test rates the text on the given website on a 100-point scale. The higher the score, the easier it is to understand the document. The Flesch-Kincaid Grade Level test rates a text on a United States school grade level. A score of 8.0, for example, means that an eighth grader can understand the document.

 

Statistical analysis: standard data entry and analysis were done using a Microsoft Excel spreadsheet (edition 2013). Inter-class Correlation Coefficient (ICC) value was calculated using the Statistical Package for Social Sciences, version 23 (IBM, Armonk, New York, US).

 

 

A total of 450 websites were reviewed between April 2017 and September 2017. A total of 150 websites for each of the three search engines (Google, Yahoo and Bing). Removal of 363 duplicate websites left a total of 87 unique sites. Thirty-seven out of the 87 unique websites met the inclusion criteria and were eligible to be evaluated. The remaining 50 websites were excluded as 11 were not freely accessible, 2 had broken link and 37 did not provide information useful for patient education. The authors further streamlined the search results leaving only websites with patient information on losartan and valsartan considering they are the most frequently prescribed ARBs as reported in the literature [30]. This left a total of 24 websites for evaluation. The Inter-class reliability for losartan and valsartan was calculated to be 0.804 and 0.695, respectively (Table 1).

 

Assessment of quality of websites

 

From the evaluation conducted using the DISCERN tool, none of the websites scored 5 for question 16 (the overall quality rating at the end of the questionnaire), 15% were rated as good, 66% as moderate, while 19% were rated as poor. The overall mean DISCERN score was 2.99 (SD±1.05). The mean quality rating across the websites is shown in Figure 1. From a maximum score of 5, the mean score for the questions 1 to 8 that addressed reliability was 2.82 while for questions 9 to 15 that focused on information regarding treatment choice was 3.20. The questions with the highest response score were as follows: "Does it describe risk of each treatment?", "Does it describe the benefit of each treatment?", "Does it provide support for shared decision-making?" and "is it relevant?" On the other hand, the lower scoring questions are: "Is it clear what sources of information are used to compile the publication?", "Does it describe what would happen if no treatment is used?" and "Does it describe how the treatment choice affects the overall quality of life?" The highest-rated websites according to our analysis are Mayo Clinic (Valsartan) 3.84 and Patient Info (Losartan) 3.84 [31, 32].

 

Readability assessment

 

Twenty-four websites were assessed for readability Table 1. The mean Flesch Reading Ease Score (FRES) was 48.87 (SD±16.12), while the mean Flesch Kincaid Reading Grade Level (FKRGL) was 9.29 (SD±1.98). The mean SMOG value was calculated to be 11.29 (SD±1.70).

 

 

The internet has the potential to provide patients and health professionals with health information, but with the increase in use, concerns arise as to the quality, reliability and readability of the information obtained from the internet [33]. In addressing these concerns, several solutions have been proposed. These solutions have included electronic filtering of web-based information, creation of ethical codes of conduct for providers of web-based information (currently done on voluntary basis) and assessment of websites by health professionals [33]. Introduction of clear critical appraisal tools with standardized website evaluation systems also appears to be useful in improving the ability of users to differentiate between trustworthy sites and inadequate ones [33].

 

This study is the first to have systematically evaluated the quality and readability of internet-based information related the two mostly prescribed ARBs. The results from this investigation provide some insights regarding online health information related to the ARBs. The overall mean DISCERN score for the 26 websites was 2.99 (SD±1.05). This suggests that information on ARBs on most of the evaluated websites was of moderate quality. This analysis is consistent with findings from previous studies which have evaluated the quality of health information on the Internet for a range of different chronic diseases (Table 2) [15, 34-36].

 

With respect to the quality of health information, most of the websites assessed scored reasonably well in describing the benefits of treatment as well as the risks of using ARBs. Most websites evaluated were also balanced and unbiased, and providing support for shared decision making between patients and physician. This is in agreement with the DISCERN criterion which suggests that a good quality health information resource should include issues for patients to discuss with their healthcare practitioners. However, most of the websites fared poorly in identifying clear sources of information or references, as only five websites met this criterion. This is important because clear sources enable online users examine the credibility and reliability of information on the website or decide to seek further information.

 

There was consistently a lack of information on the likely consequence of no treatment and how treatment choices affect the overall quality of life. Hence, patients receiving ARB therapy would potentially have poor knowledge on this aspect of their therapy. From the present study, certain websites are Mayo Clinic (Valsartan) 3.84 and Patient Info (Losartan) 3.84, received the highest DISCERN score [31, 32]. These websites were seen to have clearly met a good number of the DISCERN criteria.

 

The Inter-class reliability determined in the study was 0.804 for losartan and 0.695 for valsartan. Ideally a value of 0.7 is considered optimal [37]. Therefore, the values gotten indicated a good level of consistency for quality rating measurements between independent raters. This study highlights that most of the websites had patient information that is potentially difficult to read, as most of the websites were written at readability grade levels higher than grade 8 (Table 3). Of note, the SMOG rating scale was observed to provide a measurement of 2-4 grade levels higher than the Flesch Kincaid grade Table 1. This is considered the result of variation between different measurement scales and is consistent with studies conducted by Wilson, 2009 [38]. However, for both scales, most websites consistently scored above "9" indicating the universality of the readability problem.

 

The suboptimal readability observed in the present study is consistent with those of the broader literature. Estrada et al., reported that patient education materials related to the use of anticoagulants were written at grade levels beyond the comprehension of most patients [39]. Hutchinson et al., have also evaluated websites containing medical information pertaining to nine common general medicine diagnoses, and have reported similar readability levels beyond grade 8 for many of the websites evaluated [40]. Our finding is particularly relevant because hypertension is common in the elderly, and its prevalence increases with aging which has been associated with decline cognitive function [41]. As such, patients on ARBs might find it difficult to read and understand ambiguous online materials. Hence, patient information on these sites should be written at approximately school grade 8 or less to enhance easy understanding. The Institute of Healthcare Improvement recommends using simpler words, shorter sentences and avoiding medical jargon, all of which are important considerations in the provision of online health information for public consumption [42].

 

Limitations

 

The present study is not without a number of limitations. Only the first 50 search results per search engine were reviewed, and only the two mostly prescribed ARBs were assessed. Further, only websites written in English were evaluated, hence, findings may not be applicable to other languages. The Flesch Kincaid readability tool used may have overestimated the required readability scores as a list of polysyllabic clinical and medical terms such as ‘angiotensin' and ‘aldosterone' may be regarded as one sentence. Finally, readability formulae used was limited by the lay out of the website page. Websites vary dramatically in how they present information separately from the text that is written. Despite these limitations, the present study provides evidence on suboptimal readability and the need for quality improvement for online ARB patient education material.

 

 

While the overall quality of health information on the selected ARBs is moderate, the websites were found to possess a suboptimal readability score. Future content developers of ARB patient education material should ensure that health information is of favorable quality and easy to read by patients with varying degree of health literacy.

 

 

The authors declare no competing interests.

 

 

KO, AO and SN conceived the idea and were involved in the design of the study. AO collected the data and together with KO and SN participated in data analysis and interpretation. AO did the drafting of the paper while KO and SN critically revised the manuscript. All authors have read and agreed to the final manuscript.

 

 

Table 1: readability scores of ARB websites (N=24)

Table 2: comparison of different DISCERN scores for different studies

Table 3: category breakdown of readability scores of ARB websites (n = 24)

Figure 1: mean quality rating of websites using the DISCERN instrument

 

 

1. Fahy E, Hardikar R, Fox A, Mackay S. Quality of patient health information on the Internet: reviewing a complex and evolving landscape. The Australasian Medical Journal. 2014;7(1):24. PubMed | Google Scholar


2. Lacher D, Nelson E, Bylsma W, Spena R. Computer use and needs of internists: a survey of members of the American College of Physicians-American Society of Internal Medicine. Proceedings of the AMIA Symposium; 2000: American Medical Informatics Association.


3. Kiran DS, Bargale S, Pandya P, Bhatt K, Barad N, Shah N et al. Evaluation of Health on the Net seal label and DISCERN as content quality indicators for patients seeking information about thumb sucking habit. Journal of pharmacy & bioallied sciences. 2015;7(Suppl 2):S481. PubMed | Google Scholar


4. Centre PR. Global Attitudes & Trends. 2016. Accessed on February 17 2019


5. Zickuhr K. Who's not online and why. 2013.


6. McMullan M. Patients using the Internet to obtain health information: how this affects the patient–health professional relationship. Patient education and counseling. 2006;63(1-2):24-8. PubMed | Google Scholar


7. Nicholas D, Huntington P, Gunter B, Withey R, Russell C. The British and their use of the web for health information and advice: a survey. Aslib Proceedings; 2003: MCB UP Ltd. Google Scholar


8. Nasser S, Mullan J, Bajorek B. Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. The Australasian Medical Journal. 2012;5(3):194. PubMed | Google Scholar


9. Ong KL, Cheung BM, Man YB, Lau CP, Lam KS. Prevalence, awareness, treatment, and control of hypertension among United States adults 1999–2004. Hypertension. 2007;49(1):69-75. PubMed | Google Scholar


10. Pew Research Centre. Chronic disease and the internet. 2010. Accessed on February 17 2019


11. Kearney PM, Whelton M, Reynolds K, Muntner P, Whelton PK, He J. Global burden of hypertension: analysis of worldwide data. The lancet. 2005;365(9455):217-23. PubMed | Google Scholar


12. Volpe M, Azizi M, Danser AJ, Nguyen G, Ruilope LM. Twisting arms to angiotensin receptor blockers/antagonists: the turn of cancer. European heart journal. 2010;32(1):19-22. PubMed | Google Scholar


13. Corrao G, Zambon A, Parodi A, Poluzzi E, Baldi I, Merlino L et al. Discontinuation of and changes in drug therapy for hypertension among newly-treated patients: a population-based study in Italy. Journal of hypertension. 2008;26(4):819-24. PubMed | Google Scholar


14. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. Jama. 2001;285(20):2612-21. PubMed | Google Scholar


15. van der Marel S, Duijvestein M, Hardwick JC, van den Brink GR, Veenendaal R, Hommes DW et al. Quality of web-based information on inflammatory bowel diseases. Inflammatory bowel diseases. 2009;15(12):1891-6. PubMed | Google Scholar


16. Kaba R, Sooriakumaran P. The evolution of the doctor-patient relationship. International Journal of Surgery. 2007;5(1):57-65. PubMed | Google Scholar


17. Price SL, Hersh WR. Filtering web pages for quality indicators: an empirical approach to finding high quality consumer health information on the world wide web. Proceedings of the AMIA Symposium; 1999: American Medical Informatics Association. 1999;911-915. Google Scholar


18. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ. 1997;314(7098):1875. PubMed | Google Scholar


19. Cline RJ, Haynes KM. Consumer health information seeking on the internet: the state of the art. Health education research. 2001;16(6):671-92. PubMed | Google Scholar


20. Allain P, Schuck S, Beaufils C, Zekri O, Ganilsy M, Allain H. Drugs on the Internet. Hazards of public health. Presse medicale (Paris, France: 1983). 1998;27(3):117-21. PubMed | Google Scholar


21. Hudson M, Humphries K, Tu JV, Behlouli H, Sheppard R, Pilote L. Angiotensin II receptor blockers for the treatment of heart failure: a class effect? Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2007;27(4):526-34. Google Scholar


22. Search Engine Watch. Nielsen netratings search engine ratings. 2016. Accessed on February 17 2019


23. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324(7337):573-7. PubMed | Google Scholar


24. Moody E, Clemens K, Storsley L, Waterman A, Parikh C, Garg A et al. Improving on-line information for potential living kidney donors. Kidney international. 2007;71(10):1062-70. PubMed | Google Scholar


25. Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R et al. Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites. Patient education and counseling. 2009;77(1):33-7. PubMed | Google Scholar


26. Charnock D. The DISCERN handbook. Quality criteria for consumer health information on treatment choices Radcliffe: University of Oxford and The British Library. 1998.


27. Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Education & Behavior. 2006;33(3):352-73. PubMed | Google Scholar


28. McLaughlin G. SMOG: Simple measure of gobbledygook. 2008.


29. Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975. Google Scholar


30. Brenner BM, Cooper ME, De Zeeuw D, Keane WF, Mitch WE, Parving H-H et al. Effects of losartan on renal and cardiovascular outcomes in patients with type 2 diabetes and nephropathy. New England Journal of Medicine. 2001;345(12):861-9. PubMed | Google Scholar


31. Mayo Clinic. Drugs and Supplements Valsartan (Oral Route). 2019. Accessed on February 17 2019


32. Patient. Losartan - an angiotensin receptor blocker. 2018.


33. Batchelor JM, Ohya Y. Use of the DISCERN instrument by patients and health professionals to assess information resources on treatments for asthma and atopic dermatitis. Allergology International. 2009;58(1):141-5. PubMed | Google Scholar


34. Chong YM, Fraval A, Chandrananth J, Plunkett V, Tran P. Assessment of the quality of web-based information on bunions. Foot & ankle international. 2013;34(8):1134-9. PubMed | Google Scholar


35. Saithna A, Ajayi O, Davis E. The quality of internet sites providing information relating to hip resurfacing. The Surgeon. 2008;6(2):85-7. PubMed | Google Scholar


36. Tallgren M, Bäcklund M. Patient information about general anaesthesia on the internet. Anaesthesia. 2009;64(4):408-15. PubMed | Google Scholar


37. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological assessment. 1994;6(4):284. Google Scholar


38. Wilson M. Readability and patient education materials used for low-income populations. Clinical Nurse Specialist. 2009;23(1):33-40. PubMed | Google Scholar


39. Estrada CA, Hryniewicz MM, Higgs VB, Collins C, Byrd JC. Anticoagulant patient information material is written at high readability levels. Stroke. 2000;31(12):2966-70. PubMed | Google Scholar


40. Hutchinson N, Baird GL, Garg M. Examining the reading level of internet medical information for common internal medicine diagnoses. The American journal of medicine. 2016;129(6):637-9. PubMed | Google Scholar


41. Cohen DL, Townsend RR. Update on pathophysiology and treatment of hypertension in the elderly. Current hypertension reports. 2011;13(5):330-7. PubMed | Google Scholar


42. Institute for Healcare Improvement. Health communications toolkits: improving readability of patient education materials. 2019. Accessed on February 17 2019