Accuracy and precision of four main glucometers used in a Sub-Saharan African Country: a cross-sectional study

Accuracy and precision of four main glucometers used in a Sub-Saharan African Country: a cross-sectional study

Simeon-Pierre Choukem^1,2,3,&, Colette Sih^1,2, Daniel Nebongo², Philomene Tientcheu⁴, André-Pascal Kengne⁵

 

¹Department of Internal Medicine and Paediatrics, Faculty of Health Sciences, University of Buea, Buea, Cameroon, ²Health and Human Development (2HD) Research Network, Douala, Cameroon, ³Diabetes and Endocrine Unit, Department of Internal Medicine, Douala General Hospital, Douala, Cameroon, ⁴UNILABO Biomedical Laboratory, Douala, Cameroon, ⁵South African Medical Research Council and University of Cape Town, Cape Town, South Africa

 

 

^&Corresponding author
Simeon-Pierre Choukem, Department of Internal Medicine and Paediatrics, Faculty of Health Sciences, University of Buea, Buea, Cameroon

 

 

 

 

The growing diabetes epidemics worldwide and in sub-Saharan Africa (SSA) poses diagnostic and management challenges [1]. Self-measurement and monitoring of blood glucose (SMBG) using point-of-care (POC) glucometers is an essential component of diabetic management and follow-up [2]. There is strong evidence that rigorous glycaemic control can in the long-run cost-effectively reduce the complications associated with type 2 diabetes [3]. The American Diabetes Association (ADA) recommends and promotes SMBG using glucometers in order to allow patients living with diabetes to achieve and maintain desired glycaemic targets [4]. Glucometers are very useful in timely diagnosis of acute diabetes complications, especially hypoglycaemia and hyperglycaemia which can contribute to higher morbidity and mortality [5]. Owing to the wide acceptance and use of POC glucometers in different settings, there is a competition-driven development by manufacturers in both meter and test strips technology. These innovations have allowed for greater reliability of results when compared to the reference laboratory method. Nonetheless, there are differences in performance across these monitoring devices. These have prompted the need to develop standard performance guidelines to guide manufacturers on the minimum requirements for these glucometers [6-8]. The revised performance guidelines published by the International Standardisation Organisation (ISO) 15197:2013 stipulate in the new criteria that for glucose values above 100mg/dL, the accuracy is expected to increase from ±20% to ±15% of the reference value; and this shall apply to at least 95% of results [6]. Many recent studies have highlighted the limitations of most glucometers in meeting these recommendations [9-11], very few of which were carried out in SSA [12]. Because of limited resources in most SSA countries, glucometers are used beyond their intended purpose, including for diabetes diagnosis for instance. Inaccurate glucometers would therefore negatively impact on decision making and downstream consequences more in SSA than elsewhere. Furthermore, uncontrolled competition and weak or inexistent standard quality control procedures are likely to favour the marketing of sub-standards glucometers in SSA than anywhere else. Lastly, it has been shown that climate conditions can affect the performance of glucometers [10]. In the current study, we assessed the technical and clinical accuracy and precision of glucometers commonly used in Cameroon.

 

 

Study design, setting and population: in this cross-sectional study carried out from 1^st November 2014 to 28 February 2015, participants were recruited at the Douala General Hospital (DGH), whilst laboratory analysis of venous plasma glucose was carried out at UNILABO^® biomedical laboratory in Douala. Douala is the economic capital of Cameroon. The Douala General Hospital is a reference health institution, which provides a wide array of services, including a diabetic clinic. Diabetic patients attending the diabetic clinic at the Douala General Hospital as well as healthy volunteers (invited by advertisement) who provided written informed consent were eligible to participate. Participants with the following characteristics were excluded: hypotension with a systolic blood pressure of less than 90 mmHg, clinical anaemia, history of gout, history of jaundice and current anticoagulant therapy. Ethical approval was obtained from the Institutional Review Board of the Faculty of Health Sciences, University of Buea. Administrative clearance was obtained from the Director of the Douala General Hospital. All participants provided written informed consent.

 

Selection of glucometers: a survey of glucometers commercially available in Cameroon was conducted by visiting 5 randomly selected pharmacies in each of the ten regional headquarters of the country. Glucometers with complete system components were eligible to be included in the study. We excluded 6 glucometers for which pharmacies did not have sufficient number of strips (at least 150) per glucometer required for the study. The 4 glucometers finally evaluated were randomly purchased in the cities of Bamenda, Buea, Douala and Yaoundé: Accu-Chek^® Active (Roche Diagnostics GmbH, Mannheim, Germany), CodeFree™ (SD Biosensor Inc., South Korea), Mylife™Pura™ (Ypsomed AG, Burgdorf, Switzerland) and OneTouch^®Ultra^®2 (LifeScan Inc, USA).

 

Study procedure and data collection: for each consenting eligible participant, data were collected on age, sex and diabetes status. Participants were asked to wash their hands with tap water and neutral soap. A capillary sample was obtained from the middle finger. Blood was dropped in a systematic manner on the test strips of each glucometer and the results recorded. The glucometers were rotated so that no glucometer always occupied the same position. Concomitantly, venous whole blood samples were collected from the right or left forearm into 4mL sodium fluoride/oxalate tubes. The tubes were then appropriately labelled and stored at room temperature, transported to the laboratory for plasma glucose measurement within 120 minutes. In the laboratory, samples were centrifuged and venous plasma glucose obtained using the hexokinase enzymatic method in a Roche-Hitachi Cobas C111^® analyser (Roche Diagnostics GmbH, Mannheim, Germany; Hitachi High-Technology Corporation, Tokyo, Japan). The laboratory value was used as a reference for comparison. Every morning, calibration of the automated analyser was carried out by the same laboratory technician. Calibration of the glucometers using test strips was carried out daily. To avoid any bias, neither the technician doing laboratory analysis, nor the investigator carrying out measurements with glucometers was aware of each other results until the end of the study. Finally, we carried out repeated measurements of glucose concentrations on standard solutions of varying concentrations on each glucometer over one day. The standard solutions were prepared at the Chemistry Laboratory of the Faculty of Sciences, University of Buea. The concentrations of the standards were: 100 mg/dL, 200 mg/dL and 300 mg/dL. Each of the standard solutions was tested five times on each glucometer.

 

Data analysis: data analysis was carried out using the R statistical software V.3.1.1 (The R Foundation for Statistical Computing, Vienna, Austria). Continuous variables are reported as mean (and standard deviation, SD) or median (and 25^th-75^th percentiles). Correlation between reference laboratory and each glucometer measurement was determined via covariance estimation for multivariate t distribution, which is known to be robust to the effect of outliers. Robust linear regressions methods were then used to derive the regression coefficients for the regression curve for predicting the reference values from the glucometer values. Paired-sample t-test was used to compare the mean difference between reference values and glucometer values; then the percentage bias was calculated as follows: (glucometer reading-reference value)×100÷ reference value [13]. This was compared to the American Diabetes Association (ADA) standard of a bias of < 5% being acceptable [14, 15]. The agreement between the two measurements (that is, the glucometer and reference readings) at any given level was then examined using Bland-Altman plots [16]. Glycaemic values were also evaluated in the light of ISO 2013 requirements for accuracy. A p value of < 0.05 was considered statistically significant. To assess the clinical accuracy, we used the total error allowable (TEa) and the Parke's consensus error grid analyses. The TEa was used to determine the clinical significance of differences observed between glucometer results and the reference. The mean of each glucometer reading was compared with the reference laboratory value and should be within clinical range of reference mean±TEa. Parke's error grid analysis is based on the comparison of clinical consequences of using the POC glucometer testing vs. the reference method. The analysis plots the glucometer-measured glucose against reference glucose level, into 5 zones: zone A (results given by the glucometer allow for clinically correct management decisions to be taken either in the hypoglycaemic or hyperglycaemic range), zone B (there is a deviation of >20% of glucometer results from the reference method. It represents values that would lead to benign or no treatment error.), zone C (results given by the glucometer would begin to lead to treatment decisions opposite to those based on reference blood glucose levels), zone D (results given by the glucometer lead to a failure to detect and treat errors) and zone E (glucometer-generated results fail to identify hypoglycaemia or hyperglycaemia. Values given by the glucometers are opposite to the reference values resulting in corresponding treatment decisions opposite to those needed) [15, 17, 18]. For perfect accuracy, 95% of values should be in zone A, 5% in zone B, and 0% in other zones. The precision of glucometers was determined by the calculation of the coefficients of variation (CV) that was further compared to the ISO criteria. The CV was computed as the standard deviation divided by the mean and expressed as a percentage [13]. A CV of less than 5% was considered as being precise [19].

 

 

General characteristics of participants: of the 108 study participants (53.7% women) included, six had missing readings for at least one glucometer. Therefore, only values from 102 participants with valid results recorded on all four glucometers were included in the analyses where the four glucometers were compared. The 108 participants' age ranged from 19 years to 81. The mean (standard deviation, SD) age was 46.2 (15.6) years. Participants with diabetes represented 68.5% (74.1% of women, 62.0% of male).

 

Technical accuracy of glucometers: the values provided by the glucometers were significantly higher than the reference; the mean differences and mean biases between glucometer results and the reference varied from +12.0 and 13.8% respectively for OneTouch^® Ultra^®2 to +26.5 and 29.1% respectively for CodeFreeTM (Table 1). There was a significant positive correlation between each glucometer and the reference values, with correlation coefficients ranging from 0.87 (95%CI 0.81-0.91) for CodeFree^TM and 0.97 (0.95-0.98) for Mylife™ Pura™ (Table 2). For all glucometers, the correlation coefficients were always higher in men than women (all p < 0.01 for correlation coefficient comparisons; The linear regression allowed determination of the equation linking the reference blood glucose to each glucometer's blood glucose (Table 2). According to the ISO recommendations of 2013, 95% of values are expected to fall within the confidence limit for agreement in the Bland-Altman plots to be considered as being acceptable. Bland-Altman plots showed 5 outliers (4.9%) for Accu-Chek^® Active with 4 in the positive region and 1 in the negative region (Figure 1A), 2 outliers (1.96%) for CodeFree^TM both being in the positive region (Figure 1B), 1 outlier (0.98%) in the negative region for Mylife™ Pura™ (Figure 1C). Hence, there was a good agreement between the values generated by the reference method and these glucometers, whereas the agreement was lower for OneTouch^® Ultra^®2 as there were 6 outliers (5.88%), with 4 in the positive region and 2 in the negative region (Figure 1D). Based on ISO 2013 recommendations, no glucometer met the criteria for the required level accuracy of 99%. The lowest proportion was with CodeFree™ in the range of glycemia < 100mg/L and Accu-Chek^® Active in the range of glycemia > 100mg/dL, whereas Mylife™ Pura™ yielded the highest proportion within the two ranges (Table 3).

 

Clinical accuracy of glucometers: all differences between the reference value and those provided by the various glucometers were both statistically significant (p < 0.0001) as well as clinically significant, without any gender difference. According to Parke's consensus error grid analysis, Accu-Chek^® Active had 99% (n = 101) of its values within zones A and B and 1% (n=1) in zone C (Figure 2A), CodeFree™ had 93.1% (n=95) of its values in zone A and B, 4.9% (n=5) in zone C and 2% (n=2) in zone D (Figure 2B). Mylife™ Pura^TMhad 100% of its values in zone A and B (Figure 2C) and OneTouch^® Ultra^®2 had 98.0% (n=100) in zone A and B and 2% (n=2) in zone C (Figure 2D). Based on ISO recommendations of 2013, Accu-Chek^® Active and Mylife™ Pura™ met the criteria, with 99 to 100% of their values found within zones A and B; OneTouch^® Ultra^®2 (98%) was close to the normal limit, whereas CodeFree^TM(93.1%) fell short of the recommendations.

 

Precision of glucometers ACCU-CHEK^® Active provided an acceptable precision (CV < 5%) only for the high standard concentrations, whereas CodeFree™ Mylife™ Pura™ and OneTouch^® Ultra^®2 all had acceptable level of precision at all concentrations of standard solutions.

 

 

We have shown in this study that none of the glucometers tested consistently met all the international recommendations set by the ISO and the ADA for technical accuracy. Although none met the criteria for clinical accuracy based on TEa, Accu-Chek^® Active and MylifeTMPuraTM met the ISO 2013 requirements based on Parke's consensus error grid. All glucometers except Accu-Chek^® Active showed a satisfactory level of precision at all concentrations of standard solutions used. Many other studies have also reported that not all glucometers meet the requirements. In a study carried out in South Africa in 2009 [9] five glucometers were assessed: GlucoPlus^TM (Diabcare), OneTouch^® Ultra^TM (LifeScan Inc, Johnson & Johnson), OneTouch^® Horizon^TM (Johnson & Johnson), Accu-Chek^® Active (Roche) and Accu-Chek^® Advantage (Roche) on 115 diabetic patients. Generally, only three glucometers (GlucoPlus^TM, OneTouch^® Horizon^TM and Accu-Chek^® Active) met the ISO 2003 guidelines whereas none met the ADA requirements. Most other studies that included similar glucometers (Accu-Chek^® Active and OneTouch^® Ultra^®2) have yielded results with some similarities, but also discrepancies with our findings at various levels of accuracy [20-22]. We have not found published studies that have evaluated CodeFree^®. Amongst the four glucometers evaluated in our study, two (ACCU-CHEK^® Active and mylife^TMPura^TM) had the ConformitéEuropéenne (CE) label in 2010 [23]. In the study by Freckmann et al the conformity of 27 glucometers bearing the CE label was evaluated using 100 blood samples, with a defined distribution of blood glucose concentrations ranging from 20 mg/dL to 600 mg/dL [23]. Only 16 out of the 27 glucometers met the minimum ISO 2003 requirements. Amongst these were Accu-Chek^® Active and Mylife^TMPura^TM, with 100% of the values provided by both glucometers respecting the ISO 2003 recommendations. They used similar methods in 2012 [24] to assess 43 glucometers with the CE label, using 100 capillary samples spanning across concentrations of 50 to 400 mg/dL. Amongst these, only 34 were completely evaluated and 27 showed an acceptable level of accuracy based on ISO 2003 recommendations, amongst which Accu-Chek^® Active and Mylife^TMPura^TM. The lower accuracy observed in our study suggests the potential alteration of glucometers or strips by climate conditions after a long stay in SSA, as suggested earlier [10].

 

Also, whether glucometers of lower standards are manufactured and marketed in our environment in the absence of rigorous quality control standards is unknown. Even though POC glucometers are not intended for screening and diagnosis of diabetes based on guidelines provided by the FDA, [7, 8] these devices are nonetheless used for screening and diagnosis of diabetes in SSA. All the glucometers assessed during our study overestimated the blood glucose concentration. Hence, the clinical implication of using any of these glucometers in the screening and diagnosis of diabetes is that they correctly identify patients with diabetes, but misdiagnose individuals with borderline normal or impaired fasting glucose as having impaired fasting glucose or diabetes, respectively. With regards to the use of the glucometers evaluated in our study as tools for monitoring, there might be a greater risk of iatrogenic hypoglycaemia especially with CodeFree^TM, because they may result in the use of higher doses of hypoglycaemic agents. Also, the glucometers may fail to identify hypoglycaemia due to overestimation of blood glucose levels. Regarding the statistically significant difference between women and men in the correlation between reference values and values provided by all glucometers, we speculate that they may be due to the influence of body creams and lotions used by Cameroonian women, which may contain contaminants that interact with capillary glucose measurement [25]. The following potential limitations of our study should be considered when interpreting and generalizing our findings. Firstly, ISO recommendations were meant for self-testing. Hence, the assessment of the accuracy by a medically trained person can potentially overestimate the accuracy of the glucometers when compared to self-testing. Secondly, although all participants carefully washed their hands before capillary glucose measurement, all potential interferences to glucometer readings may not have been eliminated. Also, the investigator performing capillary glucose measurements was not blinded to the meter being used. Nevertheless, the reference values were made available only after the completion of all glucometer-based glucose determination. Finally, measurements in the hypoglycaemic range could not be carried out; hence, performance of the glucometers could not be assessed at all glycaemic ranges. Despite these potential limitations and unlike other studies that have been carried out on glucometers, we used very exhaustive analytical methods and endeavoured to compare the findings of our study to recommendations of more than one international body.

 

 

In conclusion, based on the extensive comparisons conducted in this study, no glucometer met all the required international recommendations; hence, all glucometers should always be used with caution, especially if their results mandate serious clinical decision. However, Mylife^TMPura^TM would be the best recommended glucometer in Cameroon for screening and follow-up of diabetes. Despite its satisfactory precision, CodeFree^® seems not to be recommended because it showed consistent technical and clinical inaccuracies.

 

 

The authors declare no competing interests.

 

 

Simeon-Pierre Choukem: conception and design of the study, data collection, data interpretation, drafting and review of the manuscript. Colette Sih: conception and design of the study, data collection, data interpretation, review of the manuscript. Daniel Nebongo: conception and design of the study, data collection, review of the manuscript. PhilomeneTientcheu: Conception and design of the study, editing and review of the manuscript. André-Pascal Kengne: conception and design of the study, data analysis and interpretation, review of the manuscript. All authors revised and approved the final version of the manuscript.

 

 

The authors are grateful to participants in this study. The 2HD Research Network is supported by a Cruddas Link Fellowship to SPC (Tseu Medical Institute, Harris Manchester College, University of Oxford, UK).

 

 

Table 1: absolute difference between each glucometer and reference method results

Table 2: correlations between the reference value and glucometer-based values and regression equations

Table 3: proportion of values respecting ISO 2013 criteria for each glucometer

Figure 1: Bland-Altman plots for agreement between glucometers; (A) Accu-Chek® Active; (B) CodeFree^TM; (C) Mylife^TMPura^TM; (D) OneTouch® Ultra®2) and the reference method, glucometer values minus reference value for each participant (y-axis) is plotted against the mean of the two measurements (x-axis), and are represented by the black dots; the horizontal doted blue line through zero is the line of perfect agreement between the two measurements; the parallel solid black line is the mean bias; the green colour band around the solid black line is the standard error around the mean bias estimates, while the shaded area represents the 95% confidence intervals; the linear curve of best fit is also shown (broken oblique blue line)

Figure 2: Parke’s consensus error grid analysis for glucometers; (A) Accu-Chek® Active; (B) CodeFree^TM; (C): MylifeTMPura^TM; (D): OneTouch® Ultra®2), Zone A: results give by the glucometer are clinically accurate within+/-20% of the reference; zone B: the error of the results given by the glucometer is above 20%, but would lead to minor or no treatment error; zone C: results given by the glucometer would begin to lead to treatment decisions opposite to that called for by the real blood glucose levels; zone D: results given by the glucometer lead to a failure to detect and treat errors; zone E: results given by the glucometer lead to erroneous treatment of hypo or hyperglycemia

 

 

1. International Diabetes Federation. IDF Diabetes Atlas, 8^th edn. Brussels, Belgium: International Diabetes Federation 2014. Accessed on 6 Sept 2016.


2. Solnica B, Naskalski JW, Sieradzki J. Analytical performance of glucometers used for routine glucose self-monitoring of diabetic patients. Clin Chim Acta Int J ClinChem. 2003; 331(1,2): 29-35. PubMed | Google Scholar


3. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998; 352(9131): 837-53. PubMed | Google Scholar


4. Association AD. Standards of medical care in diabetes-2016: Summary of Revisions. Diabetes Care. 2016 Jan; 39 Suppl 1: S4-5. PubMed | Google Scholar


5. Gray A, Raikou M, McGuire A et al. Cost effectiveness of an intensive blood glucose control policy in patients with type 2 diabetes: economic analysis alongside randomised controlled trial (UKPDS 41), United Kingdom Prospective Diabetes Study Group. BMJ. 2000 May 20; 320(7246): 1373-8. PubMed | Google Scholar


6. International Organization for Standardization. ISO 15197:2013-in vitro diagnostic test systems--Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. Accessed on 11 Sept 2016.


7. Denise N, Johnson-Lyles. FDA Perspective: FDA Evaluation of Point of Care Blood Glucose Meters 2014. Accessed on 11 Sept 2016.


8. Landree L. FDA Regulation of POC Blood Glucose Meters 2013. Accessed on 11 Sept 2016.


9. Essack Y, Hoffman M, Rensburg M et al. A comparison of five glucometers in South Africa. J Endocrinol Metab Diabetes South Afr. 2009; 14(2): 102-5. Google Scholar


10. Bimenya GS, Nzarubara GR, Kiconco J et al. The accuracy of self monitoring blood glucose meter systems in Kampala Uganda. Afr Health Sci. 2003 Apr; 3(1): 23-32. PubMed | Google Scholar


11. Sachse D, Bolstad N, Jonsson M et al. The Accu-Chek Mobile blood glucose monitoring system used under controlled conditions meets ISO 15197 standards in the hands of diabetes patients. Scand J Clin Lab Invest. 2012 Sep; 72(5): 374-9. PubMed | Google Scholar


12. Hawkes M, Conroy AL, Opoka RO et al. Performance of Point-of-Care Diagnostics for Glucose, Lactate, and Hemoglobin in the Management of Severe Malaria in a Resource-Constrained Hospital in Uganda. Am J Trop Med Hyg. 2014 Apr; 90(4): 605-8 Epub 2014 Mar 3. PubMed | Google Scholar


13. Harr KE, Flatland B, Nabity MB et al. AVSCP guidelines allowable total error guidelines for biochemistry. Vet Clin Pathol. 2013 Dec; 42(4): 424-36. PubMed | Google Scholar


14. Cohen M, Boyle E, Delaney C et al. A comparison of blood glucose meters in Australia. Diabetes Res Clin Pract. 2006 Feb; 71(2): 113-8 Epub 2005 Jul 11. PubMed | Google Scholar


15. Khan AI, Vasquez Y, Gray J et al. The variability of results between point-of-care testing glucose meters and the central laboratory analyzer. Arch Pathol Lab Med. 2006 Oct; 130(10): 1527-32. PubMed | Google Scholar


16. Myles PS, Cui J. Using the Bland-Altman method to measure agreement with repeated measures. Br J Anaesth. 2007 Sep; 99(3): 309-11. PubMed | Google Scholar


17. Tonyushkina K, Nichols JH. Glucose meters: a review of technical challenges to obtaining accurate results. J Diabetes Sci Technol. 2009 Jul1; 3(4): 971-80. PubMed | Google Scholar


18. Parkes JL, Slatin SL, Pardo S et al. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care. 2000 Aug; 23(8): 1143-8. PubMed | Google Scholar


19. Krouwer JS. Estimating total analytical error and its sources: techniques to improve method evaluation. Arch Pathol Lab Med. 1992 Jul; 116(7): 726-31. PubMed | Google Scholar


20. Ullal A, Parmar GM, Chauhan PH. Comparison of glucometers used in hospitals and in outpatient settings with the laboratory reference method in a tertiary care hospital in Mumbai. Indian J Endocrinol Metab. 2013 Dec; 17(Suppl 3): S688-93. PubMed | Google Scholar


21. Aghakachoei S, Zargarzadeh AH, Amini M. Comparison of Blood Glucose Values Using Two Glucose Meters and Standard Laboratory Method in Hospitalized Patients in a Teaching Hospital. Journal of Pharmaceutical Care. 2014; 2(1): 15-21. Google Scholar


22. Dhatt GS, Agarwal MM, Othman Y et al. Performance of the Roche Accu-Chek active glucose meter to screen for gestational diabetes mellitus using fasting capillary blood. Diabetes Technol Ther. 2011 Dec; 13(12): 1229-33 Epub 2011 Aug 24. PubMed | Google Scholar


23. Freckmann G, Baumstark A, Jendrike N et al. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther. 2010 Mar; 12(3): 221-31. PubMed | Google Scholar


24. Freckmann G1, Schmid C, Baumstark A, Pleus S, Link M, Haug C. System Accuracy Evaluation of 43 Blood Glucose Monitoring Systems for Self-Monitoring of Blood Glucose according to DIN EN ISO 15197. J Diabetes Sci Technol. 2012 Sep 1; 6(5): 1060-75. PubMed | Google Scholar


25. Bouché CH, Garnier JP, Choukem SP et al. Falsely elevated capillary glucose and ketone levels and use of skin lightening creams. BMJ. 2015 Jul 29; 351: h3879. PubMed | Google Scholar