Understanding evidence-based guidelines for diabetes: from drawing board to doctor’s office

Ruth Colagiuri

The Diabetes Unit, Australian Health Policy Institute, University of Sydney, Sydney, NSW, Australia

Abstract

Despite certain knowledge deficits about diabetes there is irrefutable evidence: (1) for intervening to prevent its development in high-risk individuals; and (2) about which treatments and processes of care lead to the best outcomes in people who already have diabetes.
Evidence-based guidelines are widely accepted as the most useful means of communicating current ‘best practice’ to policy-makers, clinicians and consumers. However, the processes by which they are derived are not generally well understood by clinicians and there are limitations to what guidelines can tell us, and some anomalies in guideline methodology.
This article describes the various steps in finding, appraising and synthesizing the evidence into guideline recommendations and discusses some limitations to guideline methodology. Notwithstanding these limitations, evidence- based guidelines have many benefits and are undoubtedly the best means available for summarizing the evidence about diabetes prevention and care for use by clinicians, people with diabetes and policy-makers.
One of the current difficulties for clinicians is the number of guidelines available for diabetes and it is expected the future will see diabetes guidelines rationalized into fewer but more widely used versions. Evidence-based guideline authorities in several countries are working to improve guideline methodology to accommodate a more comprehensive representation of the available evidence. Additionally, advances in translation and health services research will increasingly demonstrate what works best in getting evidence-based guideline recommendations into everyday practice.

Introduction

Given the extent and rigour of the evidence demonstrating which clinical practices and processes lead to the best outcomes, diabetes is an ideal subject for evidence-based guidelines. Diabetes evidence is well established in the area of secondary and tertiary prevention [1–8].
More recently, but no less conclusively, evidence has emerged demonstrating successful interventions for preventing or significantly delaying progression from ‘prediabetes’ to diabetes [9–11]. Recent advances in fine-tuning the evidence about successful diabetes management are exemplified by studies on multifactorial risk management [12]. Despite certain gaps in our knowledge about diabetes, these studies provide ample ‘fodder’ for evidence-based guidelines. Further, health economics is increasingly providing information from a variety of countries detailing the direct and indirect costs of diabetes [13–17] that can be used as a basis for future evaluations. However, although the diabetes evidence base is solid and the guideline methodology increasingly rigorous, many issues remain unresolved. There are between-country variations in criteria for including, assessing and grading evidence. There are also unresolved questions around the basis of clinical decision-making in the absence of definitive evidence; how to keep guidelines current and therefore optimally relevant; and how to handle clinical issues that do not lend themselves readily to randomized controlled trials (RCTs) to provide evidence.
The notion of evidence-based guidelines to promote ‘best practice’ became prominent in the mid-1990s. This concept has grown to a point where using research evidence, synthesized from the peer-reviewed medical press into guidelines, to inform clinical decision-making, clinical governance and resource allocation is now firmly entrenched in the psyche of health care funders and providers globally, irrespective of health care settings or available resources. While initial resistance from clinicians and criticisms of loss of professional autonomy, regulatory intrusion and ‘cookbook’ medicine [18, 19] have faded over time, not everyone has the same understanding of what guidelines mean. For example, how are they derived and constructed? To what extent can they assist clinical decision-making? What are their limitations? What is it that guidelines do not or cannot tell us? How can they best be interpreted and applied in routine practice? This article explores these issues, noting their relevance to, and implications for, diabetes care.

What constitutes a guideline?

The key purpose of guidelines is to make information widely available that links effective clinical treatments and care processes to optimal or desired outcomes. This is expected to reduce unacceptable variations in practice and improve quality of care and ultimately health outcomes [18]. Consequently, guidelines should reflect the ‘best available evidence’. While there are many definitions, the most widely used is by Field and Lohr [20]. This common sense definition has stood the test of time, having been developed some 15 years ago in the infancy of the evidence-based guideline trend: ‘Guidelines are systematically generated statements which are designed to assist health care clinicians and consumers to make informed decisions about appropriate treatment in specific circumstances.’ Note the word assist. Guidelines are just that — aides and guides. They are not blanket rules that are to be followed blindly. They tell us what works best in most people most of the time in a particular circumstance — no more, no less. They do not tell us what works best for everyone all of the time in all settings and circumstances. They must therefore be applied in the light of clinical experience and individual circumstances.
Consultation with and active involvement of the patient is an important factor in optimizing the uptake of guidelines [21]. The patient’s individual needs and circumstances must be factored into treatment decisions. Diabetes care providers know that treatment targets will depend on the patient’s lifecycle stage, where s/he is along the continuum of the diabetes disease process, and how these influences intersect and interact. For example, HbA1c targets and the treatments aimed at achieving them will be very different for a toddler with type 1 diabetes, a young women with gestational diabetes or a frail elderly person with type 2 diabetes.

How are evidence-based guidelines generated?

Finding and synthesizing the evidence is a rigorous process. Like any systematic research it requires meticulous attention to documenting the processes followed so that others who wish to replicate it would hopefully arrive at the same, or very similar, conclusions.
There are a number of highly credible current sources of diabetes guidelines [22–25]. These bodies also provide guidance on guideline methodology, either as part of the published guideline or as a separate publication. Additionally, between 1993 and 2000 the Journal of the American Medical Association produced an excellent series on identifying and applying the evidence [26], and the AGREE Collaboration (Appraisal of Guidelines Research and Evaluation) has developed specific tools for assessing the quality of guidelines [27]. Another good technical resource is a textbook, authored by J. Muir Gray [28], on how to apply research evidence to health policy and management decision-making.
On a simpler level, with the needs of developing countries in mind, the National diabetes programmes toolbox [29] offers an overview of considerations for guideline development and implementation. It also outlines a practical process for adapting existing evidence-based guidelines to settings where scarce resources may prohibit the local development of guidelines from scratch. The International Diabetes Federation (IDF) has addressed this issue in a comprehensive publication detailing the methods, processes and practicalities of guideline development [30]. Designed with the needs of lower resource settings uppermost, since its release in late 2003 this publication is reputed to be the most frequently requested of all IDF publications.
The steps involved in generating guidelines revolve around identifying, critically appraising and synthesizing the evidence into recommendations. Table I lists these steps as shown in the methods section of the Australian type 2 diabetes guidelines [31].

Table I: Steps for developing guidelines. Adapted from [31].

The steps were identified through literature reviews and consultation with guideline experts and they represent accepted practice in guideline methodology. The two most vital steps in guideline development are: (1) identifying the appropriate research question to guide the literature searches, and (2) searching the literature to find the evidence. Together, these components of the process exert more influence on the final recommendations than all the other developmental steps combined.

Identifying the research question

Evidence-based guidelines are usually commissioned by a national government authority or initiated by a professional organization such as a diabetes society. The research questions will be broadly determined by the brief given to the guideline developers. This will be influenced by why the guidelines are being initiated, the topic(s) or subject matter they are expected to cover, and who is expected to use them. In the case of diabetes, the focus is usually on those processes and practices for monitoring and managing diabetes that fit within the routine clinical scope of primary care physicians. If designed to cover tertiary care, the guidelines would need to concentrate on more complex and/or acute conditions. This might include managing comorbidities and acute episodes of diabetes such as ketoacidosis, non-ketotic hyperosmolar conditions and endstage renal disease.
Because diabetes is a complex condition that can affect all body systems, the development of comprehensive guidelines will require several sets of research questions. Figure 1 illustrates the complexity of type 2 diabetes, highlighting its close relationship and overlap with a range of metabolic conditions, lifestyle-mediated chronic disease risk factors and mental health [32].

Fig. 1: Diabetes, the composite chronic disease. Adapted from [32].

As a result, it is unlikely that diabetes care can be covered comprehensively within one guideline. Rather, a suite of separate guidelines detailing recommendations for blood glucose control, hypertension, retinopathy, macrovascular disease, lipids, foot problems, etc, will be required. Although their inclusion has traditionally been difficult for methodological reasons, guidelines on patient education and the identification and management of psychological issues should also be included. In the case of childhood diabetes, additional issues such as normal growth and development need to be considered.
In order to develop appropriate research questions, guideline developers involve clinical content experts in defining questions that address clinically relevant issues. This task may be assisted by reviewing existing guidelines. The questions must be framed in a way that enables the literature to be searched systematically to identify the scope and status of the available evidence for each topic.

Table II: Steps for identifying relevant research questions [31].

Table II illustrates a framework for optimizing the clinical relevance of research questions and the resulting guidelines.

Finding the evidence

The Cochrane Collaboration [33] has established state-of-the-art methods and protocols for exhaustive literature searching. Despite this, clinicians and researchers are not generally well versed in the art and science of systematic searching, nor do they necessarily have a good understanding of what is involved. For example, strict inclusion and exclusion criteria must be determined and applied. A range of databases (e.g. the Cochrane Library, Embase, Cinahl, Psychinfo, HealthStar), not Medline alone, must be searched. The search strategy must be carefully documented including key words and MeSH terms. The search yield and reasons for inclusion and exclusion of the articles identified for review must be recorded in sufficient detail to ensure that the process can be replicated by others, hopefully with the same result. Ideally, this process is duplicated by a second person to ensure accuracy and reliability of the searches. At the very least it must be subjected to random double-checks.
Following these processes for conducting the systematic reviews to answer each research question is vital to avoid bias in representing the evidence as it translates into guideline recommendations. This is very labour-intensive. In an analysis of 37 meta-analyses, Allen and Olkin [34] found that it takes an average of 1139 h to conduct a systematic review. This equates to 30 full-time-equivalent person-weeks.

Turning the evidence into published guidelines

Guidelines are designed to bring research evidence to the bedside or doctors’ office. Once the evidence has been reviewed and graded it must be collated and synthesized into recommendations. Recommendations should be based on the best available evidence and:
— should be presented as concise statements giving clear guidance that links proven practices to optimal outcomes;
— have the capacity to improve health and reduce harm;
— take account of ethical considerations and be acceptable to consumers;
— be feasible, accessible and within the scope of practice of the health care providers who are expected to apply them.
Draft guidelines are usually circulated for public consultation or, at least, for wide ‘expert’ consultation including representatives of the consumers and health care practitioners who are expected to apply them. Independent review by a methods expert may also be required and is usually overseen by the national guideline authority which will ultimately endorse the guideline(s). Dissemination of completed guidelines should ideally be supported by a strategic and systematic implementation plan designed to promote uptake of the guidelines into routine practice. This may include decision support systems, continuing professional education and academic detailing, consumer education, practice prompts, or clinical audit and feedback [35]. However, no single strategy appears to be significantly more effective than any other single strategy according to Grol and Grimshaw [36], who claim that ‘we need them all’.

Limitations and practical considerations

Methodological
Grading criteria are set out by national research/evidence-based guideline authorities. There are some between-country differences in these criteria, but all invariably overemphasize medico-scientific research. This takes two main forms. One is the heavy weighting in favour of intervention studies as opposed to non-intervention observational research. Within the framework of currently recognized grading criteria, even the most carefully designed and rigorously conducted diagnostic and risk factor studies cannot be accorded as high a level of evidence as RCTs of interventions.

Table III: Levels of evidence criteria [25].

Table III shows an example of recognized grading criteria for determining levels of evidence which illustrates the grading bias in favour of intervention studies and against epidemiological or observational population research. Further, Gleser and Olkin [37] note the bias against the publication of non-significant research results and point out that this may result in a lack of balance in conclusions drawn from meta-analyses.
The second problem is the lack of inclusion criteria for non-medico-scientific research. This applies to qualitative research and most types of sociological research, health service delivery research and implementation research that do not fit the medico-scientific paradigm. There are methodological issues with educational, behavioural and psychological research which mean that such studies rarely rate more than a lower level of evidence. Nor is there any provision for including what has become known as the ‘grey’ literature. This includes government reports and reports from non-governmental organizations such as the IDF which, strictly speaking, are not peer-reviewed and so cannot meet even the lowest level in the evidence grading criteria.
Qualitative research is underpinned by a large theoretical body of knowledge and published research literature. Its methods and techniques are tried and tested and are rigorous in their own right [38]. The lack of a mechanism for including qualitative evidence has serious implications for diabetes guidelines and other chronic conditions. Patient perspectives are usually researched via qualitative surveys, focus groups or interviews. In diabetes, patient acceptance of treatment recommendations, the subjective experience of diabetes, and understanding of and capacity for self-care are integral components of effective management. Failure to take account of patient perspectives and behaviours in these areas leaves a significant gap in guidelines designed to improve the management and outcomes of diabetes.
Another deficiency in some grading criteria (e.g. Table III) is the lack of provision for including expert consensus. It is freely acknowledged that expert consensus is not of the same weight and quality as hard research evidence. Nonetheless, it can be an important adjunct to guideline topics where there are gaps or lack of clarity in the research evidence. This is recognized in the SIGN (Scottish Intercollegiate Guidelines Network) [24] grading criteria which provide a mechanism for rating expert consensus. Other issues relate to evaluating the effect of implementing guidelines. In the dynamic environment of everyday routine practice there are many confounding factors which make assigning cause and effect problematic. This is acknowledged by Fitzner et al. [39], who point out that the complexity of service delivery factors leads to methodological difficulties which result in low generalizability of evaluations of disease management programmes. This is echoed in The evidence base for diabetes care [40], in which Griffin and Williams discuss the difficulties of conducting RCTs on health service delivery and attribute this largely to the complexity of the interventions being evaluated. They cite a review by Griffin [41] which aimed to conduct a meta-analysis on all available RCTs of hospital vs. general practice diabetes care but found few studies which met the inclusion criteria.

Ethical
RCTs are not appropriate to all situations and some research gaps are unlikely ever to be addressed by studies that attract high evidence ratings. For example, the value of screening for type 2 diabetes remains controversial. Ideally this question would be settled by a well-designed RCT. However, in view of the clearly proven benefits of good diabetes control, it would be considered unethical to screen for undiagnosed diabetes and then randomize half of the newly screened population to no treatment.

Practical
It is difficult to keep guidelines current. There are approximately 260 medical journals published every year. This amounts to around one journal reporting new evidence for every working day of each week. Diabetic renal disease is one area where there is continually emerging evidence. Even if new evidence is not accumulating quite so rapidly, the time required for public consultation, independent review and formal approval processes may take at least a year. Consequently, some aspects of any guideline may be superseded by the time it reaches the doctor’s office.
Guidelines cannot cover all contingencies, and while the evidence may show what works best most of the time, it cannot always provide guidance on how to accomplish it. A case in point for diabetes is blood glucose control. The evidence shows that lowering blood glucose to approximate normal ranges reduces complications. However, outside the research setting, there is no definitive evidence about the most effective way of translating the results of clinical trials into routine practice.

Benefits

Despite their limitations, guidelines are the best mechanism we have for translating evidence into easily accessible formats to support informed decision-making by clinicians and consumers. Grimshaw and Russell [42] demonstrated that explicit guidelines can improve clinical practice, albeit with variable effect, and Clark and Kinney [43] showed that guidelines can reduce mal-practice claims.
More recently, Benjamin et al. [44] found significant improvements in physicians conducting annual complications screening and referral to non-medical members of the diabetes health care team following the implementation of guidelines.
In addition, a systematic review of studies of clinicians’ attitudes to guidelines found that, despite criticisms by clinicians, their attitudes on the whole tended to be positive rather than negative. Positives included a perception of guidelines as a good source of clinical advice and education, and a means of improving clinical care [19].
Used wisely, guidelines can serve to assist and underpin clinical health policy and protocols, and act as an objective benchmark for clinical monitoring and surveillance. They can be used to help identify the numbers and skills required in the workforce and to guide resource allocation. Most importantly, they translate research findings into formats and language that enable clinicians, consumers, policy-makers and funders alike to access this information easily.

What can we expect in the future?

There are a number of well-constituted academic or designated guidelines groups across a range of countries that are working with or under the auspices of national health and/or research authorities to refine and improve methods. It is reasonable to expect that many of the current methodological difficulties will be resolved over the next few years.
Government investment in public health and health service research has traditionally been minimal. However, several recent reports, including the landmark Wanless report from the UK [45], have called for a significant increase in attention to these areas. Governments are heeding the call and this is likely to lead to an improved understanding of the effect and cost of implementing guidelines in the future. Once this information is available, guidelines will be increasingly used to underpin decisions about health policy and resource allocation. Hopefully they will also be widely used as the foundation for undergraduate and continuing education for health care providers.
The number of countries and regions covered by national guidelines has increased over recent years with IDF Africa having recently developed a comprehensive set of guidelines. In 1999 Benjamin et al. [44] cited the existence of 100 diabetes guidelines. This growth is likely to slow in the near future and will probably result in diabetes guidelines being rationalized to fewer more widely used guidelines. For example, the IDF has developed a global diabetes guideline that contains treatment options and levels to cover high, medium and low resource settings [46]. This will hopefully relieve many low resource countries of the need to develop their own guidelines. Additionally, several countries (e.g. USA, UK, Canada, Australia) have established highly credible authorities [22–25] that have issued recent guidelines. As a result, in the foreseeable future the emphasis on developing guidelines will most likely be overtaken by an emphasis on updating guidelines.
However, the work does not finish with the production of guidelines. The real challenge is getting them to the doctor’s office and getting the doctor and the patient to use them. Appropriate application of guidelines in routine practice has huge potential to boost widespread access to quality diabetes care, reduce complications and improve health and health-related quality of life. Evaluation methods are improving. All signs point to increasingly more systematic and appropriate research on health service delivery and guideline implementation. This will perhaps be the single most interesting area in the development and use of guidelines in the near future.

References