Skip to main content

Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist

Abstract

Background

Health practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs.

Methods

The national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014–2016. We searched PubMed to identify relevant European guidelines (n = 24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF10) was used to evaluate the differences between the national and transnational HPGs.

Results

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items (Mann Whitney U test, P = 0.048; BF10 = 1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF10 = 80.63), as well as describing the implications of costs and resources (BF10 = 55.15). Croatian HPGs better reported HPGs specified aims (BF10 = 16.90), primary intended users (BF10 = 8.70), and sources of funding (BF10 = 122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations.

Conclusions

Important methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.

Peer Review reports

Background

Health practice guidelines (HPGs) are “statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” [1]. Guidelines that can be trusted are not only based on a systematic review of best available evidence, but they should be developed by a multidisciplinary panel in a transparent, stakeholder inclusive, documented, and accessible process [1].

HPGs have become a standard in clinical practice and their number has been on a continual increase. PubMed search for MeSH terms “Practice Guideline [Publication Type]” or “Practice Guidelines as Topic” returned 126,573 publications on May 2, 2018, with 23,976 publications related to Europe. Despite this high production rate of and interest in HPGs, there is a great variation in their development, quality, implementation, and use, as demonstrated by the recent overview of the framework for HPGs development at the national level in Europe [2]. In many countries, the development of HPGs is tasked to different specialty/professional societies [2], which thus have a central role in translation of evidence to local practice, most often in the local language. The local guidelines are usually published in national journals or on the websites of professional societies, which serve as dissemination outlets for the HPGs.

Quality of reporting has been recognized as a key measure for successful translation of evidence to practice, reducing overall waste in research and helping in eliminating non-replicable studies [3]. Whereas a number of guidelines for reporting research results have been developed for different types of research [4], the quality of reporting of HPGs is not well known, primarily because there have been no tools specially developed for HPGs. Experience from other fields shows that reporting guidelines, such as Consolidated Standards of Reporting Trials (CONSORT) for randomized controlled trials and STARD for diagnostic studies, are effective in improving completeness of reporting in their respective fields [5,6,7]. Recently, two reporting checklists have become available for HPGs: Appraisal of Guidelines, Research and Evaluation (AGREE) Reporting Checklist [8] and Reporting Items for Practice Guidelines in HealThcare (RIGHT) [9]. Whereas the AGREE checklist was developed on the concept of the AGREE tools for the methodological rigor and transparency of HPGs [10], the RIGHT checklist follows standard reporting guideline approach established by other reporting statements, such as CONSORT and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [11, 12]. The RIGHT checklist was developed to assist guideline developers in reporting, journal editors and peer reviewers in decision making, and health care practitioners in understanding and implementing guidelines [9]. So far, RIGHT checklist has only been used to evaluate Chinese guidelines for sepsis and for acupuncture [13, 14].

The aim of our study was to assess and compare the quality of reporting in nationally and transnationally produced (European) HPGs by using the RIGHT reporting checklist. We assessed HPGs developed by professional societies in Croatia, a non-English speaking country, and compared them to similar European HPGs produced by transnational professional societies. We included only European HPGs as comparators on the basis of similarity of populations they address, health problems that are explored, and also because Croatian guidelines often rely on, adapt, and cite their European counterparts.

Methods

Study design

We used a cross-sectional design to assess a cohort of published Croatian and relevant European guidelines.

Cohort of Croatian HPGs

In Croatia, most of the guidelines are developed independently by specialty societies gathered around the Croatian Medical Association, except a few nationally developed guidelines, such as ISKRA HPGs concerning the use of antibiotics [15]. HPGs are most often published in Liječnički vjesnik, the official journal of the Croatian Medical Association [16]. For this study, we included all HPGs published in Liječnički vjesnik from 2014 until March 2017: 26 guidelines (last guideline published in 2016).

Search strategy for relevant European HPGs

To identify matching European guidelines, we reviewed the bibliography of included documents and searched PubMed and Google. We used the guideline topic as a search term combined with “Europe” and “guideline or recommendation.” Guideline topic included the clinical condition and intervention mentioned in the Croatian HPGs (e.g., type 2 diabetes pharmacotherapy, primary breast cancer diagnosis). Two authors (RT and MV) independently identified guidelines for inclusion, and the final decision was made by consensus.

Some of the Croatian HPGs covered more than one topic, i.e., clinical condition and/or intervention. In those cases, we searched for all topics covered by the guideline and included all relevant European guidelines.

The search for European guidelines was performed in July 2017. We excluded guidelines that were adapted for non-European populations, global or international guidelines, and those unavailable in English or Croatian.

Data extraction

We used the RIGHT checklist to evaluate the published HPGs. The RIGHT checklist [17] contains 22 requirements organized into 7 sections with a total of 35 items: basic information (6 items), background (8 items), evidence (5 items), recommendations (7 items), review and quality assurance (2 items), funding and declaration and management of interest (4 items), and other information (3 items). Two authors (RT and MV) independently assessed the adherence of HPGs with the RIGHT checklist. In case of conflicting opinions, the third author (AM) was consulted and agreement was reached by consensus. Kappa statistic was used to measure the level of agreement.

Data analysis

Data are presented as the number of RIGHT checklist items reported in each guideline, as well as number of guidelines that reported individual RIGHT checklist items and their proportion, with 95% confidence interval. Due to the non-normality of distributions, we performed Mann-Whitney U test to compare the two groups of HPGs scores overall. For each RIGHT item separately, we calculated the Bayes factor in the contingency table and frequentist chi-square test to evaluate the differences between the two groups. Bayes factor (BF10) was calculated assuming a default prior distribution. P value < 0.05 and BF10 ≥ 3 were considered significant [18]. In cases of conflicting frequentist and Bayesian results of significance, we considered the results of Bayes statistics for final interpretation. Calculations were done using MedCalc Statistical Software version 16.4.3 (MedCalc Software bvba, Ostend, Belgium; https://www.medcalc.org; 2016), and JASP software, version 0.8.6 (https://jasp-stats.org/).

Results

We reviewed all published Croatian HPGs (n = 26) and 24 matching European HPGs. Eleven Croatian HPGs did not have European counterparts, and 3 of them had multiple European counterparts. Because of these discrepancies, we did not do a matched analysis. Kappa coefficient for individual 35 items of RIGHT checklist ranged from 0.76 to 1.0. All of the European guidelines were produced by transnational specialty societies, and most of Croatian guidelines were produced by national specialty societies (23 out of 26; 3 were produced by individual authors, independently from specialty societies; 1 of those declared to be supported by professional societies). All of the guidelines were produced or updated in 2014 or later. The full list of guidelines and individual results of the adherence to the RIGHT checklist are available in the Additional file 1.

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items. Their European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items. The difference was not significant (P = 0.048, Mann-Whitney U test, BF10 = 1.543). The lowest total number of addressed RIGHT items was 11 (31%) (Croatian HPGs on melanoma and cystine urolithiasis), and the highest was 22 (63%, European resuscitation guideline).

The results of adherence to individual RIGHT checklist items are presented in Tables 1, 2, and 3. For nine checklist items, both groups of HPGs had a satisfactory level of reporting (≥ 80% of HPGs in each group adhered to the checklist item): item 1a—clear guideline titles, described as mentioning the focus of the guideline; item 1c—defining it as guidelines or recommendations in the title; item 4—providing an address of a corresponding author; item 9a—lists of authors and their affiliations; item 5description of basic epidemiology of health problems; item 7a—description of primary populations addressed by the HPG; item 13a—clear and actionable recommendations; item 13b—separate recommendations for important subgroups; and item 13c—indicated strengths of recommendations and certainty of evidence.

Table 1 Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of basic information and background (total number and percentage with 95% confidence intervals (CI) of addressed items)
Table 2 Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of evidence and recommendations (total number and percentage with 95% confidence intervals (CI) of addressed items)
Table 3 Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of review and quality assurance, funding, declaration and management of interest, and other information (total number and percentage with 95% confidence intervals (CI) of addressed items)

For 12 out of 35 RIGHT items, both sets of HPGs had poor reporting (≤ 20% of HPGs in each group adhered to the checklist item): item 9a—description of roles and responsibilities of authors and contributors; item 10a—a clear and structured healthcare question in PICO or other appropriate format; item 10b—clearly defined outcomes; items 11a and 11b—process of selection of systematic reviews; item 17—description of quality assurance process; item 18b—reporting on the roles of funding bodies in the development; item 19b—management of conflicts of interest; item 22—description of limitations of the guidelines, such as risk of bias, conflicts of interest or lack of patient involvement; and item 20—availability of guidelines and additional documents.

European guidelines were significantly better in providing a HPG summary (item 2; BF10 = 4.44), reporting stakeholder involvement and values and preferences (item 14a; BF10 = 80.63), as well as describing the implications of costs and resources (item 14b; BF10 = 55.15, Table 2). They also reported better than Croatian HPGs on existing conflicts of interest (item 16; BF10 = 55.35), external reviews (item 19a; BF10 = 1.02 × 108), and on sufficiency of evidence for recommendations or provided suggestions for future research (item 21; BF10 = 733.70).

Croatian HPGs were significantly better in reporting specified aims of the guideline (item 6, BF10 = 16.90), their primary intended users (item 8a, BF10 = 8.70), and sources of funding (item 18a, BF10 = 122.90).

For item 12 (description of the approach used to assess the certainty of the body of evidence), the Croatian and European HPGs sets had similar adherence (73.1%, 95% CI 51.9–87.7 for Croatian, and 75.0%, 95% CI 55.1–88.0 for European HPGs; BF10 = 0.30). We also extracted the information on what sort of approach was used in the HPGs. Grading of Recommendations, Assessment, Development and Evaluation (GRADE) handbook [19] and Oxford Centre for Evidence Based Medicine levels of evidence [20] were predominant approaches in Croatian HPGs (53.8% of HPGs), but were less frequent in the European HPGs set (16.7% of HPGs). European HPGs mostly mentioned adapted Infectious Diseases Society of America–United States Public Health Service Grading System [21] (45.8% of HPGs). Other grading systems, such as American College of Chest Physicians (ACCP) modified GRADE system [22] or Scottish Intercollegiate Guidelines Network (SIGN) guide [23], were mentioned in 7.7% of Croatian and 12.50% of European HPGs. Some HPGs were unclear about the grading systems, mentioning the levels of evidence but not referencing any grading systems (23.1% of Croatian and 16.7% of European HPGs). Finally, 15.4% of Croatian and 8.3% of European HPGs did not mention or imply the use of any grading systems for levels of evidence.

Discussion

Our study showed that both national and transnational HPGs were poorly reported in journals, with adequate presentation for fewer than 50% of necessary reporting items. This is, to the best of our knowledge, the first evaluation of HPGs in Europe using the RIGHT reporting checklist, developed for guidance in reporting guidelines in healthcare, including clinical and public health areas. It is important to keep in mind that the RIGHT checklist is not a guide for development nor a tool for assessment of methodological quality of HPGs, but a tool for checking the clarity of presentation of the HPGs to a wide audience, including both developers and readers, so that HPGs could be adequately understood and implemented in practice [9].

Our finding of inadequate reporting of HPGs matches previous observations of poor methodological quality of HPGs produced by specialty societies [24, 25]. There was no difference between the quality of reporting of national HPGs, which are directly implemented in practice in individual health care systems, and those produced by transnational societies, which put emphasis on multidisciplinary collaboration and have standard operating procedures for guideline development [26].

The relatively small sample size is the most important limitation of our study. The sample size was determined by the cohort of available Croatian HPGs. We did not include HPGs published earlier than 2014 as they were not numerous and their presentation was of poor quality (our unpublished observation). The small sample size was also the reason why we based the interpretation of results on the results of Bayes and not frequentist statistics. The advantages of coherency, independency of the intention with which data are collected, and minimum bias toward null hypothesis made Bayesian inference valid for small sample sizes [27]. It is also important to keep in mind that we analyzed national HPGs in a single country, so the results may not be generalizable to the whole population of nationally produced guidelines. However, the similarity of findings for Croatian and matching transnational guidelines indicates that the level of reporting quality is similar across professional societies in Europe. Finally, it is important to keep in mind that the RIGHT checklist does not evaluate the validity of the actual recommendations in the HPGs, but only the quality of their publicly available presentation.

The analysis of compliance with individual reporting items demonstrated that there was an overall insufficient emphasis on providing clear aims and a summary of recommendations. Guideline developers failed to provide clear instructions on how the guidelines are intended to be used and by whom. In this way, basic principles of evidence-based medicine had not been addressed because there was little information on the users of HPGs, the purpose of HPGs, and acknowledging barriers and constraints that may limit their implementation [28].

Both national and transnational sets of HPGs adequately reported on strengths of recommendations and certainty of supporting evidence, but details on how the evidence was searched for and evaluated were unsatisfactory. Established grading systems, such as GRADE and Oxford EBM, were used in half or less than a half of the analyzed HPGs, and 22% of HPGs were unclear about grading evidence. This means that current HPGs do not satisfy the preference of clinicians for recommendations that have clearly presented processes behind them and provide evidence summaries [29, 30]. Clear and transparent reporting should also help address problems such as relying on lower levels of evidence than stated in HPGs [31], relying on low levels of evidence to form recommendations [32], and inadequate presentation of evidence grading [33].

In national HPGs, reporting on the inclusion of patient values and preferences in the making of recommendations was poor, and reporting cost and resource considerations was almost non-existent. This reflects generally unsatisfactory involvement of the patients in guideline development [34], as well as integration of evidence on patient preferences [35], even though the involvement of patients and other stakeholders is especially important for development and adaptation of HPGs oriented at lower- and middle-income populations, which often have limited resources and different patient experiences and priorities [36, 37].

We also found that reporting of conflicts of interest (COI) was worse in Croatian than in European HPGs, but that both groups of guidelines showed low adherence to providing declarations on the conflicts of interest and elaborating policies regarding their management. Available data show that the prevalence of conflicts of interest in guideline development is high [38], and that conflicts of interest can influence interpretation and recommendations of treatments [39], as well as definitions of diseases [40]. Croatia is a country that follows the practice of the European Federation of Pharmaceutical Industries and Associations (EFPIA) Code for disclosure of financial transfers [41], but does not have a national legislation that requires disclosure on a central platform, such as Denmark [42]. Even in systems with mandatory conflict of interest declaration, like Denmark, there is a discrepancy between disclosures in the national register and self-reporting in publications [42]. In Croatia, only 11% of physicians made their individual data available in 2016, when the first report from the Innovative Pharmaceutical Initiative, an association that gathers 33 pharmaceutical companies that cover 60% of Croatian pharmaceutical market, was published [43].

Conclusion

Our study provides evidence to recommend the use of reporting checklists to guideline developers both at national and transnational levels. A variety of tools have been developed to help translation of evidence to reliable and high-quality recommendations, including GRADE system for rating quality of evidence [44], AGREE tools for guideline methodology [10], checklists for development, implementation, evaluation, and quality [45, 46]. Without complete and transparent reporting, there is no way of knowing how good and how appropriate HPGs are for use in specific clinical circumstances. In times of overwhelming production of guidelines, which are supposed to help the end-user in finding and summarizing the best evidence [47], it is particularly important to achieve clarity and enable informed decisions. In future research, the usefulness of RIGHT reporting guideline should be tested in actual health practice settings and they should be compared with similar reporting guidelines, such as AGREE Reporting Checklist [8]. Endorsement of reporting guidelines for HPGs, such as RIGHT, by professional associations, health websites, and journals would contribute to the effort of transparency for developers and to the aim of adequate implementation for users of guidelines. Only then can we hope to achieve the goal of full transparency, comprehension, and availability of information that concerns a variety of stakeholders.

Abbreviations

AGREE:

Appraisal of Guidelines for Research and Evaluation

CONSORT:

Consolidated Standards of Reporting Trials

GRADE:

Grading of Recommendations Assessment, Development and Evaluation

HPG:

Health practice guideline

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RIGHT:

Reporting Items for Practice Guidelines in HealThcare

SIGN:

Scottish Intercollegiate Guidelines Network

STARD:

Standards for Reporting of Diagnostic Accuracy Studies

References

  1. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In: Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from: https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/books/NBK209539/. doi: 10.17226/13058.

    Google Scholar 

  2. Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health policy. 2012;107(2–3):146–56. Epub 2012/09/04

    Article  Google Scholar 

  3. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. Epub 2014/01/15

    Article  Google Scholar 

  4. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med. 2010;8:24. Epub 2010/04/28

    Article  Google Scholar 

  5. To MJ, Jones J, Emara M, Jadad AR. Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One. 2013;8(12):e84779. Epub 2014/01/07

    Article  CAS  Google Scholar 

  6. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67(5):792–7. Epub 2006/09/13

    Article  CAS  Google Scholar 

  7. Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992–5. Epub 2001/04/20

    Article  CAS  Google Scholar 

  8. Brouwers MC, Kerkvliet K, Spithoff K. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016;352:i1152. Epub 2016/03/10

    Article  Google Scholar 

  9. Chen Y, Yang K, Marusic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166(2):128–32. Epub 2016/11/29

    Article  Google Scholar 

  10. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421–4.

    Article  Google Scholar 

  11. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. Epub 2010/03/25

    Article  Google Scholar 

  12. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. Epub 2009/07/22

    Article  Google Scholar 

  13. Chen RB, Chen YL, Gai GZ, Xie YM, Wang YY. Analysis of reporting specification about sepsis practice guideline based on RIGHT standard. Zhongguo Zhong yao za zhi. 2017;42(8):1514–7. Epub 2017/10/27

    PubMed  Google Scholar 

  14. Xiaoyang S, Lu G, Ru L. Assessment of reporting quality of guidelines on acupuncture and moxibustion by using RIGHT Statement. Chinese Medical Doctor Association Annual Conference on Evidence-based Medicine 2017.

    Google Scholar 

  15. Intersectoral Coordination Mechanism for the Control of Antimicrobial Resistance (ISKRA). Croatian National Guidelines. Available from: http://www.iskra.bfm.hr/eng/Guidlines.aspx?id=61

  16. Liječnički vjesnik - glasilo Hrvatskoga Liječničkog Zbora. Available from: https://lijecnicki-vjesnik.hlz.hr/. Accessed 28 May 2018.

  17. RIGHT working group. RIGHT Checklist. Available from: http://www.right-statement.org/right-statement/checklist. Accessed 28 May 2018.

  18. Jeffreys H. Theory of probability. Oxford: Oxford University Press; 1961.

    Google Scholar 

  19. Schünemann H BJ, Guyatt G, Oxman A, editors. GRADE handbook for grading quality of evidence and strength of recommendations. Updated October 2013. Available from: http://gdt.guidelinedevelopment.org/app/handbook/handbook.html Accessed 28 May 2018.

  20. Howick J. Levels of evidence. Oxford: Centre for Evidence Based Medicine; 2009. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/

    Google Scholar 

  21. Dykewicz CA. Summary of the guidelines for preventing opportunistic infections among hematopoietic stem cell transplant recipients. Clin Infect Dis. 2001;33(2):139–44.

    Article  CAS  Google Scholar 

  22. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129(1):174–81. Epub 2006/01/21

    Article  Google Scholar 

  23. SIGN guidelines—an introduction to SIGN methodology for the development of evidence-based clinical guidelines. SIGN Publication No. 39, SIGN Secretariat, Royal College of Physicians of Edinburgh. Edinburgh 1999.

  24. Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355(9198):103–6. Epub 2000/02/16

    Article  CAS  Google Scholar 

  25. Tudor KI, Kozina PN, Marusic A. Methodological rigour and transparency of clinical practice guidelines developed by neurology professional societies in Croatia. PLoS One. 2013;8(7):e69877. Epub 2013/07/31

    Article  CAS  Google Scholar 

  26. ESMO Guidelines Methodology. Available from: http://www.esmo.org/Guidelines/ESMO-Guidelines-Methodology. Accessed 28 May 2018.

  27. Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J, et al. Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications. Psychon Bull Rev. 2018;25(1):35–57. Epub 2017/08/06

    Article  Google Scholar 

  28. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.

    Article  Google Scholar 

  29. Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.

    Article  Google Scholar 

  30. Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al. Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol. 2018;99:33–40. Epub 2018/03/14

    Article  Google Scholar 

  31. Lee D, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of america practice guidelines. Arch Intern Med. 2011;171(1):18–22.

    Article  Google Scholar 

  32. Alexander PE, Brito JP, Neumann I, Gionfriddo MR, Bero L, Djulbegovic B, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance. J Clin Epidemiol. 2016;72:98–106. Epub 2015/01/27

    Article  Google Scholar 

  33. Neumann I, Brignardello-Petersen R, Wiercioch W, Carrasco-Labra A, Cuello C, Akl E, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implement Sci. 2016;11:93. Epub 2016/07/16

    Article  Google Scholar 

  34. Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res Involv Engagem. 2017;3:19. Epub 2017/10/25

    Article  Google Scholar 

  35. Chong CA, Chen IJ, Naglie G, Krahn MD. How well do guidelines incorporate evidence on patient preferences? J Gen Intern Med. 2009;24(8):977–82. Epub 2009/04/24

    Article  Google Scholar 

  36. Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442. Epub 2016/09/15

    Article  Google Scholar 

  37. Gonzalez-Suarez C, Grimmer-Somers K, Dizon J, King E, Lorenzo S, Valdecanas C, et al. Contextualizing Western guidelines for stroke and low back pain to a developing country (Philippines): an innovative approach to putting evidence into practice efficiently. J Healthc Leadersh. 2012;4:141–56.

    Article  Google Scholar 

  38. Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153. Epub 2011/11/01

    Article  CAS  Google Scholar 

  39. Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ. 2010;340:c1344. Epub 2010/03/20

    Article  Google Scholar 

  40. Moynihan RN, Cooke GP, Doust JA, Bero L, Hill S, Glasziou PP. Expanding disease definitions in guidelines and expert panel ties to industry: a cross-sectional study of common conditions in the United States. PLoS Med. 2013;10(8):e1001500. Epub 2013/08/24

    Article  Google Scholar 

  41. EFPIA. EFPIA code on disclosure of transfers of value from pharmaceutical companies to healthcare professionals and healthcare organizations. Consolidated version, June 2014. Available from: https://efpia.eu/media/25837/efpia-disclosure-code.pdf. Accessed 28 May 2018.

  42. Rasmussen K, Schroll J, Gotzsche PC, Lundh A. Under-reporting of conflicts of interest among trialists: a cross-sectional study. J R Soc Med. 2015;108(3):101–7. Epub 2014/11/13

    Article  Google Scholar 

  43. IFI. Innovative pharmaceutical initiative: public announcement 2016. Available from: http://ifi.hr/en/public-announcement-2016/. Accessed 28 May 2018.

    Google Scholar 

  44. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336(7652):1049–51.

    Article  Google Scholar 

  45. Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42. Epub 2013/12/18

    Article  Google Scholar 

  46. Grimmer K, Dizon JM, Milanese S, King E, Beaton K, Thorpe O, et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Med Res Methodol. 2014;14:63. Epub 2014/06/03

    Article  Google Scholar 

  47. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, et al. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6. Epub 2013/01/11

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was funded by the Croatian Science Foundation, grant No. IP-2014-09-7672 “Professionalism in Health Care.” The funder had no role in the design of this study, during its execution and data interpretation.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

AM and RT designed the study. RT and MV extracted the data. IB and RT performed data analysis. RT drafted the manuscript. All authors made intellectual contributions to the manuscript and approved the final version.

Corresponding author

Correspondence to Ružica Tokalić.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

AM declares that she contributed to the development of the RIGHT guideline. RT, MV, and IB declare that they have no conflicts of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

File contains Table S1. and S2, which contain full lists of assessed HPGs and number of RIGHT checklist items reported in each individual HPG. (DOCX 23 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tokalić, R., Viđak, M., Buljan, I. et al. Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist. Implementation Sci 13, 135 (2018). https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-018-0828-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-018-0828-4

Keywords