Keywords

1 Introduction

Clinical guidelines (GLs) are widely used. According to the National Guidelines Clearinghouse, more than 320 organizations have been involved in the development of GLs by 2016, including CGLs authoring organizations, academic research groups, and commercial publishers. There is no real length restriction for GLs and they are produced in a variety of digital and print formats. They can be as short as a single page or as long as a booklet of more than 15 pages. Presenting the GL content in PDF format reduces the chance of finding a relevant answer quickly [1]. To increase accessibility, some GL authoring organizations publish their documents on the Web.

Studies on the accessibility of GLs have revealed that clinicians need to find answers to their questions within 2 min [2]. It is therefore highly important that relevant, accurate information is available as quickly and easily as possible and well within this time limit. However, publishing GLs on the Web may not necessarily make them easier to use, as a poor design can make a system difficult to learn and complicated to use and leads to negative consequences [3]. Furthermore finding the right information in a GL and comparing GLs of interest can be challenging for users, especially as the number of electronically available GLs increases. Therefore, a usability evaluation of published GLs on the Web is necessary to investigate how presentation affects GL use. In this paper we systematically reviewed the literature to summarize the existing publications on usability evaluation of published GLs on the Web considering evaluation criteria, metrics and evaluation methods.

2 Materials and Methods

To retrieve and extract data from relevant studies, we performed a systematic literature review using PubMed and Google Scholar databases. The selection process is presented in Fig. 1. The last search was conducted in March 2017. In order to find more relevant literature, we used a backward snowballing method. Note that our literature search did not include clinical decision support system and their usability evaluation. We employed a Thematic analysis method [4] to identify the evaluation themes presented in Table 1: (1) Usability, (2) Using iconic language, (3) Searching, and (4) Patients guidelines.

Fig. 1.
figure 1

Selection process of retrieved articles

Full size image

3 Results

Table 1. Identified articles, their evaluation criteria and methods
Full size table

4 Discussion and Conclusion

According to the results, no gold standard has been considered in the GL website evaluations. We identified that measuring efficiency was the most used criteria in GL evaluation including task completion time, time spent and number of made errors. The second most popular evaluative criteria was perceived usefulness by applying SUS method, followed by presentation format. The number of evaluations focusing on usability and usefulness, however reviewing the articles revealed that it is not clear how they evaluated them. It is necessary for researchers clearly report how they evaluate and measure usefulness and usability. Although searching function is one of the important factors in findability of information on a GL website, not much attention has been paid to it. GL websites should not only be assessed by ease of use, presentation format, layout, and supported digital features with intuitive and simplified navigation, but also it is necessary that efficient search and the format of search results presentation are evaluated. As clinicians’ time constraints and information overload are two factors in GLs adoption, evaluation of the search function and its retrieval performance in efficiently identifying relevant GLs is needed (i.e. a trained search function for clinical terms, especially for synonyms, acronyms, and abbreviations).