A Narrative Overview of the Effectiveness of E-Learning in Pharmacy Education

In response to the increasing use of a range of technologies within pharmacy education, this review aimed to establish the effectiveness and acceptance of e-learning within pharmacy education and to identify limitations within the research carried out since 2013. The e-learning interventions studied were defined as any online or computer-based technology* used to deliver pharmacy education to pharmacy students or qualified pharmacists in a remote (out of classroom) setting. Evidence of short-term effectiveness of e-learning was found and a small number of studies provided evidence of long-term effectiveness. In comparisons, e-learning was demonstrated to be at least as effective as traditional face-to-face teaching methods and superior to no training at all. Six studies also demonstrated that e-learning could result in a change of pharmacists’ practice, while no evidence was found that e-learning could lead to patient benefit or improvements in care. E-learning also appears to be an engaging learning method, which is generally well received among participants. Despite a significant increase in the number of publications investigating e-learning within pharmacy education in the past five years, further research is still required to address limitations within the current literature and to fully establish the effectiveness of e-learning within pharmacy education.


Introduction
E-learning can be simply defined as learning which is facilitated by technology. This can be either synchronous or asynchronous, and includes both fully online learning and also blended learning, where e-learning is used in combination with traditional face-to-face teaching [1]. Educators appear to be drawn to the unique advantages afforded by e-learning [2,3] and the development of new and effective technologies is driving its increased adoption within higher education. This is reflected by the large volume of e-learning literature in recent years [1,4]. Within pharmacy education, e-learning has been integrated into undergraduate programmes, preregistration training and continuing education and it appears reasonable to assume that this trend will continue. With decreasing resources in pharmacy education, e-learning may be seen as an efficient method to deliver pharmacy undergraduate curricula to free up time for 'active' learning strategies. There are potentially numerous educational and individual benefits from e-learning -flexibility being high up on the list. For example, pharmacists can take part in e-learning at home or from their workplace, there is less time and cost involved as there is no need to travel and there is more choice as online events are not restricted by location. Submit Manuscript . 02 .

Ocimum Scientific Publishers
There have been huge technological advancements in the past 5 years and marked changes predicted in how people will work in the future [5]. The growing trend towards more remote and more flexible working comes with a requirement for new knowledge and skills and more flexible and accessible methods of learning for the acquisition of these skills [5]. It is therefore timely to examine the literature relating to the effectiveness of e-learning in pharmacy education in the five-year period since the publication of the only other review into this area by Salter and colleagues [6]. This review concluded that further research was required to establish the long-term effectiveness of e-learning and whether e-learning could result in changes in pharmacists' practice or patient benefit. Therefore, establishing whether any progress had been made in these areas was of particular importance. In the review by Salter and colleagues [6], e-learning interventions were evaluated according to an adaptation of Kirkpatrick's four levels of training evaluation model [7], which may be applied to higher education for the purposes of assessing educational effectiveness [8]. The model appraises training programmes on the basis of four potential outcomes: 1. Reaction, which relates to participants' satisfaction and opinions. 2. Learning, which measures an actual or perceived change in attitudes, knowledge or skills of participants. 3. Behaviour, which assesses the application of knowledge or skills gained from the programme. 4. Results, which evaluates the extent to which the specific goals are achieved, as a result of the programme [8].
It was our aim to review the literature published since the Salter review [6] and apply the same classification of effectiveness (the adapted Kirkpatrick's four level model) used by Salter and colleagues to subsequent studies around e-learning in pharmacy education.

Method
The purpose of this literature review was to provide a summary of, and identify gaps within, the literature around the effectiveness of e-learning in pharmacy education between 2013 and 2018. This time period was chosen, as a systematic review has previously examined the literature relating to the effectiveness of e-learning in pharmacy education before 2013 (search date of 4 th June 2013) [6].
The definition of e-learning intervention and the search process, including databases used and search terms are detailed in table 1. Studies returned by this search were then reviewed to determine if they met pre-defined inclusion criteria.

Inclusion and Exclusion Criteria
Any study designed to evaluate the effectiveness of e-learning within the pharmacy profession was considered for inclusion in this review. Participants had to be pharmacy students, pharmacists or pre-registration pharmacists. Studies had to measure participants' perceptions of an e-learning intervention and/or the effect of an e-learning intervention on participants' knowledge, skills or confidence. Ideally studies would also have investigated the impact of the e-learning intervention on participants' practice and/or if the e-learning intervention resulted in patient benefit or an improved quality of care. Studies involving other health professionals/students, other pharmacy staff, faculty members or patients were not included, with the exception of one study where a control group of medical students received no training [9]. Studies using technologies that were based solely in a classroom (not remote) setting were excluded. Studies utilising technologies that are recreational in nature, such as computer games, were not included in the review. Studies which examined ways to improve e-learning, factors affecting the success of e-learning or opinions of e-learning in general (not a specific intervention), were not included. Studies which used technology that was not online or computer-based, such as high-fidelity simulations, were also not included. Studies that were unpublished or were not in English, were excluded from the review.
The full text of each included study was subjected to an in-depth review. During this review, key information was extracted from each study including the number of participants, topic, type of e-learning intervention, whether or not a control group was included, type of assessment of effectiveness and classification of effectiveness according to an adapted model of Kirkpatrick's four levels of training evaluation model. The limitations of the various studies were identified and summarised.

Results
The initial search process returned 2207 studies, which were then screened according to the inclusion and exclusion criteria and duplicate studies were excluded. The literature search flow diagram is shown in figure 1. Fifty-six studies were subsequently deemed suitable for inclusion in the review.

Intervention
Studies included a wide variety of topics covering many aspects of both undergraduate and postgraduate pharmacy education. The most common topics were ambulatory care, pharmaceutical calculations and pharmaceutical compounding.
A diverse range of technologies were used as the e-learning intervention, but the two most common were online virtual patient cases and online courses. did find that students felt more confident managing ethical dilemmas following completion of the e-learning intervention.
In the third study that did not have a positive outcome, McCabe et al., [42] compared the face-to-face and online delivery of a self-care and non-prescription medications course to pharmacy students. Students completed pre-and postcourse knowledge tests. Students in the face-to-face cohort scored higher than those in the online cohort and there was no significant difference between the pre-and post-intervention scores of the online cohort.
Learning was also assessed subjectively, through pre-, mid-and post-intervention surveys in twenty-

Limitations of the Studies
Of the fifty-six studies that were reviewed, all were limited by methodological flaws. . This is particularly significant for studies which relied on self-reported data to measure the effect of the intervention. The methodological flaws of studies were not always identified by researchers and threaten both the internal and external validity of their findings.

Strengths of the Studies
Despite the limitations of the studies, many have been properly designed so as to reduce the risk of bias and secure the validity and reliability of the results. there was a control group and a random allocation process. Therefore, it seems reasonable to consider the overall conclusions reached by the studies as legitimate.

Discussion
The current literature supports the short-term effectiveness of e-learning for pharmacy education and also suggests that the effect of the e-learning intervention could be maintained several months post-intervention. E-learning appears to be at least as effective as face-to-face learning and superior to no training at all. However, further research is required to fully establish the long-term effectiveness of e-learning within pharmacy education. However, common to all of these studies was that the topic of e-learning was pharmacy practice. This most likely explains why these studies found evidence that e-learning interventions could result in an intention to change practice, as such topics are immediately applicable and more obviously relevant to the workplace. Only one study reported an actual change in practice rather than an intention to change practice [14]. In that study, pharmacists reported that completion of an online education program led to improved patient interactions, more effective inter-professional working and enhanced performance of pharmacy-specific tasks [14]. No studies attempted to objectively measure patient benefit or improvements in care.
In the five years since the only other previous review into the effectiveness of e-learning within pharmacy education [6], there has been continued adoption of e-learning within both undergraduate and postgraduate pharmacy education and a large increase in the volume of literature relating to this topic. However, the evidence base to further our understanding of the effectiveness of e-learning for pharmacy education is limited. There still exists insufficient evidence to establish the impact of e-learning on long-term learning and changes in the practice of pharmacists; and a complete lack of evidence required to determine if an e-learning intervention can lead to an increase in the knowledge or skills of pharmacists sufficiently, so as to result in patient benefit or improved quality of care. The same methodological weaknesses that were identified in the research prior to 2013 are still present in the more recent studies [6].
This review had several strengths. The search process was completed using four bibliographic databases resulting in a comprehensive literature review since 2013. Pre-defined inclusion and exclusion criteria were implemented in order to reduce the likelihood that bias from the researcher would affect the decision to include or exclude studies. Each e-learning intervention was evaluated according to Kirkpatrick's four levels of training evaluation model. This model allows for effectiveness of each intervention to be established objectively, on the basis of study outcomes.
This study had a number of limitations. Unlike the previous review by Salter and colleagues [6], this was not a systematic review and studies were not assessed for quality. A narrative overview is not subject to the same rigorous methodology as a systematic review. Despite the use of specific search terms and pre-defined inclusion criteria, the decision on whether to include or exclude individual studies may still have been affected by bias on the part of the researchers. Therefore, a relationship between an individual study's quality and its findings cannot be established, nor can the impact of this on the validity of the overall conclusions be determined. All studies that were reviewed had methodological flaws and the overall conclusions of this review will have to be considered in this light.

Conclusion
This review found e-learning to be both a well-accepted and effective teaching method (albeit in the short term), across a Ocimum Scientific Publishers . 07 .

Journal of Pharmacy Practice and Pharmaceutical Sciences O'Hare C and Girvin B.
wide range of topics and technologies. For these reasons, it is likely that e-learning will continue to be integrated into all levels of pharmacy education. It appears that the role of e-learning within pharmacy education is in combination with face-to-face teaching, and that achieving the correct balance between these two pedagogical approaches is crucial to ensuring its successful use. The review exposed that within the current research, there is a need for high quality studies and that there are a number of directions for further research. In particular, there is a need for more studies to determine whether e-learning is effective in the long term and whether e-learning results in changes to practice and improvements in patient outcomes. Addressing these areas will provide a comprehensive understanding of the effectiveness of e-learning within pharmacy education. 5. Shank P (2016) How will we work. How will your job change?.