Abstract
Many IS studies focus on initial user perceptions as drivers of initial IS success, but these perceptions may change over time. This study aims to examine whether there is any difference in the drivers of IS success between the initial adoption and the initial assimilation stages. With the Expectation Confirmation Model as the theoretical lens, a repeated cross-sectional study was conducted regarding clinicians’ perceptions of the implementation of electronic medication management systems (eMMS) in an Australian hospital, one at the time of implementation and another one at one year after the implementation. The results indicated that the means of perceived usefulness, perceived ease of use, social influence, clinician satisfaction, and organizational benefits are significantly higher in the initial assimilation stage than in the initial adoption stage, which means that expectations have been positively confirmed in the initial assimilation stage. More importantly, while perceived usefulness and social influence remain important one year after the implementation, perceived ease of use and facilitating conditions are moving toward a decrease in importance to the success of eMMS. This study provides a better understanding of how the importance of the factors influencing eMMS success changes over time.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Information technology has changed our way of life more conveniently and comfortably. It is no exaggeration to say that IT is increasingly being applied to every single activity and providing solutions to the enormous problems we face. One of the major application areas of IT is health, which has received increasing and ongoing investments in it. One thing that differentiates health from other application fields of IT is that it requires a more rigorous and robust evaluation of the technologies, given that it is related to human life. Thus, some health technology evaluation frameworks have been developed and used widely in health-related organizations and governments in the world [1].
Studies on HIT (Health Information Technology), including EMR (Electronic Medical Record), EHR (Electronic Health Record), CPOE (Computerized Physician Order Entry), and eMMS (Electronic Medication Management System), have increased enormously over the last decade [2,3,4]. In particular, the effect of CPOE, EMR, and EHR on clinical work efficiency has been studied for a long time. However, many of such studies mostly focused on the initial perceptions and effects of HIT at the very early stage of HIT implementation [5]. As increased and consistent IT use is believed to improve patient care and clinical outcomes, it is becoming more necessary to continuously monitor HIT use and user satisfaction [6, 7].
While there is increasing adoption of HIT by hospitals, that does not necessarily mean that individual clinicians are satisfied and will continue to use HIT. As most HIT systems are mandatory, clinicians have no choice but to use the system regardless of their intention or system quality [8]. Similarly, it is impossible to discontinue using the system for individual clinicians. Though there has already been a lot of literature about technology acceptance and continuance, many of the extant models are based on the voluntary and individual use of the system, which can be different from the organizational perspective. As the majority of systems introduced in organizational settings, including in the health sector, are expected to be used and therefore are mandatory, more organizational perspectives with the extant models are important in interpreting organizational and mandatory context.
More importantly, many IS continuance studies mostly compare the perceived performances with the expectations. These studies focus on initial perceptions, but these perceptions may change over time [5]. As users do not have enough hands-on experience with the system at the initial stage, they might face considerable difficulties in adapting themselves to the new system functions and the new way of working. According to the theory of belief updating [9], an initial evaluation made by prior knowledge is revised by subsequent interactions or experiences of new information. This theory has been applied to many studies of marketing, which examine intertemporal changes in customer attitudes, customer satisfaction, and intention to purchase. In line with this theory, Venkatesh et al. [10] suggested that initial beliefs such as performance expectancy, effort expectancy, social influence, and facilitating conditions have positive effects on modified beliefs, respectively. In addition, some scholars suggest that the importance of these factors and their effects on outcomes tend to be different at different stages of implementation [10,11,12].
Among many HITs, the electronic medication management system (eMMS) was chosen as a case for this study. The eMMS is changing the way of working in hospitals more efficiently and effectively. With raised awareness of the importance of eMMS, there have been growing investments in eMMS in many countries across the world. Australia is one of the countries which has invested a huge amount of budget into eMMS to provide better healthcare. In particular, the New South Wales (NSW) Government of Australia has been making impressive progress in implementing health IT at the state level. With the eHealth Strategy for NSW Health: 2016–2026, $236 million over seven years, including $18.7 million in 2017–2018, are being invested in the roll-out of the eMMS (Electronic Medication Management System) across NSW [13].
Among the existing IT implementation models, this study employs the three-stage model from Wright et al.[14]: (1) initiation, during which a new IS is evaluated for its usefulness before adoption, (2) adoption, in which the decision to adopt and initially use the IS is made, and (3) assimilation, during which the IS becomes widely used as an integral part of an organization’s business processes. Figure 1 shows different stages of eMMS project in this study to distinguish clearly “perception in initial adoption” from “perception in initial assimilation”. While there have been many studies that examine the factors influencing acceptance of health IS, such as EMR, CPOE, and so on, there is a lack of study that looks at how the importance of these factors changes over time. Adoption or acceptance of a new system indicates success in the early stage, but it is also important to ensure that the system has moved beyond the initial trial and becomes ongoing success in the long term [3].
The primary research questions of this study are as follows: (1) What are the factors driving clinician satisfaction and organizational benefits of eMMS in the initial adoption stage and initial assimilation stage? (2) How do user evaluations change over time? The rest of this paper is organized as follows: First, the theoretical backgrounds are discussed together with the research gaps. Second, a series of research hypotheses are derived to be tested. Third, research methods are discussed, including study context, data collection procedure, etc. Fourth, the results of hypothesis testing are discussed. Lastly, the theoretical and practical implications of the results are discussed, including the limitations.
2 Related works
There are many seminal theoretical frameworks related to technology acceptance, such as TAM [15, 16], IS Success Model [17], and Expectation Confirmation Model (ECM) of Information System Continuance [18]. These theoretical frameworks have been used for the evaluation of HIT as well. TAM [15] is the most widely used model of technology acceptance, which conceptualizes the perceptions and intentions people have in relation to accepting a particular technology, whereas the IS Success model [17] presents a process model which provides a multidimensional view of IS success. Additionally, the ECM of IS Continuance [18] compares expectations and initial perceptions of a system and focuses on continuance intention as an outcome.
2.1 IS success model
Delone and Mclean’s IS Success Model [17] suggested two factors for IS success, namely system quality and information quality, which have effects on use, user satisfaction, individual impacts, and ultimately organizational impacts. Extension of this model has been presented by the authors [19] and some other scholars [20, 21] with an additional success factor, service quality, and applied in evaluating many types of information systems. Another difference from the original model is that Delone and McLean [19] presented net benefits as an ultimate outcome, which can integrate all the “impact” measures such as individual impacts, organizational impacts, and so on.
Petter et al. [22] conducted a qualitative literature review of 180 studies found in the academic literature from 1992 to 2007 dealing with some aspects of IS success. Six dimensions of the DeLone and McLean model [19], such as system quality, information quality, service quality, use, user satisfaction, and net benefits, were used to analyze 90 empirical studies, and the results were synthesized. A total of 15 pairwise associations between the success constructs were examined. Many different kinds of IS under a variety of conditions were considered, and the majority of relationships were reasonably supported, suggesting the value of the DeLone and McLean model [19] when evaluating utilitarian IS.
Moreover, the IS success model has been applied in many HIT studies and provides empirical support in HIT as well [23, 24]. Garcia-Smith and Effken [25] developed the clinical information systems success model (CISSM) based on IS success model and UTAUT (Unified Theory of Acceptance and Use of Technology; [26]. This model was tested with the data from 234 registered nurses in four hospitals and significantly supported IS success model.
A big difference between TAM [15, 16] and IS success model is that TAM focuses on the acceptance of technology in terms of intention to use and actual use, whereas the IS success model focuses on distal outcomes after using information systems such as individual and organizational benefits. It is important and necessary to integrate and link these frameworks under a common logical structure with a common linking pin. In order to do this, it is necessary to have a further look at TAM.
2.2 Technology acceptance and IS continuance model
Two of the most applied models in technology acceptance are TAM and UTAUT. The original TAM [15, 16] proposed two major constructs influencing IT adoption, which are perceived usefulness and perceived ease of use. TAM has been further developed as UTAUT. Many studies showed that TAM and UTAUT could be successfully applied in predicting the acceptance behavior in the health care context [3, 8].
Specifically, perceived usefulness and perceived ease of use are found to have significant effects on technology acceptance and use in healthcare settings [27,28,29]. There are also some studies that successfully applied all four drivers of UTAUT in healthcare contexts [30,31,32]. However, there are some differences in the results of other studies depending on the type of information systems and characteristics of the system users [32]. Holden and Karsh [33] showed that effort expectancy was not a significant predictor of intention to use in six out of 13 HIT studies. Maillet et al. [34] studied 616 nurses’ satisfaction and actual use of EPR (Electronic Patient Record) from four hospitals and found that only performance expectancy influences both satisfaction and actual use and that facilitating conditions and effort expectancy has no significant effect on actual use.
Summarizing these findings, while TAM and UTAUT have been widely used as a general information systems evaluation framework, the findings in health care are quite dependent on the context of the study. That is why many authors added variables to the original TAM and UTAUT models or revised the survey instruments to better adapt to the context of health care.
Another extensively used model with regard to technology acceptance in the post-adoption stage is the Expectation Confirmation Model (ECM) of IS Continuance [18]. Adapted from the traditional Expectation Confirmation Theory (ECT) [35], the ECM is different from the ECT in several ways. First, while the ECT has a construct, “expectation”, which means pre-purchase expectation measured at t1, the ECM does not have a construct like pre-acceptance expectation but focuses only on post-acceptance expectation, represented by perceived usefulness. Originally in the ECT, consumers first have an initial expectation prior to purchase (t1). After the purchase and experience, consumers will compare the perceived performance (t2) with their initial expectations (t1). However, it is hard for users to have a realistic expectation of a new IS without first-hand experience of it, as IS belongs to experience goods whose value can only be assessed after they are experienced [36]. Even in mandatory IS, it may be harder for users to have any specific and defined expectations before use [37]. Thus, in many IS studies, users were given at least a short introduction to the system before the questionnaire survey to measure their initial expectations of the system. This is not the problem of the ECT but a methodological and contextual difference between traditional ECT and IS studies. Thus, whether it is purely a pre-use expectation or an initial post-use expectation is not a critical issue in IS study. The more important point is that both pre-use expectation and initial post-use expectation are initial perceptions that may change over time and provide the baseline or reference level for later perception.
The ECM of IS Continuance [18] suggested that the continuance intention of the information system is determined by user satisfaction and perceived usefulness. The model also showed that user satisfaction is determined by the confirmation of expectation and perceived usefulness. Many studies [10, 38,39,40,41] applied and updated this model. In addition, the ECM of IS Continuance has been applied in the health context as well. Palm et al. [42] suggested Information Technology Post Adoption Model (ITPAM) and conducted a clinical information system post-adoption evaluation by customizing the ECM of IS Continuance. Furthermore, Hadji and Degoulet [43] extended ITPAM (ITPAM2) with a construct (system quality) from IS success model and provided empirical support. ITPAM2 was further developed as UMISC (Unified Model of Information System Continuance [44]) and has been validated in some health settings [45].
2.3 Guiding theoretical framework
Notwithstanding the extensive use of TAM in IS research, it is still valuable to reposition TAM and its extended models in terms of conceptual, methodological, and replication treatments [46, 47]. One of them is the integration with other theories and the contextualization for specific settings [1, 48]. Lim et al. [49] addressed the problem of theoretical simplicity of TAM by integrating UTAUT and IS Success Model [17], and provided an exemplar of contextualizing technology acceptance theories for the specific setting (i.e., social media adoption by non-urban communities for non-profit activities). Moreover, Samhan [50] emphasized the importance of understanding the context as outcomes with the same technology may be different in different settings. Furthermore, the contextualization of technology acceptance needs to consider the type of technology enforcement, such as voluntary versus mandatory, utilitarian versus hedonic systems, etc. [50]. This paper tries to integrate three theories (UTAUT, IS success model, and ECM of IS Continuance) that share some common constructs, and take them as the starting point for a conceptual framework that describes the perceptions and intentions of clinicians in Australian hospitals regarding the use of eMMS. In this way, this study not only addresses the problem of theoretical simplicity of TAM by linking UTAUT, IS Success Model, and IS Continuance model, but also provides more practical value by contextualizing technology acceptance theories for a specific setting (i.e., mandatory eMMS in Australia).
TAM is an individual-level acceptance model and is mostly used for the voluntary use of IT systems. However, most systems in organizations, including HIT, are mandatory systems that users do not have a choice not to use. As eMMS is also a mandatory system, this study applies the constructs from UTAUT for the conceptual framework of this study due to its comprehensiveness and applicability to the organizational context. UTAUT is applicable in explaining technology acceptance in a mandatory setting as UTAUT focuses on large organizations in the business environment, which has a mandatory context [26, 51, 52]. Furthermore, the outcome variable of UTAUT, behavioral intention, needs to be modified to clinician satisfaction to fit the mandatory context, which can be used further as a linkage to connect existing theories. Moreover, TAM [15, 16] or UTAUT [26] emphasizes on user’s perspective, such as individual user’s acceptance. However, these models need to be extended to distal outcomes, which can mean “success” of IS because user satisfaction has a lack of robust theoretical underpinnings as a measure of IS success [53]. Hence, organizational benefits of using eMMS such as effectiveness, efficiency, and safety, need to be added as a measure of eMMS success.
Another common construct in the three theories is perceived usefulness, the degree to which users feel that the system is useful, which means the degree to which users think that the system helps their work and performance [26]. Perceived usefulness matches with the concept of system quality (e.g., usefulness of specific functions) and individual impact (e.g., performing tasks more quickly) in IS success model [17]. Thus, with this conceptual similarity, the perceived usefulness of eMMS in this study represents more integrative meaning which includes the clinician’s individual benefits as well as the quality of eMMS functions. Table 1 shows the mapping of the contextualized constructs of this study and constructs from three theories, which are used to link three theories to this study accordingly.
Last but not least, as this study focuses on how initial perceptions of eMMS change over time, a core guiding theoretical framework of this study is ECM of IS Continuance [18]. The major construct of ECM is confirmation, which compares expectation and actual performance and, in turn, confirms to what extent actual performance is better than expected. The methods of operationalizing confirmation in the ECT literature are divided into three ways: objective, inferred, and perceived [54]. Objective confirmation is not an appropriate way for the experience goods or services like eMMS as they are normally not assessed based on the objective criteria. The ECM employed the method of perceived confirmation, which compares perceptions of t2 and t1 retrospectively at the time of t2 without separately measuring at t1. However, it is hard for users to recall the initial perceptions at a later stage (e.g., one year later). In contrast, inferred confirmation measures both expectation (t1) and actual performance (t2) separately, and it calculates the difference between expectation and actual performance. In this study, both perceptions at each stage are measured respectively. Thus, initial perception can be regarded as expectation, and perception one year later can be regarded as actual performance.
3 Hypothesis
Drawing on the works by Venkatesh et al. [26], DeLone and Mclean [17], and Bhattacherjee [18], this paper proposes a research model on key factors that influence clinician satisfaction and organizational benefits of eMMS. The three theories were connected with clinician satisfaction as the linking pin, and further adapted to a mandatory IS context. In addition, this study looks at not only the relationships at a particular stage, but also how they change from the initial adoption stage to the initial assimilation stage. Figure 2 shows the research model proposed in this paper, which is trying to test a set of hypotheses developed accordingly. The clinician’s perceptions between the initial adoption stage and the initial assimilation stage will be compared with this research model to see if there is any difference. The control variables such as user groups (doctors, nurses, pharmacists) and clinical work experience are included in this research model but not hypothesized as they are used only for controlling extraneous effects.
Perceived ease of use, or effort expectancy, of eMMS refers to the extent of ease associated with the use of eMMS [26]. If clinicians feel that eMMS is very complicated and hard to use, they will not be satisfied with eMMS. Conversely, if they feel eMMS is user-friendly and easy to use, their satisfaction will increase. Particularly, perceived ease of use of eMMS in this specific context stands for the concept of relative ease than the old system (paper-based medication), which gives a novel perspective to UTAUT constructs. It has been found that healthcare professionals’ perceived ease of use has a positive effect on behavioral intention to use health IT [55,56,57]. This is because the relationship between perceived ease of use and behavioral intention has a hidden mediator, user attitude, which means emotion toward health IT itself. The original model of TAM shows such a mechanism that users’ beliefs lead to users’ intention to use a system via user’s attitude. User satisfaction is conceptually similar to user attitude and one of the most comprehensive, appropriate, and important evaluations of IS success [17].
Some literature showed that perceived ease of use of health IT is also a factor positively affecting clinicians’ satisfaction in using health IT [31, 58, 59]. Perceived ease of use in the initial adoption stage is the post-acceptance expectation with very little and superficial use experience of eMMS. This initial post-acceptance expectation will be confirmed positively in the initial assimilation stage as perceived ease of use can be increased by enough use of eMMS for one year. In other words, clinicians may feel that eMMS is getting easier to use as time goes on due to the repetitive use of eMMS for one year. However, the effect of perceived ease of use on clinician satisfaction will be decreased in the initial assimilation stage, and perceived ease of use may not necessarily serve as an influencing factor of clinician satisfaction as clinicians might be already accustomed to the functions of eMMS and ease of use itself does not affect clinician satisfaction anymore. In fact, Karahanna et al. [11] found that effort expectancy has non-significant effects on attitude toward continuance to use in the later stage. Maillet et al. [34] also showed that effort expectancy has no significant effect on the actual use of health IT in the post-implementation stage. Based on such theoretical backgrounds, the following hypotheses are suggested:
H1a
Clinicians’ perceived ease of use on eMMS has a positive effect on clinician satisfaction with eMMS in the initial adoption stage.
H1b
Clinicians’ perceived ease of use on eMMS has no effect on clinician satisfaction with eMMS in the initial assimilation stage.
H1c
The effect of perceived ease of use on clinician satisfaction in the initial adoption stage is greater than the effect in the initial assimilation stage.
H1d
The level of clinicians’ perceived ease of use on eMMS in the initial assimilation stage is higher than in the initial adoption stage.
Perceived usefulness of eMMS in this study is defined as the extent to which a clinician perceives that using eMMS will help him or her to attain gains in job performance [26]. Enhanced job performance is instrumental in achieving various rewards in an organization, such as promotions or monetary gains [18, 60]. As human tends to subconsciously pursue instrumental behaviors or strive for rewards, perceived usefulness is an individual benefit and extrinsic motivation to activate the positive emotional state [18]. In the context of this study, perceived usefulness of eMMS also includes the concept of relative usefulness than the old system (paper-based medication), which contextually means perceived individual improvements in medication management such as increased time saving and convenience. If clinicians think that eMMS is more beneficial for their task performance than the old system, they will be happy to use eMMS. Many studies showed that perceived usefulness, or similarly, performance expectancy, has a positive effect on behavioral intention to use Health IT [61,62,63] or clinician satisfaction in using Health IT [64, 65]. As discussed in hypothesis 1, the relationship between perceived usefulness and behavioral intention also has a hidden mediator, user attitude, so the underlying mechanism of this relationship is the same as the one in hypothesis 1. Perceived usefulness is a cognitive belief noticeable to IS use, which also forms a post-acceptance expectation in the initial acceptance stage, and plays a role of confirmed performance in the initial assimilation stage. In the initial adoption stage, clinicians’ perceived usefulness is relatively lower than in the initial assimilation stage as clinicians are not able to feel the full usefulness of eMMS due to the very short time of use experience of eMMS. In addition, the effect of perceived usefulness of eMMS on clinician satisfaction in the initial assimilation stage will be greater than in the initial adoption stage as clinicians will be satisfied more if they experience actual usefulness fully for one year. Alsyouf and Ishak [66] supported this argument, which showed that nurses’ performance expectancy of EHR had a positive effect on continuous intention to use EHR in the later stage. Thus, the perceived usefulness of health IT can influence a clinician’s intention to use health IT regardless of whether it is for the first time use or continued use. Based on these, the following hypotheses are proposed:
H2a
Clinician’s perceived usefulness on eMMS has a positive effect on clinician satisfaction with eMMS in the initial adoption stage.
H2b
Clinician’s perceived usefulness on eMMS has a positive effect on clinician satisfaction with eMMS in the initial assimilation stage.
H2c
The effect of perceived usefulness on clinician satisfaction in the initial assimilation stage is greater than the effect in the initial adoption stage.
H2d
The level of clinician’s perceived usefulness on eMMS in the initial assimilation stage is higher than in the initial adoption stage.
In this study, facilitating conditions reflect the organizational supports and technical infrastructure of a hospital for using eMMS. Generally, facilitating conditions means an ICT help desk or a call centre which provides users with remote assistance services in most cases. In the context of eMMS, however, the ICT support team provided 24 × 7 (24 h a day, 7 days a week) services, having people on site in order to go to the point of care where issues are being raised. This is crucial point for the users as it gives completely different feeling that they are well catered and well supported. Other facilitating conditions specific to the context of this study are adequate user training on eMMS, a 2:1 device-to-patient ratio, and so on. If clinicians feel that they are well supported by the hospital in using eMMS, they may have satisfaction of the hospital, which in turn leads to satisfaction in using eMMS. Some HIT studies supported this argument, which showed that facilitating conditions have a positive effect on behavioral intention to use health IT [67, 68]. In addition, Garcia-Smith and Effken [25] showed that facilitating conditions in hospitals have a positive effect on clinician satisfaction. Along with other beliefs such as perceived ease of use and perceived usefulness, facilitating conditions in the initial adoption stage is also a post-acceptance expectation with little experience related to eMMS use. This initial evaluation on facilitating conditions will decrease in the initial assimilation stage as clinicians may feel that they do not need much support from IT help desk if they become proficient in using eMMS. In addition, facilitating conditions may not have a significant effect on clinician satisfaction, and the effect of facilitating conditions on clinician satisfaction will also be decreased in the initial assimilation stage as facilitating conditions will become less important in making clinicians satisfied. Some post-implementation studies of health IS supported this argument. Mellikeche et al. [44] showed that facilitating conditions of hospitals have no significant effect on user satisfaction in the later stage. Hadji et al. [69] also suggested that facilitating conditions such as technical support and user training are not determinant in shaping clinicians’ satisfaction in clinical IS use during the later stage. Thus, the following hypotheses can be derived:
H3a
Facilitating conditions of hospitals have a positive effect on clinician satisfaction with eMMS in the initial adoption stage.
H3b
Facilitating conditions of hospitals have no effect on clinician satisfaction with eMMS in the initial assimilation stage.
H3c
The effect of facilitating conditions on clinician satisfaction in the initial assimilation stage is less than the effect in the initial adoption stage.
H3d
The level of facilitating conditions of hospitals in the initial assimilation stage is lower than in the initial adoption stage.
Next, social influence among clinicians is defined as the extent to which other clinicians’ belief on eMMS use has an influence on an individual clinician’s perception [26]. In this study context, social influence among clinicians is mostly about colleagues’ favorable opinions on eMMS use, which can be their perceived benefits. Moreover, social influence in the context of this study is closer to informational influence rather than normative influence [70] as clinicians have relatively high autonomy in the practice of their profession [71]. Another thing special in this context is that there have been proactive and well-informed superusers (1:6 ratio) who act as support to their colleagues in eMMS implementation, which may be a factor that influences other clinicians’ perception on eMMS. If most of the peers agree that the way of working with eMMS is a good idea, favorable peer opinions can increase clinicians’ satisfaction with eMMS. Some health IT studies supported this argument, which showed that social influence among clinicians has a positive effect on behavioral intention to use health IT [61, 72, 73]. Social influence among clinicians is also a post-acceptance expectation, which will be confirmed positively in the initial assimilation stage as more colleagues will become favorable to eMMS with proficiency and adaptation of eMMS for one year. Increased social influence among clinicians will have an effect on clinician satisfaction in the initial assimilation stage, and the effect will be greater than in the initial adoption stage. Kumar and Natarajan [74] supported this argument, which has shown that social influence had a positive effect on continuance intention to use an e-health service in the later stage. Based on these, the following hypotheses can be proposed:
H4a
Social influence among clinicians in a hospital has a positive effect on clinician satisfaction with eMMS in the initial adoption stage.
H4b
Social influence among clinicians in a hospital has a positive effect on clinician satisfaction with eMMS in the initial assimilation stage.
H4c
The effect of social influence among clinicians on clinician satisfaction in the initial assimilation stage is greater than the effect in the initial adoption stage.
H4d
The level of social influence among clinicians in the initial assimilation stage is higher than in the initial adoption stage.
Organizational benefits are defined as the increase in organizational outcomes and the decrease in organizational inputs [75]. In the context of eMMS use, organizational benefits can be improvements in quality, safety, and effectiveness of medication management and reduction of medication errors and adverse drug events, and so on. If clinicians are happy to use eMMS use, they will positively think that eMMS will be beneficial for the hospital as well. Gelderman [76] showed significant correlations between user satisfaction and organizational impact. In the updated IS Success Model [19], Delone and Mclean also showed that user satisfaction has a positive influence on net benefits. Moreover, Garcia-Smith and Effken [25] have found that clinician satisfaction has a positive effect on net benefits. As discussed in Table 1, organizational impact and net benefits are conceptually the same as organizational benefits of eMMS in this study. Thus, these studies empirically supported our arguments. Organizational benefits of eMMS can be increased in the initial assimilation stage, and the effect of clinician satisfaction on organizational benefits will also be increased in the initial assimilation stage as the benefits from IT may not appear immediately but can be felt more after some time [77]. Clinician satisfaction can also be increased in the initial assimilation stage as some concerns regarding eMMS use in the initial adoption stage may be resolved already during one year. According to the mere ownership effect [78], people would evaluate an object more favorably merely because they owned it. This theory can also be applied to the context of clinician satisfaction and organizational benefits in this study. Based on this theoretical background, the following hypotheses can be suggested:
H5a
Clinician satisfaction with eMMS has a positive effect on organizational benefits of eMMS in the initial adoption stage.
H5b
Clinician satisfaction with eMMS has a positive effect on organizational benefits of eMMS in the initial assimilation stage.
H5c
The effect of clinician satisfaction on organizational benefits of eMMS in the initial assimilation stage is greater than the effect in the initial adoption stage.
H5d
The level of clinician satisfaction in the initial assimilation stage is higher than in the initial adoption stage.
H5e
The level of organizational benefits of eMMS in the initial assimilation stage is higher than in the initial adoption stage.
4 Research methods
This study is to conduct predictive- and prescriptive-oriented research (i.e., establish cause and effect) rather than descriptive- and association- (e.g., exploratory insights) oriented research. Moreover, in order to see the differences in impact over time, this study takes a conditional approach in which time period (initial adoption stage and initial assimilation stage) can be the boundary condition [79]. With the data collected from two points in time, a quantitative approach is adopted to empirically validate the hypotheses presented in the previous section. In this section, the study setting, data collection procedure, and survey design are described.
4.1 Study setting
The research model is examined using survey data collected from a public tertiary teaching hospital in Sydney, with about 1200 employees. The case hospital was in the position of a leader in electronic medication record (EMR) implementation in New South Wales and had been proactively adopting digital transformation initiatives. Unlike most Australian hospitals that have implemented eMMS prior to a hospital-wide EMR, which consequently led to prolonged adoption [80], the case hospital is the first site in Australia to have full EMR utilisation for 2 years prior to eMMS implementation. The experience of implementing EMR prior to eMMS had raised awareness and readiness of electronic health care.
The eMMS implemented in the case hospital is composed of different modules to support a variety of medication management processes such as prescription, verification, reconciliation, and administration of medication. The different modules are used by different user groups, including doctors, pharmacists, and nurses. For example, modules related to medication prescription processes are used by doctors, whereas modules related to the medication administration processes are used by nurses, and those related to medication verification and dispensing processes are used by pharmacists. There are also modules used by multiple user groups, such as those related to medication reconciliation processes.
4.2 Data collection
A survey study was conducted with clinicians regarding their perceptions of the implementation of eMMS, and it was divided into two stages. The initial adoption survey was conducted for about six weeks in 2017. The questionnaire was distributed to the users of eMMS, including doctors, nurses, and pharmacists. In the initial adoption survey, a total of 202 responses were collected, which included responses from 65 junior doctors (32%), 24 senior doctors (12%), 101 nurses (50%), and 12 pharmacists (6%). The survey in the initial assimilation stage was conducted one year after the initial adoption survey. An online questionnaire was first distributed to the users of eMMS. The online questionnaire was followed by a paper questionnaire distributed to the users in different wards who had not responded to the online questionnaire. In the initial assimilation stage survey, 196 complete responses were collected, including responses from 74 junior doctors (38%), 21 senior doctors (11%), 93 nurses (47%), and 8 pharmacists (4%).
Since this research involved the participation of humans, formal approval from an institutional review board was compulsory. This research was reviewed and approved by a Human Research Ethics Committee and has been undertaken in accordance with the National Statement on Ethical Conduct in Human Research [81]. As per the requirement for consent [81], all participations were voluntary and only the subjects who had provided consent to the survey were given the questionnaire. Moreover, the participants were informed that their answers would be kept anonymous and confidential. In addition, the research team checked whether participants were eligible users of eMMS.
The data collection is conducted using mainly convenience sampling where participants are selected based on their accessibility to the research.. The key advantages of convenience sampling are that it is cheaper, more efficient, less time-intensive, easier, and simpler to implement than other sampling methods [82]. However, the key disadvantage of convenience sampling is that as not every unit in the population has a chance of being selected in the sample, it can have a self-selection bias, so the scientific generalization derived from convenience sampling is limited [83]. To complement this disadvantage of convenience sampling, the research team contacted as many clinicians as possible to collect data in person by visiting ‘ward by ward’. As such, the data collection for each survey took more than five weeks (much longer than originally planned) and almost all eligible eMMS users in the hospital were at least given a chance to be included in the sample. As a result, the risk of self-selection bias was minimized and the shortcomings of convenience sampling are deemed minimal in this research.
4.3 Measures
The survey items were adapted from extant literature and theories such as UTAUT [26] and the same items are used in both surveys to compare the results in the initial adoption stage and initial assimilation stage. Items for perceived ease of use, perceived usefulness, social influence among clinicians, and facilitating conditions of hospitals were adapted from UTAUT [26]. Items for clinician satisfaction were adapted from Sirdeshmukh et al. [84] and Kim and Kankanhalli [85]. Items for organizational benefits were adapted from Moore and Benbasat [86] and Kim and Kankanhalli [85]. Table 2 summarizes the items used in the survey. All the items were measured using 7-point Likert scale (from strongly disagree to strongly agree). To establish the face and content validity of the items, a pre-test was undertaken by three IS researchers and four clinical researchers including a doctor, a nurse, and two pharmacists.
Since there are multiple groups of users in eMMS such as doctors, nurses, and pharmacists, and these different group of users use different modules of eMMS, there could be a possibility of differences between user groups. Hence, to minimize such potential biases, the control variables such as user group and work experience of respondents are included in the analysis. These data of the control variables were collected together with other measures.
5 Results
5.1 Measurement model
The measurement model was assessed for convergent and discriminant validity by performing a confirmatory factor analysis (CFA) using Smart PLS 2.0 [87]. Firstly, convergent validity was investigated by the standardized factor loadings, CR (Composite Reliability), and AVE (Average variance extracted). As can be seen in Tables 3 and 4, the standardized factor loadings were all significant (p value < 0.001) and greater than 0.7. The values of CR for all latent variables are greater than 0.7, and the values of AVE for all latent variables are also greater than 0.5. Hence, the convergent validity of the survey instrument was supported. Secondly, discriminant validity was examined with the square root of AVE for each latent variable and the correlations between the latent variables. As shown in Tables 3 and 4, the square root of AVE for each latent variable (diagonal element) is greater than the correlations between latent variables (off-diagonal elements). Thus, the discriminant validity of the survey instrument was supported.
As this study is based on the survey data completed by respondents themselves, there can be vulnerability to common method bias, which can have a significant impact on research findings [88]. For this reason, two approaches are adopted to identify common method bias. One is a full collinearity test [89, 90], and the other is a comprehensive CFA marker technique [91]. Firstly, a full collinearity test was conducted for both 2017 and 2018 datasets and generated VIF (Variance Inflation Factor). If any VIF is equal to or greater than 3.3, then it can be concluded that common method bias is present. The results showed that all VIFs of latent variables in both 2017 and 2018 range from 1.082 to 2.862, all below the recommended threshold of 3.3 [89, 90]. Secondly, following the procedures of a comprehensive CFA marker technique described in [91], the Method-C model was compared with the baseline model to test the presence of method variance associated with the marker variable. The null hypothesis that the method factor loadings associated with the marker variable are not related to each of 18 substantive indicators was tested by the chi-square difference test comparing the two models. The result in the initial adoption stage showed that the method-C model (Chi-square = 304.7, df = 158, CFI = 0.965, RMSEA = 0.068) does not fit significantly better than the baseline model (Chi-square = 305.1, df = 159, CFI = 0.965, RMSEA = 0.068). The result in the initial assimilation stage also showed that the method-C model (Chi-square = 299.8, df = 158, CFI = 0.971, RMSEA = 0.068) does not fit significantly better than the baseline model (Chi-square = 305.1, df = 159, CFI = 0.971, RMSEA = 0.067). Thus, there is no evidence of shared common method variance between the indicators of the substantive variables and the latent marker variables. In conclusion, both approaches indicate that common method bias is deemed not a concern in this study.
5.2 Comparison of construct means
Table 5 shows the descriptive statistics of five latent variables in the initial adoption stage and initial assimilation stage. The means of perceived ease of use, perceived usefulness, social influence, clinician satisfaction, and organizational benefits in the initial assimilation stage are greater than those in the initial adoption stage, providing statistical support for hypotheses H1d, H2d, H4d, H5d, and H5e respectively. Only facilitating conditions have no statistical difference in the mean between the initial adoption stage and the initial assimilation stage. Thus, hypothesis H3d is not supported statistically.
5.3 Structural model
Since CFA results showed the validity of the underlying structure of the items measured, we moved to the testing of the structural model. The path analysis of the structural equation model was performed using Smart PLS 2.0 [87] for testing the hypotheses 1–5 proposed in the previous section. The results of the structural model are shown in Figs. 3 and 4.
In the initial adoption stage, all four factors showed significant effects on clinician satisfaction. Among these four factors, perceived usefulness has the highest positive significant effect on clinician satisfaction (path-coefficient = 0.453, p < 0.001), supporting hypothesis H2a. The other three antecedents also have significant positive effects on clinician satisfaction. Perceived ease of use has a lower path-coefficient of 0.227 (p < 0.001), providing empirical support for hypothesis H1a. Similarly, social influence among clinicians shows the path-coefficient of 0.193 (p < 0.001), supporting hypothesis H4a. Facilitating conditions indicate the lowest path-coefficient of 0.134 (p < 0.05), providing statistical support for hypothesis H3a. A total of 75.9 percent of the variance of clinician satisfaction is explained by the above four relationships. Moreover, clinician satisfaction has a significant positive effect on organizational benefits, with a standardized effect size of 0.668 (p < 0.001), statistically supporting hypothesis H5a, which accounts for 42.9 percent of the variance of organizational benefits.
In the initial assimilation stage, only two factors indicated significant effects on clinician satisfaction, which accounts for 74.4 percent of the variance of clinician satisfaction. Perceived usefulness has a higher path-coefficient of 0.543 (p < −0.001), providing empirical support for hypothesis H2b. Social influence among clinicians also has a significant influence on clinician satisfaction with the path-coefficient of 0.257 (p < 0.001), supporting hypothesis H4b. Perceived ease of use and facilitating conditions have no significant effects on clinician satisfaction. Thus, hypotheses H1b and H3b are also supported. Additionally, clinician satisfaction has a significant positive effect on organizational benefits with the path-coefficient of 0.815 (p < 0.001), empirically supporting hypothesis H5b, which accounts for 64.1 percent of the variance of organizational benefits.
The overall results suggest that two factors, perceived usefulness, and social influence, continue to play an important role in driving clinician satisfaction over one year. Whereas the other two factors, perceived ease of use and facilitating conditions, only influence in the first year. Moreover, the result in the initial assimilation stage showed that clinician satisfaction explains a greater amount of the variance in organizational benefits (64.1%) than in the initial adoption stage (42.9%).
5.4 Direct effects over time
The effects of perceived ease of use and facilitating conditions on clinician satisfaction were found to be significant in the initial adoption stage but become insignificant in the initial assimilation stage. To determine whether the changes were significant over time, a multi group analysis using Smart PLS was conducted to test the magnitude and statistical significance of the difference for the coefficients in initial adoption and initial assimilation stages.
As shown in Table 6 above, only the coefficient of clinician satisfaction on organizational benefits has a statistically significant difference between the initial adoption stage and the initial assimilation stage, supporting hypothesis H5c. Hypotheses H1c, H2c, H3c, and H4c are not statistically supported.
5.5 Indirect effects
As four drivers seem to have indirect effects on organizational benefits through clinician satisfaction, a post-hoc analysis to examine the mediation effects of clinician satisfaction has been performed. The results of the test are shown in Table 7. The indirect effects of perceived ease of use, perceived usefulness, social influence among clinicians, and facilitating conditions of hospitals on organizational benefits through the mediator (clinician satisfaction) are all significant in the initial adoption stage. However, the indirect effects of perceived ease of use and facilitating conditions of hospitals on organizational benefits through clinician satisfaction have become insignificant in the initial assimilation stage. In addition, the indirect effects between the initial adoption stage and the initial assimilation stage are not significantly different, which means that there is no statistically differential impact over time.
6 Discussion
Our results suggest that the factors have slightly different effects on clinician satisfaction and organizational benefits between the initial adoption stage and the initial assimilation stage. In the initial adoption stage, all four factors significantly affect clinician satisfaction and further organizational benefits, whereas in the initial assimilation stage, only perceived usefulness and social influence among clinicians have a significant effect on clinician satisfaction and further organizational benefits. The importance of perceived ease of use and facilitating conditions have been moving toward a decrease after one year but the degree of their decrease is not enough to be statistically significant yet.
6.1 Confirmation of expectation (initial perception)
As shown in Table 5, the mean differences in perceived usefulness, perceived ease of use, social influence among clinicians, clinician satisfaction, and organizational benefits are statistically significant between the initial adoption stage and the initial assimilation stage. As perceptions in the initial stage can be regarded as expectations, the results can be interpreted as confirming the difference between the expectation and initial assimilation stage perception (actual performance).
The results also suggest that eMMS has moved beyond the initial trial and has become embedded into the routines of a hospital. Clinicians’ perceptions of the same eMMS have changed more positively in a year. The status quo bias theory [92] can provide a plausible interpretation of such results. As people prefer the current state, there can be resistance to the change. However, once they get used to the change, such a change becomes the current state. The eMMS was a change for the clinicians in the initial adoption stage, so the resistance (or reversely satisfaction) is higher (or lower). After one year, however, eMMS became the current state, which reduces (or increases) the resistance (or satisfaction).
6.2 Change of importance
Individual users are not used to a new IT system at the beginning so they may require some experience prior to becoming proficient [77]. According to the learning effect or learning curve [93], workers tend to need less time to perform tasks if task repetition occurs. As clinicians have been using eMMS repetitively for one year, they may become proficient due to familiarity with the operation and tools of eMMS. Once they become proficient, their perception of ease of use will be greater, but that is not the factor that leads to clinician satisfaction. Unfortunately, the decrease in importance over the one-year period is not statistically significant. However, the coefficient values have decreased over the period, suggesting that perceived ease of use has become a less important determinant for the subsequent success of eMMS over time.
Similarly, the importance of facilitating conditions has decreased over the one-year period, but the difference is not statistically significant. Clinicians may feel that they need less help from the IT help desk if they become proficient in using eMMS. Moreover, facilitating conditions such as user training may have an influence on the workers’ learning process. The skill learning process of the workers can be autonomous learning with the passage of time or induced learning by training [94]. Facilitating conditions are related to induced learning which is normally provided during the roll-out period. Venkatesh et al. [26] showed that perceived behavioral control, a root construct of facilitating conditions, has a significant effect on behavioral intention immediately following training (T1), but the effect disappears by the later stage (T2). Thus, users may not think that facilitating conditions in the initial assimilation stage are better than in the initial adoption stage. Consequently, facilitating conditions are not a statistically significant factor influencing clinician satisfaction in the initial assimilation stage in spite of its statistically insignificant decrease in importance.
Unlike perceived ease of use and facilitating conditions, the impact of clinician satisfaction on organizational benefits is significantly different between the two stages. This change of importance statistically means that the effect of clinician satisfaction on organizational benefits has increased after one year. Thus, it can be concluded that the impact of clinician satisfaction on organizational benefits is even stronger in the initial assimilation stage as clinicians may have felt actually realized benefits for one year.
In terms of the indirect effect, the impact of perceived ease of use and facilitating conditions is also moving toward a decrease, but there is no statistically significant difference between the two stages. Again, once clinicians become proficient in eMMS, their perception of ease of use and facilitating conditions does not significantly lead to clinician satisfaction, even though the decrease of the impact is not enough to be statistically significant.
6.3 Theoretical implications
With suggested explanations for changes in user evaluations on eMMS in an Australian hospital, this study contributes to the understanding of temporal dynamics of important beliefs on IS use. The difference between this study and other IS continuance studies is that this study compares the early experience and the later experience but IS continuance studies generally compare expectations and experiences. Expectations can be either confirmed or disconfirmed after actual use. However, there are no special reasons to measure the expectations of users before adoption in a mandatory context. Moreover, little research has been carried out in comparing user evaluations at two different time points, especially at the one-year interval. Thus, this study can contribute especially to the longitudinal studies of mandatory information systems.
This study provides some theoretical implications and contributes to the body of knowledge, especially in the health IS field. As many scholars suggested that technology should be studied with its social context [50, 95,96,97], this study explains changes in clinician’s perception between the initial adoption stage and initial assimilation stage in a specific health setting, which may be different when implementing same technology in another context. This is the first novelty of this study, and the contextualization of technology acceptance in this study is in line with the recommendations by Lim [46] as this study considers the type of technology enforcement, such as mandatory enforcement of utilitarian system (eMMS).
In addition, a major contribution is in linking technology acceptance theory, IS success model, and IS Continuance Model to provide a holistic view of how users evaluate a new information system in a health setting. This linkage to adapt UTAUT and other two theories to the context of eMMS also answers the call by Lim [46, 47] as they illustrate novel and meaningful findings which show the emerging realities of user perception in less researched settings (e.g., health). In addition, as this study focuses not just on technology acceptance but on the changes in clinicians’ perception in the initial assimilation stage, it is a different replication treatment to test whether variables from three major theories can suit the context of changes in clinicians’ perceptions. Furthermore, the adaptation mechanism in this study will help future research to employ the same mechanism for adaptation to generate a novel theory based on reconfiguration of existing theory.
6.4 Practical implications
This study also provides practical implications. This study suggests what factors the hospital’s management should focus on for the success of eMMS during both the initial adoption and initial assimilation stages. In the initial adoption stage, all four factors are important. However, as can be seen in the results, the relative importance of the factors slightly changes over time. Though expectations or initial perceptions have been positively confirmed in the initial assimilation stage, confirming something less important is not that important practically. Thus, confirming perceived ease of use and facilitating conditions does not matter as the importance of such factors is naturally moving toward a decrease in the initial assimilation stage. Confirming something still important after one year has more practical implications.
This study shows that perceived usefulness and social influence among clinicians are critical factors for subsequent success as well as initial success. Thus, work compatibility, usefulness, and utility for the users should be considered a high priority when introducing IT. Additionally, the hospital’s management should promote positive word-of-mouth through influential individuals who have a strong social influence on other users.
6.5 Limitations and future research
The findings of this study should be interpreted together with its limitations. First of all, this study was conducted in a single context, i.e., one Australian hospital with a specific information system, eMMS. Thus, this study has a limitation in applying to other contexts or being a generalized theory at this stage. Currently, this study should be interpreted in the context of eMMS, and future studies could validate this study’s results in other contexts. Secondly, this study is based on self-report survey data on users’ perceptions. Thus, the ultimate outcome construct, organizational benefits, may not actually be realized benefits. Thirdly, the effects of perceived ease of use and facilitating conditions have no statistically significant difference between the two stages. This result may not be conclusive until further examination with more than one-year interval is conducted. Future studies need to see when and whether the impact of perceived ease of use and facilitating conditions becomes significantly different. Additionally, qualitative data such as interviews with different user groups need to be added to the survey data to provide more in-depth and specific implications for the healthcare context.
7 Conclusion
This paper tried to mainly understand how user evaluations change over one year and examine whether there are any differences in factors influencing clinician satisfaction and organizational benefits between the initial adoption stage and the initial assimilation stage. The results showed that perceived ease of use and facilitating conditions have no significant effect on clinician satisfaction in the initial assimilation stage, whereas all factors have an effect in the initial adoption stage. The means of perceived ease of use, perceived usefulness, social influence, clinician satisfaction, and organizational benefits in the initial adoption stage have significantly increased in the initial assimilation stage, meaning that user evaluations have changed positively after one year. It is believed that the results and experiences of this study can benefit other researchers in the field of study.
References
Kim ED (2022) Multifaceted understanding of user behaviour and implementation success of electronic medication management system. The University of Sydney, Sydney, Australia
Ford EW, Menachemi N, Huerta TR, Yu F (2010) Hospital IT adoption strategies associated with implementation success: implications for achieving meaningful use. J Healthc Manag 55(3):175–188
Kim ED, Kuan KK, El Amrani R, Vaghasiya MR, Penm J, Gunja N, Poon S (2019) Understanding health information technology assimilation at individual level: the case of electronic medication management system. In: Proceedings of the 27th European Conference on Information Systems (ECIS), Stockholm & Uppsala, Sweden, June 8–14, 2019. ISBN 978-1-7336325-0-8 Research Papers. https://aisel.aisnet.org/ecis2019_rp/97
Sulaiman H, Wickramasinghe N (2014) Assimilating healthcare information systems in a Malaysian Hospital. CAIS 34:77
Baysari MT, Hardie RA, Lake R, Richardson L, McCullagh C, Gardo A, Westbrook J (2018) Longitudinal study of user experiences of a CPOE system in a pediatric hospital. Int J Med Inform 109:5–14
Hadji B, Martin G, Dupuis I, Campoy E, Degoulet P (2016) 14 Years longitudinal evaluation of clinical information systems acceptance: the HEGP case. Int J Med Inform 86:20–29
Pynoo B, Devolder P, Voet T, Sijnave B, Gemmel P, Duyck W et al (2013) Assessing hospital physicians’acceptance of clinical information systems: a review of the relevant literature. Psychol Belg 53(2):15–31
Ammenwerth E (2019) Technology acceptance models in health informatics: TAM and UTAUT. Stud Health Technol Inform 263:64–71
Hogarth RM, Einhorn HJ (1992) Order effects in belief updating: the belief-adjustment model. Cogn Psychol 24(1):1–55
Venkatesh V, Thong JY, Chan FK, Hu PJH, Brown SA (2011) Extending the two-stage information systems continuance model: incorporating UTAUT predictors and the role of context. Inf Syst J 21(6):527–555
Karahanna E, Straub DW, Chervany NL (1999) Information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. MIS Q 66:183–213
Sun Y, Jeyaraj A (2013) Information technology adoption and continuance: a longitudinal study of individuals’ behavioral intentions. Inf Manag 50(7):457–465
NSW Health Ministry (2017) Over half a billion dollars injected into digital healthcare conversion in NSW. Healthcare IT News. https://www.healthcareit.com.au/article/over-half-billion-dollars-injected-digital-healthcare-conversion-nsw.
Wright RT, Roberts N, Wilson D (2017) The role of context in IT assimilation: a multi-method study of a SaaS platform in the US nonprofit sector. Eur J Inf Syst 26(5):509–539
Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 66:319–340
Davis FD, Bagozzi RP, Warshaw PR (1989) User acceptance of computer technology: a comparison of two theoretical models. Manag Sci 35(8):982–1003
DeLone WH, McLean ER (1992) Information systems success: the quest for the dependent variable. Inf Syst Res 3(1):60–95
Bhattacherjee A (2001) Understanding information systems continuance: an expectation-confirmation model. MIS Q 66:351–370
DeLone WH, McLean ER (2003) The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst 19(4):9–30
Petter S, Mclean ER (2009) A meta-analytic assessment of the DeLone and McLean IS success model: an examination of IS success at the individual level. Inf Manag 46(3):159–166
Pitt LF, Watson RT, Kavan CB (1995) Service quality: a measure or information systems effectiveness. MIS Q 19(2):173–188
Petter S, DeLone W, McLean E (2008) Measuring information systems success: models, dimensions, measures, and interrelationships. Eur J Inf Syst 17(3):236–263
Jen WY, Chao CC (2008) Measuring mobile patient safety information system success: an empirical study. Int J Med Inform 77(10):689–697
Lian JW (2017) Establishing a cloud computing success model for hospitals in Taiwan. INQUIRY J Health Care Organ Provis Financ 54:46958016685836
Garcia-Smith D, Effken JA (2013) Development and initial evaluation of the clinical information systems success model (CISSM). Int J Med Inform 82(6):539–552
Venkatesh V, Morris MG, Davis GB, Davis FD (2003) User acceptance of information technology: toward a unified view. MIS Q 66:425–478
Djamasbi S, Fruhling AL, Loiacono E (2009) The influence of affect, attitude and usefulness in the acceptance of telemedicine systems. J Inf Technol Theory Appl 10(1):66
Lu CH, Hsiao JL, Chen RF (2012) Factors determining nurse acceptance of hospital information systems. CIN Comput Inform Nurs 30(5):257–264
Tubaishat A (2018) Perceived usefulness and perceived ease of use of electronic health records among nurses: application of technology acceptance model. Inform Health Soc Care 43(4):379–389
Hennington A, Janz BD (2007) Information systems and healthcare XVI: physician adoption of electronic medical records: applying the UTAUT model in a healthcare context. Commun Assoc Inf Syst 19(1):5
Kim ED, Kuan KK, Vaghasiya M, Gunja N, Penm J, Poon S (2017) Evaluating user satisfaction and organisational benefits of electronic medication management system in an Australian Hospital. In: ACIS 2017 proceedings, vol 73. https://aisel.aisnet.org/acis2017/73
Kim S, Lee KH, Hwang H, Yoo S (2015) Analysis of the factors influencing healthcare professionals’ adoption of mobile electronic medical record (EMR) using the unified theory of acceptance and use of technology (UTAUT) in a tertiary hospital. BMC Med Inform Decis Mak 16(1):12
Holden RJ, Karsh BT (2010) The technology acceptance model: its past and its future in health care. J Biomed Inform 43(1):159–172
Maillet É, Mathieu L, Sicotte C (2015) Modeling factors explaining the acceptance, actual use and satisfaction of nurses using an Electronic Patient Record in acute care settings: an extension of the UTAUT. Int J Med Inform 84(1):36–47
Oliver RL (1980) A cognitive model of the antecedents and consequences of satisfaction decisions. J Mark Res 17(4):460–469
Brynjolfsson E, Hu YJ, Rahman MS (2013) Competing in the age of omnichannel retailing. MIT, Cambridge, pp 1–7
Khalifa M, Liu V (2004) The state of research on information system satisfaction. J Inf Technol Theory Appl 5(4):4
Bhattacherjee A, Lin CP (2015) A unified model of IT continuance: three complementary perspectives and crossover effects. Eur J Inf Syst 24(4):364–373
Bhattacherjee A, Perols J, Sanford C (2008) Information technology continuance: a theoretic extension and empirical test. J Comput Inf Syst 49(1):17–26
Bhattacherjee A, Premkumar G (2004) Understanding changes in belief and attitude toward information technology usage: a theoretical model and longitudinal test. MIS Q 66:229–254
Maruping LM, Bala H, Venkatesh V, Brown SA (2017) Going beyond intention: Integrating behavioral expectation into the unified theory of acceptance and use of technology. J Am Soc Inf Sci 68(3):623–637
Palm JM, Dart T, Dupuis I, Leneveut L, Degoulet P (2010) Clinical information system post-adoption evaluation at the georges pompidou university hospital. In: AMIA annual symposium proceedings, vol 2010, p 582. American Medical Informatics Association
Hadji B, Degoulet P (2016) Information system end-user satisfaction and continuance intention: a unified modeling approach. J Biomed Inform 61:185–193
Mellikeche S, Boussekey O, Martin G, Campoy E, Lajonchère JP, Degoulet P (2018) Evaluation of the unified model of information systems continuance (UMISC) in two hospital environments. Int J Med Inform 117:66–81
Mellikeche S, de Fatima Marin H, Benítez SE, de Lira ACO, de Quirós FGB, Degoulet P (2020) External validation of the unified model of information systems continuance (UMISC): an international comparison. Int J Med Inform 134:103927
Lim WM (2018) Dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. Australas J Inf Syst 22:66
Lim WM (2018) Revisiting concepts and theories in information systems and technology. Australas J Inf Syst 22:66
Kim ED, Kuan KK, Vaghasiya MR, Penm J, Gunja N, El Amrani R, Poon SK (2022) Passive resistance to health information technology implementation: the case of electronic medication management system. Behav Inf Technol. https://doi.org/10.1080/0144929X.2022.2117081
Lim WM, Lim AL, Phang CSC (2019) Toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. Australas J Inf Syst 23:66
Samhan B (2018) Revisiting technology resistance: current insights and future directions. Australas J Inf Syst 22:66
Tarhini A, El-Masri M, Ali M, Serrano A (2016) Extending the UTAUT model to understand the customers’ acceptance and use of internet banking in Lebanon: a structural equation modeling approach. Inf Technol People 29(4):830–849
Venkatesh V, Thong JY, Xu X (2012) Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q 36(1):157–178
Goodhue DL (1995) Understanding user evaluations of information systems. Manag Sci 41(12):1827–1844
Yi Y (1990) A critical review of consumer satisfaction. Rev Market 4(1):68–123
Aqleh MM, Al-Hashimi M, Hamdan A (2021) Medical mobile applications (e-Health) and the acceptance of the electronic medical record systems (EMRS) among physicians in Bahrain. In the importance of new technologies and entrepreneurship in business development: in the context of economic diversity in developing countries: the impact of new technologies and entrepreneurship on business development. Springer, pp 61–74
Barzegari S, Ghazisaeedi M, Askarian F, Jesmi AA, Gandomani HS, Hasani A (2020) Hospital information system acceptance among the educational hospitals. J Nurs Midwif Sci 7(3):186
Barzekar H, Ebrahimzadeh F, Luo J, Karami M, Robati Z, Goodarzi P (2019) Adoption of hospital information system among nurses: a technology acceptance model approach. Acta Inform Med 27(5):305
Johnson MP, Zheng K, Padman R (2014) Modeling the longitudinality of user acceptance of technology with an evidence-adaptive clinical decision support system. Decis Support Syst 57:444–453
English D, Ankem K, English K (2017) Acceptance of clinical decision support surveillance technology in the clinical pharmacy. Inform Health Soc Care 42(2):135–152
Vroom VH (1964) Work and motivation. Wiley, New York
Abdekhoda M, Salih KM (2017) Determinant factors in applying picture archiving and communication systems (PACS) in healthcare. Perspect Health Inf Manag 14:66
Adenuga KI, Iahad NA, Miskon S (2017) Towards reinforcing telemedicine adoption amongst clinicians in Nigeria. Int J Med Inform 104:84–96
Chi WC, Lin PJ, Chang IC, Chen SL (2020) The inhibiting effects of resistance to change of disability determination system: a status quo bias perspective. BMC Med Inform Decis Mak 20(1):1–8
Chow SK, Chin WY, Lee HY, Leung HC, Tang FH (2012) Nurses’ perceptions and attitudes towards computerisation in a private hospital. J Clin Nurs 21(11–12):1685–1696
Hung SY, Nakayama M, Chen CC, Tsai FL (2019) Physician perceptions of electronic medical records: the impact of system service quality, and generation/experience gaps. Int J Healthc Technol Manag 17(4):229–254
Alsyouf A, Ishak AK (2018) Understanding EHRs continuance intention to use from the perspectives of UTAUT: practice environment moderating effect and top management support as predictor variables. Int J Electron Healthc 10(1–2):24–59
Akinnuwesi BA, Uzoka FME, Okwundu OS, Fashoto G (2016) Exploring biometric technology adoption in a developing country context using the modified UTAUT. Int J Bus Inf Syst 23(4):482–521
Hoque R, Sorwar G (2017) Understanding factors influencing the adoption of mHealth by the elderly: an extension of the UTAUT model. Int J Med Inform 101:75–84
Hadji B, Dupuis I, Leneveut L, Heudes D, Wagner JF, Degoulet P (2014) Determinants of continuance intention in a post-adoption satisfaction evaluation of a clinical information system. In: e-Health–For Continuity of Care. IOS Press, pp 990–994
Burnkrant RE, Cousineau A (1975) Informational and normative social influence in buyer behavior. J Consumer Res 2(3):206–215
Chau PY, Hu PJ (2002) Examining a model of information technology acceptance by individual professionals: an exploratory study. J Manag Inf Syst 18(4):191–229
Engin M, Gürses F (2019) Adoption of hospital information systems in public hospitals in Turkey: an analysis with the unified theory of acceptance and use of technology model. Int J Innov Technol Manag 16(06):1950043
Huang WM, Chen T, Hsieh CW (2014) An empirical study on the physicians’behavioral intention with electronic medical record systems in Taiwan
Kumar KA, Natarajan S (2020) An extension of the Expectation Confirmation Model (ECM) to study continuance behavior in using e-health services. Innov Mark 16(2):15–28. https://doi.org/10.21511/im.16(2).2020.02
Joshi K (1991) A model of users’ perspective on change: the case of information systems technology implementation. MIS Q 15(2):66
Gelderman M (1998) The relation between user satisfaction, usage of information systems, and performance. Inf Manag 34(1):11–18
Brynjolfsson E (1993) The productivity paradox of information technology. Commun ACM 36(12):66–77
Beggan JK (1992) On the social nature of nonsocial perception: the mere ownership effect. J Pers Soc Psychol 62(2):229
Lim WM (2021) Conditional recipes for predicting impacts and prescribing solutions for externalities: the case of COVID-19 and tourism. Tour Recreat Res 46(2):314–318
Gunja N, Dunlop I, Vaghasiya M, Kuan K, Poon S (2018) Patient-centric implementation of an electronic medication management system at a tertiary hospital in Western Sydney. J Innov Health Inform 25(3):169–175
National Health and Medical Research Council (2007) National Statement on Ethical Conduct in Human Research
Bornstein MH, Jager J, Putnick DL (2013) Sampling in developmental science: situations, shortcomings, solutions, and standards. Dev Rev 33(4):357–370
Bhattacherjee A (2012) Social science research: principles, methods, and practices. textbooks collection. 3. http://scholarcommons.usf.edu/oa_textbooks/3
Sirdeshmukh D, Singh J, Sabol B (2002) Consumer trust, value, and loyalty in relational exchanges. J Mark 66(1):15–37
Kim HW, Kankanhalli A (2009) Investigating user resistance to information systems implementation: a status quo bias perspective. MIS Q 66:567–582
Moore GC, Benbasat I (1991) Development of an instrument to measure the perceptions of adopting an information technology innovation. Inf Syst Res 2(3):192–222
Ringle CM, Wende S, Will A (2005) SmartPLS 2.0.M3. Hamburg: SmartPLS. Retrieved from http://www.smartpls.com
Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 88(5):879
Kock N (2015) Common method bias in PLS-SEM: a full collinearity assessment approach. Int J e-Collab 11(4):1–10
Kock N, Lynn G (2012) Lateral collinearity and misleading results in variance-based SEM: an illustration and recommendations. J Assoc Inf Syst 13(7):66
Williams LJ, Hartman N, Cavazotte F (2010) Method variance and marker variables: a review and comprehensive CFA marker technique. Organ Res Methods 13(3):477–514
Samuelson W, Zeckhauser R (1988) Status quo bias in decision making. J Risk Uncertain 1(1):7–59
Wright TP (1936) Factors affecting the cost of airplanes. J Aeronaut Sci 3(4):122–128
Vits J, Gelders L (2002) Performance improvement theory. Int J Prod Econ 77(3):285–298
Boiney LG (1998) Reaping the benefits of information technology in organizations: a framework guiding appropriation of group support systems. J Appl Behav Sci 34(3):327–346
Gopal A, Prasad P (2000) Understanding GDSS in symbolic context: shifting the focus from technology to interaction. MIS Q 66:509–546
Orlikowski WJ, Iacono CS (2001) Research commentary: desperately seeking the “IT” in IT research—a call to theorizing the IT artifact. Inf Syst Res 12(2):121–134
Funding
Open Access funding enabled and organized by CAUL and its Member Institutions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kim, E.D., Kuan, K.K.Y., Vaghasiya, M.R. et al. A repeated cross-sectional study on the implementation of electronic medication management system. Inf Technol Manag 25, 33–50 (2024). https://doi.org/10.1007/s10799-023-00398-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10799-023-00398-8