10.1016/j.nedt.2015.07.022: Nurse Education Today
10.1016/j.nedt.2015.07.022: Nurse Education Today
10.1016/j.nedt.2015.07.022: Nurse Education Today
Effectiveness of virtual classroom training in improving the knowledge and
key maternal neonatal health skills of general midwifery students in Bihar,
India: A pre – and post- intervention study
PII: S0260-6917(15)00292-0
DOI: doi: 10.1016/j.nedt.2015.07.022
Reference: YNEDT 3022
Please cite this article as: Agrawal, Neeraj, Kumar, Somesh, Balasubramaniam, Sud-
harsanam Manni, Bhargava, Saurabh, Sinha, Pallavi, Bakshi, Bhawna, Sood, Bulbul,
Effectiveness of virtual classroom training in improving the knowledge and key maternal
neonatal health skills of general midwifery students in Bihar, India: A pre – and post-
intervention study, Nurse Education Today (2015), doi: 10.1016/j.nedt.2015.07.022
This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.
ACCEPTED MANUSCRIPT
TITLE PAGE
Study Title:
T
INDIA: A PRE – AND POST- INTERVENTION STUDY
IP
Authors’ Name:
R
Neeraj Agrawal
SC
Somesh Kumar
Sudharsanam Manni Balasubramaniam
Saurabh Bhargava
NU
Pallavi Sinha
Bhawna Bakshi
Bulbul Sood
MA
Affiliation:
Jhpiego – an affiliate of John Hopkins University, 221, Okhla Phase 3, New Delhi, India
D
110020
TE
Corresponding Author:
Name: Sudharsanam Manni Balasubramaniam
P
1
ACCEPTED MANUSCRIPT
Ayyasami from Cisco technologies, India for their commitment and hard work without which
the establishment of virtual classrooms was not possible.
Financial Support: This work was supported by grants from Department for International
Development (DFID). DFID had no role in study designing, data collection/analysis and
T
IP
decision to publish, or preparation of the manuscript.
R
SC
NU
MA
D
P TE
CE
AC
2
ACCEPTED MANUSCRIPT
Abstract
BACKGROUND: In 2008-09, the National Health Systems Resource Center of India reported
overall quality of nurse-midwifery education in Bihar as grossly sub-optimal. To address this,
we implemented a competency based training using virtual classrooms in two general nurse
T
midwives (GNM) schools of Bihar. The students from remotely located nursing institutions
IP
were now able to see live demonstrations of maternal and newborn health (MNH) practices
R
performed by a trained faculty on simulation models at instructor location.
SC
OBJECTIVE: To evaluate the effectiveness of virtual classroom training in improving the
MNH related skills of the nursing, midwifery students in Bihar, India.
NU
DESIGN: This study used a pre post intervention design without control group.
MA
SETTINGS: Students from two public GNM schools of Bihar.
PARTICIPANTS: Final year students from both the GNM schools who have completed their
D
coursework in MNH.
TE
METHOD: A total of 83 students from selected GNM schools were assessed for their
competencies in six key MNH practices using objective structured clinical examination
P
competencies.
RESULTS: The mean student score assessed before the intervention was 21.3 with (95% CI
19.9, 22.6) which increased to 62.0 (95% CI 60.3, 63.7) post intervention. This difference was
statistically significant. When adjusted for clustering using linear regression analysis, the
students in post intervention scored 52.3 (95% CI 49.4% – 55.3%) percentage points higher
than pre intervention and this was statistically significant.
3
ACCEPTED MANUSCRIPT
Key Words Virtual Classroom ● Nursing Students ● Clinical Skills ● Quality of Care ● Maternal
Neonatal Health
INTRODUCTION
Optimum midwifery services is among the most effective ways in providing high quality
T
IP
maternal, newborn care and thus in reducing maternal and infant mortality especially in
developing countries (Renfrew et al., 2014; ten Hoope-Bender et al., 2014). Some of the key
R
causes for maternal and neonatal deaths like infection, post-partum haemorrhage,
SC
eclampsia, birth asphyxia and pre-term birth can be averted through competent nursing and
midwifery care (SEARO, International day of the Midwife, 5 May 2012). With the launch of
NU
Global Strategies for Women’s and Children’s Health in 2010, many developing countries
started prioritizing their focus on improving access to quality midwifery services to women
MA
during intra-partum, post-partum period by scaling up the numbers of competent nurses
and midwifes as part of their national agenda (Campbell, J. et al., 2011).
In India, current shortage of nursing & midwifery workforce is a major human resource
D
constraint for providing quality maternal and child health services and thus in achieving
TE
personnel from 1992-93 to 2007-08 (WHO, Fulfilling the health agenda for women &
CE
children, 2014), still there is an acute shortage of more than 2 million nursing staff (Wanted:
2.4 million nurses, and that's just in India, 2010)
AC
In Bihar, the scenario is even more alarming as the shortfall in the number of staff nurses is
around 50% in the Medical College Hospitals and much higher in the public health facilities.
Similarly, the shortfall of the multipurpose health workers i.e. auxiliary nurse midwives
(ANMs), in the existing public health facilities of Bihar is approximately 28% (NHSRC, Nursing
services in Bihar, 2009). Apart from shortage in numbers, inadequate level of clinical
competency of these nurse -midwifes is another challenge in improving quality of MNH
services especially in remote areas. In 2008-09, the National Health Systems Resource
Center of India conducted a study on the nursing services in Bihar and reported overall
quality of nursing-midwifery education in existing institutions as grossly sub-optimal
(NHSRC, Nursing services in Bihar, 2009). The study also highlighted other key concerns like
inadequate faculty strength, infrastructure, suboptimal academic/clinical training and poor
4
ACCEPTED MANUSCRIPT
structural management in the existing nursing education system. A high variance in the
knowledge and clinical skills of the available faculty was also observed. The report also
flagged an urgent need to improve institutional teaching methods, student evaluation
system and inclusion of advance practical training methodologies for betterment of the
T
overall educational quality.
IP
During the past decades, with the advancement in modern information technology, use of
R
internet for knowledge sharing and teaching has increased tremendously. Several studies
SC
have demonstrated that virtual classrooms are an efficient and effective way to enhance
students learning (Bertsch et al., 2007; Hortos et al., 2013; Mosalanejad et al., 2014). A
NU
virtual classroom is a set up where qualified trained faculty located at central institution
teaches a class of students in remotely located schools through internet based information
MA
technology interphase. A study conducted to evaluate the effectiveness of virtual versus
traditional teaching in achieving competencies among nursing students in Iran. The study
reported that virtual teaching was more effective in achieving higher theoretical knowledge
D
but no significant difference was found between the virtual and traditional teaching groups
TE
The usefulness of lectures attended via Interactive Video Conferencing versus in-person was
studied among 52 medical students attending a University of Vermont medicine clerkship
AC
program in the United States. Students were asked to attend half lectures in person and rest
half via videoconferencing. The mean student scores attending lectures via both methods
were found almost similar, suggest that technology mediated learning can be used for
effective clinical learning at off-site locations (Bertsch et al., 2007). Another study
conducted to understand the level of satisfaction, also generate evidences for its
acceptability among the users (Callas et al., 2004).
5
ACCEPTED MANUSCRIPT
objective of this intervention was to improve the knowledge and skills of nursing, midwifery
students.
The virtual class room training program: CISCO is an American multinational corporation
having expertise in designing and manufacturing high quality networking equipment. Using
T
IP
Cisco’s Education Enabled Development (CEED) technology, virtual classrooms were
established in 4 ANM and 3 GNM institutions of Bihar, India. Another virtual classroom was
R
established at CoN, IGIMS Patna as an instructor location. CEED is an energy efficient cloud-
SC
based video interaction solution, developed by Cisco technologies, with a vision to provide
quality education to rural areas at very affordable price of less than 1 US dollar per child per
NU
month (The Hindu, 2013). This method allows teachers and students from remote areas to
connect with experts from urban settings (instructor location) through live streaming of
MA
training sessions for efficient delivery of education and skills development courses. Each
virtual classroom was having some basic technology components needed for successful
operation like CEED box, Central Processor Unit integrated with a liquid crystal display (LCD)
D
projector. All peripheral components like internet line, public address (PA) system, universal
TE
organization affiliated to John Hopkins University, based on national curricula for second-
year ANM and third-year GNM students, incorporating evidence-based techniques such as
AC
6
ACCEPTED MANUSCRIPT
This study was designed to evaluate the effectiveness of this virtual class room training in
improving the MNH knowledge and clinical skills of nursing-midwifery students using OSCE
as the evaluation tool. Although some earlier studies demonstrate the benefits of virtual
training, to the best of our knowledge there are no studies conducted so far to assess the
T
effectiveness of virtual training in improving knowledge and clinical skills of nursing-
IP
midwifery students in resource constrained settings like India. It was anticipated that the
R
findings from the study will add to the existing knowledge base and help find solutions to
SC
address the current shortfall in nursing faculty and finding ways to improve institutional
capacity and quality nursing education in resource limited settings.
NU
METHODS
Study design: This was a pre-post intervention study design to evaluate the effectiveness of
MA
virtual classroom training in improving knowledge and clinical skills related to MNH among
GNM students. Pre and post intervention data were collected from the students from 2
D
participating GNM schools of Bihar. Complete study data were gathered in two phases
TE
approximately 12 months apart. Pre and post intervention data were collected in the month
of March 2013 and Feb 2014 respectively.
P
Study site and population: All available students from selected 2 GNM schools i.e. Shri
CE
Krishna Medical College and Hospital (SKMCH), Muzaffarpur and Patna Medical College and
Hospital (PMCH), Patna of Bihar who were in their final year of enrollment and had
AC
completed their course work in maternal and neonatal health were invited to participate in
the study. To start with, students from both the GNM schools were assessed for their MNH
knowledge and clinical skills. A 72 hour standardized training package was then
administered among the final year students from the next batch along with their attendance
of regular teaching sessions. Post intervention data was collected from the trained cohort
who had attended the virtual classroom training.
The pre and post cohorts included in the study were different as the intervention was
targeted at the final year students. The 72-hour package was administered to a new batch
of final year GNM students as the cohort assessed at pre-intervention had already passed
out by the time the intervention was implemented.
7
ACCEPTED MANUSCRIPT
Sample size: Sample size calculation was not conducted as all eligible students were invited
to be a part of this assessment. In 2013, prior to implementation of the intervention, 83
GNM students out of total i.e. 94 from 2 GNM schools were assessed for their competencies
in key MNH skills using OSCE method. Rests of the student were either denied or absent
T
during the day of assessment. Conversely, all 92 students who attended virtual training
IP
participated in the post intervention assessment for skill assessment. Competency of a
R
student was defined as score of 75% or above in the OSCE (averaged across 6 stations).
SC
Data collection tools and process: A standardized pre-tested OSCE checklist was used for
clinical skill assessment of the students. The questionnaires of the checklist were broadly
NU
divided into six sections to cover some of the key MNH practices taught as a part of the
intervention. Each section of the OCSE checklist was designed to test clinical decision
MA
making skills of the students against the pre-determined standards. Two of the sections i.e.
use of Partograph and infection prevention were non observed practices and signified
‘knowledge’ whereas four sections i.e. management of second stage of labor, active
D
management of third stage of labor, essential newborn care and newborn resuscitation were the
TE
observed practices and signified ‘clinical skills’. For observed practices, student clinical
knowledge and skills were evaluated with the use of simulation models and mannequins
P
against the set OSCE standards. For plotting of partograph and infection prevention,
CE
checklist was established and scoring was done against each critical step which is essential
to be performed as subsets to complete a given practice. The participants were evaluated in
batches and it was ensured that the groups of students waiting to be evaluated did not get
exposed to the OSCE stations before the evaluation. On an average, 10 minutes time was
allotted to perform a skill at each station. The total duration to complete OSCE assessment
for 6 participants was approximately 60 minutes and maximum possible OSCE score a
participant can attain was 76. All Participants were evaluated using same standardized
checklist by a group of standardized observers of Jhpiego, India trained by the study
investigator team prior to start data collection.
Statistical Analysis: Data were analyzed using software package “STATA” version 13
(StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.)
8
ACCEPTED MANUSCRIPT
The mean scores of participants at pre and post intervention assessments were compared
and tested for statistical significance using paired t-test. Differences in the two groups for
their competencies were also calculated using Pearson chi square test. Maximum allowable
alpha error considered was 5%. Linear regression modelling with robust standard error
T
estimation was also performed to adjust the clustering effect by school.
IP
Ethical consideration: This study was approved by the institutional review board of Johns
R
Hopkins University. Before initiating the study, each member of the research team of
SC
Jhpiego, India involved in data collection received a research ethics training to improve their
understanding of study objectives, how to obtain informed consent and maintain
NU
participants’ privacy and confidentiality throughout the data collection using John Hopkins
School of Public Health (JHSPH) Institutional Review Board (IRB) Human Subject Research
MA
Ethics Field Training Guide (Field Training Guides for Data Collectors, 2010). A verbal
informed consent was obtained from all the participants prior to start their OSCE
assessment.
D
TE
RESULTS
assessment. 83 out of 94 third-year GNM students were assessed for their competencies in
CE
key MNH skills using a structured OSCE checklist prior to implementation of the
intervention. Subsequently, a total of 92 students from the next batch of the same
AC
institutions was then underwent a 72 hour virtual training and then were assessed as post
intervention group for the same skills. While applying the test of normality (Shapiro Wilk
test) on the OSCE scores each cohort, it was found that they followed a normal distribution
(p>0.05). The mean student score assessed prior to intervention was 21.3 out of 76 with
(95% CI 19.9, 22.6) which increased to 62.0 (95% CI 60.3, 63.7) after the intervention.
Statistically significant difference in the mean scores (40.6) of students is seen between pre
and post intervention assessment (p < 0.001). The mean scores in each station at pre and
post intervention is elaborated in table 3. Mean difference in each station was also
statistically significant. The pre – post intervention changes in the mean scores related to
individual practices are given in Table 3.
9
ACCEPTED MANUSCRIPT
Reasonable level of skills associated with competency of a student in key MNH practices was
defined as a score 75% or above in the OSCE, across the six stations. Surprisingly, none of
the students assessed prior to the intervention were found to have the reasonable level of
skills associated with competency. However, at post intervention, a substantial difference
T
has been observed in the competency where 77.2% of the participants were found
IP
competent (Table 4) and this difference was statistically significant. Competency as an
R
outcome was difficult to model as no one was having this as an outcome at pre intervention
SC
stage. Therefore, linear regression modelling was performed adjusting for the clustering
effect with robust standard error estimation. On an average, a student at post intervention
scored 52.3% (95% CI 49.4% – 55.3%) marks more than the student at pre intervention. The
NU
average difference in score between the schools was 4.9% (95% CI 1.9% - 8.0%) adjusting for
pre-intervention and post intervention (Table 5).
MA
DISCUSSION
D
The virtual classroom program was designed with an objective to provide quality MNH
TE
like virtual classrooms training could be an efficient and effective strategy to address local
CE
This study aimed to evaluate whether virtual classroom trainings can improve key MNH
AC
related clinical skills of the nursing-midwifery students using a pre – post study design. The
findings of the study reveal a statistically significant change in the mean OSCE score of the
students. Also much higher number of students were found competent in the key MNH
(scored 75% or more in OSCE) skills post intervention compared to none during pre-
intervention assessment. Even when adjusted for clustering using linear regression analysis,
the students during post intervention scored 52.3% more marks than pre intervention. To
the best of our knowledge, no studies in the literature of midwifery have shown effect of
virtual classrooms in improving the clinical skills of the nursing students in resource
constrained settings.
Though the utilization of similar classroom setting have been studied, no statistically
significant differences in the mean scores were found between the students who attended
10
ACCEPTED MANUSCRIPT
lectures via Interactive Video Conferencing from the remote locations versus In-Person
while attending medicine clerkship program (Bertsch et al., 2007). Another study while
evaluating the effectiveness of live lectures delivered via videoconferencing with that of in-
person lectures also reported no statistically significant difference in the performance of
T
medical students attended national licensing examination (Hortos et al., 2013). In our study,
IP
although the context and location of study was different, statistically significant differences
R
in the mean scores have been observed in the group of students assessed before and after
SC
the implementation of virtual classroom training program.
NU
significant increase in the mean scores in the theoretical exams but not in practical
competencies for the students attended virtual sessions versus the traditional learning
MA
group were observed (Mosalanejad et al., 2014). A study conducted in India to evaluate the
effectiveness of video assisted teaching module also concluded that it was highly effective in
improving the knowledge of nursing students regarding care of dementia patients (Austin &
D
Balasubramanian, 2012). On the other hand, our study on virtual classroom effectiveness
TE
revealed a statistically significant improvement in the practical skills of the students related
to MNH. The reason for such a drastic improvement might be because the intervention was
P
focused and designed in such a way where students were encouraged, trained for improving
CE
Our study was a pre-post intervention study without a control group as control group
was not feasible due to logistic reasons. Although sample size was not calculated and all
willing students were invited to participate, the post hoc power calculation showed a
power of more than 90%, hence makes the study valid. Additionally, the student cohorts
assessed before and after the intervention were different as the intervention was
targeted at the final year students who have completed coursework on maternal and
new born health, which was one of our inclusion criteria for recruiting the participants. It
was unrealistic to assess the same cohort of students before and after implementation of
11
ACCEPTED MANUSCRIPT
the program. The students whom we assessed at pre-intervention had already completed
their course work on MNH skills through traditional learning method before the
implementation of virtual classroom training. The post intervention assessment was
conducted among the another cohort of students who have attended 72 hour virtual
T
classroom training along with attendance of regular teaching sessions similar to earlier
IP
cohort. Addressing the time lag between two assessment (pre-post intervention), an
R
assumption was made that there would be no significant difference between
SC
characteristics of students passing out subsequently from the institutions.
NU
Our study indicates that virtual classroom training is effective in enhancing knowledge and
clinical skills of nursing-midwifery students in resource constrained settings, especially
where trained faculty are not available in adequate numbers. Considering the existing
MA
shortfall in faculty in nursing educational institutions as well as the sub-optimal educational
quality, this technology based approach could be utilized in consort with traditional learning
D
for student skills enhancement. In the long term, this can help us to improve the quality of
TE
graduating nurse-midwives, thereby improving the quality of service provision at the sites
where these GNM graduates are posted in future. Although the tracking of these capacity-
P
built graduate nurse-midwives till their employment will be helpful to understand the long
CE
term impact in reducing maternal and neonatal morbidity and mortality in the study state of
India, this was beyond the scope of this virtual classroom training program. Though the
AC
outcome of this program is promising, a long term follow-up of nursing graduates who were
trained through virtual classroom training can identify the real impact of the program in
improving the standards of care being delivered by them at the time of their deployment to
health facilities.
CONCLUSION
The findings of our study suggest that the knowledge and skills of nursing-midwifery
students can be enhanced by leveraging virtual training concept blended with traditional
classroom learning. Addressing the current issues of sub optimal nursing education,
infrastructure and shortage of faculty this approach can be utilized to deliver high quality
MNH training to nursing-midwifery students based at remotely located institutions.
12
ACCEPTED MANUSCRIPT
T
IP
R
SC
NU
MA
D
P TE
CE
AC
13
ACCEPTED MANUSCRIPT
References:
Bertsch, T.F., Callas, P.W., Rubin, A., Caputo, M.P., & Ricci, M.A. (2007). APPLIED RESEARCH:
Effectiveness of Lectures Attended via Interactive Video Conferencing Versus In-Person in
Preparing Third-Year Internal Medicine Clerkship Students for Clinical Practice Examinations
(CPX). Teaching and Learning in Medicine: An International Journal, 19, 4–8.
T
doi:10.1080/10401330709336616
IP
Callas, P.W., Bertsch, T.F., Caputo, M.P., Flynn, B.S., Doheny-Farina, S., & Ricci, M.A. (2004). Medical
Student Evaluations of Lectures Attended in Person or From Rural Sites via Interactive
R
Videoconferencing. Teaching and Learning in Medicine: An International Journal. 16, 46–50.
doi:10.1207/s15328015tlm1601_10
SC
Cisco’s ‘Dwara’ to connect educators. (2013, March 10). Tech News, The Hindu. Retrieved from
http://www.thehindu.com/sci-tech/technology/tech-news/article4492072.ece
Hazarika, I. (2013). Health workforce in India: assessment of availability, production and distribution.
NU
WHO South-East Asia Journal of Public Health, 2(2), 106-112. doi:10.4103/2224-3151.122944
Hortos, K., Sefcik, D., Wilson, S.G., McDaniel, J.T., & Zemper, E. (2013). Synchronous
Videoconferencing: Impact on Achievement of Medical Students. Teaching and Learning in
MA
Medicine, 25(3), 211–215. doi:10.1080/10401334.2013.797344
JHSPH (2009). JHSPH human subjects research ethics field training guide. Retrieved from Johns
Hopkins Bloomberg School of Public Health website: http://www.jhsph.edu/offices-and-
D
services/institutional-review-board/training/field-training-guides-for-data-collectors/index.html
Austin, M., & Balasubramanian, N. (2012). A study to evaluate the Video Assisted Teaching Module
TE
(VATM) on care of dementia patients developed for B. Sc Nursing Students in a selected college
of nursing, Mangalore. Asian J. Nursing Edu. and Research, 2(3), 113–117.
Mosalanejad, L., Shahsavari, S., Sobhanian, S., & Dastpak, M. (2014). The effect of virtual versus
P
14
ACCEPTED MANUSCRIPT
WHO. (2014). International day of the Midwife, 5 May 2012. SEARO. Retrieved from World Health
Organization website:
http://www.searo.who.int/entity/nursing_midwifery/events/international_midwife_day/en/
WHO. (2014). Wanted: 2.4 million nurses, and that’s just in India. Retrieved from World Health
Organization website: http://www.who.int/bulletin/volumes/88/5/10-020510/en/
T
R IP
SC
NU
MA
D
P TE
CE
AC
15
ACCEPTED MANUSCRIPT
T
4. New-born care 8
IP
5. Family Planning 4
6. Others 10
R
Total 72
SC
Table 2: Background information of the schools participated in the assessment
GNM institutions
Number of schools assessed NU Pre Intervention
2
Post Intervention
2
MA
Number of students available 94 92
Number of students attended OSCE 83 92
D
TE
Table 3: Comparison of pre and post intervention scores among the students who attended OSCE
16
ACCEPTED MANUSCRIPT
T
Post-Intervention 71 (77.2) 21 (22.8) 92
Number (%)
R IP
Table 5: Linear regression of the mean scores adjusted for clustering by schools
SC
Variable Adjusted Mean P value (t-test) Coefficient of
difference (95% CI) determination
Overall P value (F test)
NU
Pre - Post Intervention 52.34 (49.4-55.3) <0.001 0.9
<0.001
Difference between 4.9 (1.9-8.0) <0.001
PMCH and SKMCH
MA
D
P TE
CE
AC
*
Competency of a student was defined as score of 75% or above in the OSCE (averaged across 6 skill stations).
17
ACCEPTED MANUSCRIPT
Highlights
T
R IP
SC
NU
MA
D
P TE
CE
AC
18