npj | digital medicine Article

Published in partnership with Seoul National University Bundang Hospital

https://doi.org/10.1038/s41746-024-01252-z

A systematic review and meta analysis on

digital mental health interventions in
inpatient settings
Check for updates
1,2 1,2 1,2 1,2
Alexander Diel , Isabel Carolin Schröter , Anna-Lena Frewer , Christoph Jansen ,
Anita Robitzsch1,2, Gertraud Gradl-Dietsch3, Martin Teufel1,2 & Alexander Bäuerle1,2

E-mental health (EMH) interventions gain increasing importance in the treatment of mental health
disorders. Their outpatient efficacy is well-established. However, research on EMH in inpatient
1234567890():,;
1234567890():,;

settings remains sparse and lacks a meta-analytic synthesis. This paper presents a meta-analysis on
the efficacy of EMH in inpatient settings. Searching multiple databases (PubMed, ScienceGov,
PsycInfo, CENTRAL, references), 26 randomized controlled trial (RCT) EMH inpatient studies
(n = 6112) with low or medium assessed risk of bias were included. A small significant total effect of
EMH treatment was found (g = 0.3). The effect was significant both for blended interventions (g = 0.42)
and post-treatment EMH-based aftercare (g = 0.29). EMH treatment yielded significant effects across
different patient groups and types of therapy, and the effects remained stable post-treatment. The
results show the efficacy of EMH treatment in inpatient settings. The meta-analysis is limited by the
small number of included studies.

Mental health disorders represent a prevalent set of clinical conditions monitoring. Here, the term EMH is used to describe any digitally delivered
associated with substantial personal and economic burdens. However, interventions with the goal of improving mental health outcomes. Due to
despite their prevalence and impact, there exists a conspicuous deficit in the the easy accessibility of EMH products, such interventions have many
provision of effective treatment1–4. Across Europe, estimates suggest that advantages: They can 1) fill structural supply gaps for rural areas, 2) bridge
15–40% of the population experiences some form of mental disorder, yet long waiting times for in-person mental health treatment, and 3) provide
fewer than one-third of these cases receive treatment that meets the estab- additional anonymity for those concerned about stigmatization18,19. Thus,
lished standards of adequacy5–9. EMH tools have the potential to be a viable method to overcome the various
One reason for the lack of adequate treatment of mental disorders are issues hindering adequate mental health treatment.
structural supply issues, for example caused by a shortage of mental In outpatient settings, EMH interventions are effective tools to treat
healthcare providers in more rural areas10. Furthermore, negative attitudes mental disorders according to several meta-analyses, including the treat-
towards mental health treatments hinder seeking help especially in mild to ment of anxiety and depression20–22 eating disorders23, posttraumatic stress24,
moderate cases11. Finally, prompt access to mental health treatment is or work-related stress25. Furthermore, EMH interventions find pre-
paramount for its efficacy, yet mental health facilities and specialists often dominantly positive acceptance from both patients and mental health
impose prolonged waiting periods spanning several months12. These practitioners26–28. Thus, EMH interventions are viable and accepted tools in
extended waiting intervals amplify the economic strain of mental the treatment of mental disorders in outpatient settings.
disorders13, exacerbate clinical manifestations14,15, diminish treatment Inpatient treatment signals an especially high need for timely and ade-
adherence, and elevate dropout rates16,17. In summary, providing adequate quate intervention and is indicated for cases considered too severe for out-
mental health treatment is complicated by a variety of structural issues patient treatment29. Inpatient interventions can profit from supportive EMH
leading to several other problems like economic and patients’ personal costs. procedures either to bridge waiting times, to blend with in-person inter-
E-mental health (EMH) interventions aim to provide adequate treat- ventions, or to ensure stabilization and relapse prevention in aftercare
ment of mental health disorders through technological means and channels, treatments. Especially the implementation of post-treatment aftercare
such as app- or web-based systems, text messages, videos, or digital improves the chances of a favorable and sustained development30–32. Thus,

1
Clinic for Psychosomatic Medicine and Psychotherapy, LVR-University Hospital Essen, University of Duisburg-Essen, Essen, Germany. 2Center for Translational
Neuro- and Behavioral Sciences, University of Duisburg-Essen, Essen, Germany. 3Department of Child and Adolescent Psychiatry, Psychosomatics and Psy-
chotherapy, LVR-University Hospital Essen, University of Duisburg-Essen, Essen, Germany. e-mail: alexander.diel@lvr.de

npj Digital Medicine | (2024)7:253 1

https://doi.org/10.1038/s41746-024-01252-z Article

Table 1 | Overview of studies

Study (authors, year, country) Treatment type Type of therapy Patient group Risk of Bias Assessment
Holländare et al. (2011), Sweden65 Aftercare CBT Mood disorder Some concerns
Bauer et al. (2012), Germany53 Aftercare CBT Eating disorder Low risk
Fichter et al. (2012), Germany57 Aftercare CBT Eating disorder Some concerns
54
Bauer et al. (2013), Germany Aftercare CBT Eating disorder Some concerns
Bischoff et al. (2013), Germany72 Aftercare NA Transdiagnostic Low risk
Ebert et al. (2013a), Germany58 Aftercare CBT Transdiagnostic Some concerns
Ebert et al. (2013b), Germany73 Aftercare CBT Transdiagnostic High risk
55
Fichter et al. (2014), Germany Aftercare CBT Eating disorder Some concerns
Gulec et al. (2014), Hungary46 Aftercare NA Eating disorder Some concerns
Schmädeke et al. (2015), Germany74 Aftercare CBT Mood disorder High risk
Harrington et al. (2016), USA59 Aftercare NA Substance abuse Some concerns
Kordy et al. (2016), Germany40 Aftercare NA Mood disorder Low risk
Willems et al. (2016), Netherlands75, Intervention CBT Somatic comorbidity High risk
Välimäki et al. (2017), Finland50 Aftercare NA Psychotic disorder Low risk
Jacobi et al. (2017), Germany41 Aftercare NA Eating disorder Some concerns
37
Zwerenz et al. (2017a), Germany Aftercare PD Transdiagnostic Some concerns
Zwerenz et al. (2017b), Germany76 Aftercare PD Return to work Some concerns
Zwerenz et al. (2017c), Germany66 Intervention PD Mood disorder Some concerns
Norlund et al. (2018), Sweden64 Intervention CBT Somatic comorbidity Low risk
77
Schlicker et al. (2018), Germany Aftercare CBT Mood disorder High risk
Neumayr et al. (2019), Germany56 Aftercare NA Eating disorder Some concerns
Zwerenz et al. (2019), Germany78 Aftercare PD Mood disorder Some concerns
Alvarez et al. (2021), Australia51 Aftercare NA Psychotic disorder Some concerns
Gallinat et al. (2021), Germany42 Aftercare CBT Psychotic disorder Some concerns
Nolte et al. (2021), Germany43 Intervention CBT Mood disorder Some concerns
Shaygan et al. (2021), Iran79 Intervention NA Somatic comorbidity Low risk
Becker et al. (2022), Germany45 Aftercare PD Return to work Some concerns
Levis et al. (2022), USA80 Intervention Whole Health Transdiagnostic Low risk
Sharma et al. (2022), Canada36 Intervention CBT Anxiety symptoms Some concerns
Bruhns et al. (2023), Germany47 Aftercare CBT Mood disorder Some concerns
Studies included in risk of bias assessment (k = 30), categorized by treatment type, type of therapy, patient group, and assessed risk. Studies with a high risk were excluded for the meta analysis.

EMH treatment can be an important factor in the long-term success of Results

inpatient treatments. Adequate aftercare enhances rehabilitation according to Selected literature
several reviews and meta-analyses33–35. Randomized controlled trials (RCT) A total of 26 research studies containing 123 effects and a sample size of
exist on EMH treatment as add-ons to regular inpatient interventions36 and n = 6112 (intervention group = 3041, control group = 3071) were included.
aftercare37. Furthermore, a systematic review38 found support for the efficacy A summary of the included studies is shown in Table 1. Five studies used
of EMH aftercare treatments but was limited by the small number of studies. blended treatment during inpatient stay while 21 studies conducted post-
Yet as of now, there are to our knowledge no meta-analyses on the use of inpatient aftercare treatment. The most common patient groups
EMH in inpatient treatments, nor are there meta-analyses on EMH for (according to the number of studies) were eating disorders (k = 7) fol-
inpatient aftercare. In addition, the systematic review on EMH inpatient care lowed by mood disorders (k = 6), transdiagnostic (k = 4), psychotic dis-
is several years old and does not incorporate the more recent research38. orders (k = 3), return to work treatments (k = 2), mental comorbidities
The present study seeks to summarize the findings of previous RCTs with somatic disorders (k = 2), anxiety disorders (k = 1), and substance
on EMH treatments in inpatient settings in a meta-analysis. In addition, the abuse (k = 1).
risk of bias of the studies are assessed39. Specifically, this meta-analysis seeks Thirteen out of 26 studies utilized a passive control group for which
to investigate 1) the total effect of EMH treatments on mental health out- participants did not receive any type of active treatment (e.g., waiting list);
comes in inpatient settings; 2) the effects of EMH treatments, divided into eight studies used an active control group with an active treatment alter-
interventions and aftercare treatments; 3) the effect of EMH treatments in native to the EMH treatment (e.g., aftercare e-mail reminders for mental
relationship depending on the mental health disorder; 4) the effect of type of health tools, psychoeducation, rehabilitation activities, or psychosocial
therapy on EMH efficacy; 5) the effects of follow-up measures on EMH support such as counsling); five studies used an active control group to
interventions are investigated to test long-term effects; and 6) the assessment which the EMH treatment as added to in the intervention group; finally, one
of bias of the currently published RCT literature. Furthermore, post-hoc study used both active and passive control groups.
analyses were conducted to investigate 1) the role of EMH medium (e.g., Three studies used SMS-based EMH interventions. Eighteen studies
app-based, web-based, SMS-based) on EMH treatment efficacy, and 2) the used web-based interventions such as SUMMIT40, IN@41, HEINS42,
effect of the type of control group on EMH efficacy. Deprexis37,43,44, GSA Online45, and EDINA46. Five studies used app-based

npj Digital Medicine | (2024)7:253 2

https://doi.org/10.1038/s41746-024-01252-z Article

tools such as MCT & More47 and Mindshift36. Each specified tool was used Funnel plot analysis. Funnel plots with effect sizes plotted against
by only one study except for Deprexis, which was used in three studies. standard errors are depicted in Fig. 2a.
Out of all included studies, 17 were conducted in Germany, two in Publication bias would express itself in a preference for publishing
Sweden and USA respectively, and one in Hungary, Iran, Finland, Canada, significant compared to non-significant results. Because smaller studies
and Australia, respectively. need a higher effect size to reach significant effects compared to larger
Study search and selection flow is depicted in Fig. 1. studies, an asymmetrical distribution with more smaller studies with larger
effect sizes compared to larger studies would indicate publication bias. A
Risk of bias assessment regression analysis using standard error as a predictor of effect sizes suggests
Four studies were rated as high risk of bias and excluded from the significant asymmetry (z = 3.6, p < 0.001, i = 123 effects).
analysis. Out of the remaining studies, 19 were rated as medium risk Publication bias can be controlled by excluding the smallest studies48.
of bias and seven as low risk of bias. Among the most common bias After excluding studies with the largest standard errors (i = 4 effects, 3% of
concerns were asymmetrical attrition rates in control and interven- the total effects), another regression test showed no indicators of funnel plot
tion groups, high attrition rates with unclear reasons, alternating asymmetry (z = 1.89, p = 0.058, i = 119, Fig. 2b). The total effect size
allocations (rather than random allocation), and inadequate infor- remained unaltered (g = 0.33 [0.2, 0.46], p < 0.001), showing that the pub-
mation on blinding procedures (e.g., no specifications for statements lication bias correction did not impact the results. Thus, the results do not
such as “the procedure was blinded”). All four high risk studies were indicate publication bias.
excluded also due to unclear, high, or uneven attrition rates between
groups. P-curve analysis. P-curve analysis was used to investigate publication
The risk assessment is summarized in Table 1. bias further. A right-skewed p-curve would indicate an existing effect
while a left-skewed p-curve would indicate publication bias or p-hacking
Publication bias analyses as the latter curve would result from a tendency to acquire significant
Preliminary analyses were conducted to test for publication bias using p-values of just below .05 despite the absence of a true effect indicated by a
funnel plot and p-curve analyses. higher rate of results with smaller p-values. The p-curve is depicted in
Fig. 3.
Out of all effects, i = 109 effects provided a significant effect size of
p < 0.05, out of which i = 108 showed a p-value of p < 0.025. The significant
right-skewedness test (pbinominal < 0.001, zFull = -65.51, zHalf = -64.65,
pHalf < 0.001) suggested the existence of a true effect. Furthermore, the non-
significant flatness test (pbinominal = 1, zFull = 64.13, zHalf = 65.88, pHalf = 1)
provided no indicators that a true effect is not present.
In total, both funnel plot and p-curve analysis show no indicators of
publication bias or p-hacking, and that the observed effect is true.

Effect size analysis

A summary of all results is presented in Fig. 4.

Total effect. Total effect size with study as random effect revealed a
significant positive effect of EMH intervention (g = 0.3 [0.2, 0.39],
p < .001, k = 118). When only including effects of measures relevant to the
mental disorder symptoms (e.g., Beck depression scores for depressive
disorder patients) and removing measures not directly related to the
mental disorder’s symptoms or clinical outcomes (e.g., social support,
Fig. 1 | Study selection process. Flowchart depicting study selection. The first self-esteem), effect size increased (g = 0.36 [0.22, 0.5], p < 0.001, k = 83).
selection of 726 studies was found in five different databases. Following the eva- As expected given the variety of study designs and conditions, sig-
luation by exclusion criteria, 30 studies were selected for risk of bias evaluation. After nificant heterogeneity was observed for both the total effect
four studies were excluded for risk of bias, 26 studies were included in the meta- (Q(117) = 408.25, p < .001) and when including only clinically relevant
analysis. EMH e-mental health, RCT randomized controlled trial. outcomes (Q(82) = 647.91, p < 0.001).

Fig. 2 | Funnel plots of included studies. Funnel

plot across all effects (a) and after excluding studies
with the largest standard errors (b). The funnel plots
depict the effect sizes (Hedges’ g) plotted against the
studies’ (reversed) standard errors. Asymmetry
analyses found a significant asymmetry (a), but not
when excluding four effects with the largest standard
errors (b). As the effect size remains unaltered, the
results do not indicate publication bias.

npj Digital Medicine | (2024)7:253 3

https://doi.org/10.1038/s41746-024-01252-z Article

[0.18, 0.34], p < 0.001, k = 43) and psychodynamic (PD) treatments

(g = 0.35 [0.27, 0.43], p < 0.001, k = 39).

Follow-up stability. To investigate potential effects of measurement time

(e.g., a decrease of intervention efficacy for longer intervals after treat-
ment), a linear mixed model with measurement time as the fixed effect
and study as the random effect for effect sizes was calculated. Results
showed no significant effect of measurement time (t(61) = −00.97,
p = 0.337), showing no indication that the strength of the treatment effect
is influenced by the time passed between intervention and measurement.

Post-hoc analyses. Post-hoc analyses were conducted to investigate

differences between EMH medium/channel and effects of type of control
group. EMH medium analysis revealed significant effects for EMH tools
implemented as web-based tools (g = 0.32, CI [0.25, 0.37], p < 0.001) and
multimedia interventions (g = 0.79, CI [0.29, 1.29], p = 0.002). Effects for
app-based and SMS-based EMH tools were not significant. However,
multimedia was used by only one study43. The only specific EMH tool
used by multiple studies was Deprexis, which showed a significant effect
(g = 0.61, CI [0.46, 0.77], p < 0.001).
Control group analysis revealed that EMH interventions significantly
improve mental health outcomes compared to passive controls (no active
Fig. 3 | P-curve of included studies. P-curve including the meta-analysis’ 109 sig-
nificant effects, compared to a hypothetical null-effect curve and a hypothetical 33%
treatment; g = 0.29, CI [0.19, 0.39], p < .001), active controls (active treat-
power effect curve. Analysis shows a significant right skewedness, indicating the ment alternative to EMH; g = 0.32, CI [0.24, 0.4], p < .001), and active
existence of a true effect. controls to which the EMH intervention was added to in the intervention
condition (g = 0.3, CI [0.22, 0.39], p < 0.001). Thus, EMH interventions
show efficacy compared to active treatments and usual and when used in
addition to usual treatments.
Few studies focused on patients affected by anxiety disorders, com-
plicating interpretations of the presented results. Meanwhile, studies with
transdiagnostic patients often included patients with anxiety disorders and
measured anxiety symptoms (e.g., GAD-7). To gain further insight into the
effects of EMH treatment on anxiety disoders, an additional post-hoc
analysis has been conducted measuring the efficacy of EMH treatment on
anxiety symptoms specifically. The analysis showed a significant effect on
anxiety symptoms (g = 0.39, CI [0.18, 0.59], p < 0.001).

Discussion
EMH procedures have shown to be a viable tool for the treatment of
mental disorders, yet research on EMH in inpatient settings is rela-
tively sparse. The current work presents, to our knowledge, the first
meta-analysis providing evidence for the efficacy of EMH in inpatient
Fig. 4 | Synthesized effect sizes. Effect sizes, confidence intervals, and number of treatment and aftercare. We found a significant small effect of EMH
effects across conditions, controlled for study. Note. Total = across all data; relevant treatment (g = 0.3).When focusing on disorder symptoms and clini-
effects = only effects of measures relevant to the mental condition are included;
cally relevant outcomes, the effect size is further increased (g = 0.36),
blended = treatment with EMH blended with inpatient care; aftercare = treatment
signalling that EMH procedures are suitable as interventions tailored
after inpatient care. CBT cognitive-behavioural therapy, PD Psychodynamic therapy.
to mental disorders in inpatient settings. A preliminary analysis further
found no indicators of publication bias or p-hacking within the
Treatment type. By-treatment type analysis revealed that both blended literature.
interventions during inpatient stay (g = 0.42 [0.27, 0.58], p < 0.001, The effect remained significant when dividing the studies into the
k = 19) and aftercare treatments following inpatient stay (g = 0.29 [0.24, common implementation types of EMH, first when blended with in-person
0.34], p < 0.001, k = 99) showed significant effects. inpatient treatment (g = 0.42) and second as an aftercare treatment following
inpatient intervention (g = 0.29). The majority of studies (21 out of 26) used
Mental condition. By-condition analysis revealed significant effects of an aftercare setting with the goal to ensure stabilization and prevent relapse of
EMH interventions for eating disorder (g = 0.19 [0.07, 0.32], p = .003, inpatient cases. Inpatient cases tend to be more severe compared to outpatient
k = 17), mood disorder (g = 0.38 [0.28, 0.49], p < 0.001, k = 22), psychotic cases, with worse post-treatment outcomes when not sufficiently supported
disorder (g = 0.43 [0.27, 0.58], p < 0.001, k = 10), return to work (g = 0.21 by aftercare following discharge30–32. The present results suggest that EMH
[0.12, 0.3], p < 0.001, k = 24), and transdiagnostic patients (g = 0.4 [0.31, can provide such an effective tool, closing an important mental health sup-
0.49], p < 0.001, k = 34). No significant effects were found for anxiety ply gap.
disorders (g = 0.35 [−0.22, 0.93], p = 0.23, k = 3), mental comorbidity By-disorder analysis found that EMH was especially effective for
with somatic disorders (g = 0.19 [−0.02, 0.39], p = 0.072, k = 6), and psychotic disorders (g = 0.42), transdiagnostic patient groups (g = 0.4), and
substance abuse (g < 0.01 [−0.27, 0.28], p = 0.964, k = 2). mood disorders (g = 0.38). The results are comparable to meta-analyses
finding small yet significant effects of EMH in outpatient settings for mood
Type of therapy. Analysis by type of therapy revealed significant effects disorders22, providing evidence that the effects are comparable to inpatient
for cognitive behavioural therapy (CBT)-based treatments (g = 0.26 settings.

npj Digital Medicine | (2024)7:253 4

https://doi.org/10.1038/s41746-024-01252-z Article

The positive effect of EMH treatment for psychotic disorders is sur- improvements and remission prevention following inpatient treatment
prising given that EMH interventions may worsen psychotic patients’ since other aftercare practices are lacking or minimal. Especially web-based
concerns about technology and being recorded due to psychopathological EMH treatment has been shown to be effective throughout multiple studies
paranoid tendencies49. Furthermore, the effect contrasts the negative out- (g = 0.32) compared to SMS- or app-based approaches. Hence, practitioners
comes reported in studies investigating psychotic patients42,50,51. While the may use EMH tools both as additives and as alternatives to regular treat-
results complement previous research on the effectiveness of EMH out- ment, and especially for aftercare following inpatient treatment. Web-based
patient treatments for schizophrenia and psychosis52, the usage of EMH EMH tools have shown efficacy in most studies.
interventions for psychotic disorders remains not well developed, and their The meta-analysis is limited by the small number of studies especially
efficacy cannot be reliably estimated with the current research. For inpatient for subgroup analyses, as some subgroups (e.g., anxiety disorder or substance
settings, SMS-based aftercare reminders for medication adherence did not abuse patients, or whole health approaches) only include a single study each
improve patient outcomes50. The HEINS web-based aftercare program and can thus not be properly interpreted. Although a total effect was found
containing multiple modules (including psychoeducation, crisis plans, with a sufficient number of trials, further RCT research is needed to conduct
contacts to psychiatrists, and supportive monitoring) meanwhile showed more conclusive meta-analyses for subgroup-related research areas.
positive user acceptance and adherence42, and Horyzons, an online social The small number of studies precludes further interesting analyses
therapy aftercare program containing multiple features (including psy- relevant to the design and implementation of EMH methods. For example, a
choeducation, skill development support, peer-to-peer conversations, and previous meta-analysis on outpatient settings found that specific EMH
expert support), improved patient employment and reduced emergency methods were more effective for certain disorders (e.g., chatbots for
room visits compared to usual care51. Given that both Horyzons and HEINS depression, mood monitoring features for anxiety). Such research questions
are interactive support units containing multiple modules, the results sug- may be tackled in future meta-analyses when an adequate number of RCTs
gest that more extensive EMH treatment is needed to ensure aftercare of have been conducted. Meta-analyses and reviews are generally limited by
patients with psychosis. Patients with severe illnesses such as psychosis may the terms used and search outputs when conducting literature searches.
not be able to effectively utilize digital health tools. EMH tools are to be used Even though two literature searches (February 2024 and July 2024) were
with caution when treating patients with psychosis and should be used in done for this meta-analysis, it may still not include all relevant literature.
addition to in-person treatment instead of an alternative. Furthermore, this meta-analysis was not preregistered. However, all relevant
For outpatient treatment, the efficacy of EMH treatment for anorexia documents are publicly available.
nervosa is not well researched, potentially due to the severity of the disorder Specific neuropsychological and cognitive measures were excluded
and the presumed necessity for face-to-face treatment by clinicians21. Out of from this meta-analysis to focus the research on explicit mental health
seven studies investigating eating disorder patients, four focused mainly on outcomes. However, mental health deficits often co-occur with cognitive
bulimia nervosa41,46,53,54. When excluding a follow-up study55 and a pilot deficits, for example in memory, concentration, or problem-solving tasks.
RCT56, only one proper RCT study focused on anorexia nervosa57. Although Although disorder-specific questionnaire measures encompass the mea-
the initial results are promising, caution should be taken when transferring surement of such deficits, future research can focus on the effect of EMH
the results onto patients with anorexia nervosa given that the disorder leads interventions for the improvement of cognitive skills in patients affected by
to severe consequences including somatic complications that may be mental health disorders specifically.
insufficiently tracked and treated through digital means. Out of all 26 included studies 20 were conducted in Western or
Meanwhile, no significant effects for anxiety symptoms, comorbidity Northern Europe (17 in Germany, two in Sweden, one in Finland), three
with somatic disorders, or substance abuse disorders were found. However, were conducted in North America (two in the USA and one in Canada), one
only one study investigated anxiety symptoms36. Meanwhile, multiple stu- in Australia, one in Hungary, and one in Iran. Research from other regions,
dies with transdiagnostic patient groups included patients with anxiety such as Africa or East Asia, was absent. This may be due to differences in
disorders37,39,58. A post-hoc analysis focusing on anxiety symptoms revealed healthcare systems in different regions, and treatments alternative to
a significant effect (g = 0.39). Inpatient treatment is typically not indicated inpatient treatment for more severe health cases. Thus, the results of this
for anxiety disorders, which may explain the low number of studies. Given meta-analysis are mainly derived from studies conducted in countries with
that EMH interventions are effective in treating anxiety disorders in out- populations majorly of European descent. In order to generalize the
patient settings22, and that the post-hoc analysis revealed a significant reported findings, future research may aim to investigate EMH tools in more
improvement in anxiety symptoms, the current negative findings on EMH diverse populations.
inpatient treatment for anxiety disorders are to be interpreted with caution. Engagement and adherence are major concerns when applying EMH
A similar caution can be expressed for the negative result on substance abuse tools60–63. Effects of EMH on attrition were mitigated in this analysis by
patients, which has been investigated by only one study59. Furthermore, including group attrition effects in the RoB assessment: in fact, all four high
future research ought to differentiate effects of EMH for different anxiety risk studies were excluded due to unclear or uneven attrition rates.
diagnoses in inpatient care, as EMH outpatient treatment effectiveness has Engagement can be defined as usage as intended, measured for example
been found to differ across anxiety disorders22. through use frequency or completion60. Various included studies excluded
Analysis by type of therapy revealed the effectiveness of both CBT- participants with low engagement despite completion41 and hence con-
(g = 0.26) and PD- (g = 0.35) based interventions, showing that EMH trolled for low engagement. Included studies mostly did not report direct
treatment is effective when based on either of these types of psychotherapy effects on engagement on outcomes. One study found no effect of EMH tool
Finally, the result that observation period did not affect outcomes use (assessed via logs) on symptom severity56. Similarly, other studies did not
indicates that EMH-based treatment effects do not deteriorate with time find a correlation between EMH use frequency and symptom
passed after treatment, indicating the long-term stability of the effects. improvement47, completed models and symptom improvement64, or dif-
However, the latest measurement used in this analysis was 24 months after ferences between high- and low-frequency users59. Meanwhile, the number
treatment. Hence, results cannot be interpreted for longer periods. of completed EMH courses did significantly improve symptoms in patients
In general, the meta-analysis shows the efficacy of EMH treatment with anorexia nervosa55. Although there are only few studies and results are
across different mental health disorders and types of therapy. Hence, mental not consistent, the results nevertheless indicate that use frequency or
health treatment can profit from integrating EMH into the patient journey. intensity does generally not affect the treatment efficacy. Finally, some
Given that EMH add-on also significantly improves outcomes compared to studies report improved engagement in the intervention compared to a
a regular active control group (g = 0.3), adding EMH to regular practices can control group65,66, indicating that EMH intervention may improve
improve overall treatment outcomes. Since treatment as usual tends to be engagement behaviour. Future research may investigate effects of such
minimal for aftercare treatment, EMH can facilitate long-term engagement when implementing EMH tools.

npj Digital Medicine | (2024)7:253 5

https://doi.org/10.1038/s41746-024-01252-z Article

Given that various measurement outcomes were used and summarized randomized controlled trials, 5) did not provide sufficient information to
to generalize a wider range of findings, results do not consistently reflect the extract the relevant data (e.g., outcome measures or sample sizes), and 6)
most clinically relevant outcomes (e.g., remission or relapse rates) which showed a high risk of bias assessed via the Risk of Bias tool (see next
were only reported by six studies for varying mental disorders. Instead, the section)39. Neuropsychological or cognitive were excluded to focus the meta-
majority of research studies relied on symptom questionnaires. In total, a analysis on mental health treatment effects. Although cognitive or neu-
majority of the studies included were assessed with some concerns regarding ropsychological deficits can be symptoms of mental health disorders,
risk of bias. Due to the low number of high-quality research with low bias symptom-focused measures of mental health deficits (e.g., depressiveness
and large sample sizes, results should be interpreted with some degree of questionnaires for clinical depression) provide a more discriminative esti-
caution. EMH implementations furthermore involve certain risks67 such as a mation of mental health deficits.
lack of quality standards68, data privacy issues18, or a lack of digital literacy by Three independent raters took part in the literature selection. In case of
practitioners19. Despite promising results in this meta analysis, in the context disagreements, the raters discussed the study until agreement was found.
of such risks, more high quality RCT research is necessary for a more
rigorous assessment of EMH efficacy. Risk of bias assessment. Risk of bias was assessed using the Cochrane
In conclusion, the results indicate that EMH procedures are an effective risk-of-bias tool for randomized trials (RoB 2)39. RoB 2 is designed to
tool in the treatment and aftercare of inpatients, especially for psychotic, assess the risk of an RCT’s bias by classifying the level of risk for the
mood disorder, and eating disorder, and patient groups combining different following domains: random sequence generation, allocation conceal-
diagnoses. EMH tools can be used both in addition to in-person treatment ment, blinding of participants and personnel, blinding of outcome
and when in-person treatment is not available, e.g., for aftercare. Future assessment, incomplete outcome data, and selective reporting. Examples
research should investigate effects of EMH tools for the inpatient treatment of risk of bias include non-random or semi-random participant grouping
of specific disorders and the relevance of the specific tools used. Larger (incl. alternating allocation); high, uneven, or unexplained participant
sample sizes and randomized trials are warranted to substantiate these attrition between groups; lack of blinding; or unreported discrepancies
effects. between the study protocol and study. If no information on a domain was
provided, the particular domain was assessed with medium risk.
Methods Domains were rated on three levels: low, medium, or high risk of bias.
This review was conducted in accordance to the Preferred Reporting Items Research studies with a high risk of bias were excluded from the analysis.
for Systematic Reviews and Meta-Analyses (PRISMA) guidelines69 and
Cochrane Handbook guidelines for meta-analyses and systematic reviews39. Measurement selection
For the total analyses, only measures related to clinical symptoms and
Literature psychosocial performance were included. These include: metric variables of
Literature search. The literature databases SciencGov, PsycInfo, disorder-related incidents (relapses, readmissions, abstinence, admissions);
PubMed, and CENTRAL were searched for published literature. In disorder-related symptom severity measurements; general psychopathol-
addition, the ProQuest Database was searched for dissertation theses, and ogy, well-being, or quality of life; employment-related measures (when
ICTRP and ClinicalTrials were searched for trial result registers. relevant); and specific mental or psychosocial measures expected to corre-
To aim for high sensitivity according to Cochrane guidelines39, we used late with symptom severity (e.g., self-esteem, positive and negative affect,
multiple search terms in relation to the following topics: e-mental health stress). All relevant measures in a study were included in an analysis and
(digital, online, e-mental health, technology-based, web-based, internet- controlled by treating study as a random effect.
based, mobile-based), treatment setting (psychotherapy, psychiatric, psy-
chosomatic), inpatient setting (inpatient, ward patient, hospitalized), and Variable summarization
experimental design (RCT, randomized controlled trial). The search term To investigate the relevant research questions, studies and measures were
used corresponds to the following: (“digital” OR “online” OR “e-mental categorized by the following system.
health” OR “technology-based” OR “web-based” OR “internet-based” OR EMH treatment type was categorized into either blended intervention
“mobile-based”) AND (“psychotherapy” OR “psychiatric” OR “psychoso- (EMH was implemented into the inpatient setting) or aftercare treatment
matic”) AND (“inpatient” OR “ward patient” OR “hospitalized”) AND (EMH was provided after completing inpatient setting).
(“RCT” OR “randomized controlled trial”). Two researchers conducted the The variable Disorder type was classified into the following categories
literature search in February 2024. Literature search was performed in based on the patient group investigated in the study: anxiety disorders (ICD-
English and German. 10 diagnoses F40 and F41), eating disorders (ICD-10 diagnoses F50) mood
A secondary search was conducted in July 2024 by extending the search disorders (ICD-10 diagnoses F3), psychotic disorders (ICD-10 diagnoses F2),
to use the terms “e-health”, “mhealth”, and “telemedicine”, and using the substance abuse disorders (ICD-10 diagnoses F1x.2), or their DSM-5 diag-
mesh terms “digital health”, “telemedicine”, “psychotherapy”, “psychoso- nostic equivalents. A study treating patient groups from different categories
matic medicine”, and “inpatients” if applicable, for the databases CENTAL was classified as transdiagnostic. A study was categorized as somatic
and PubMed. The secondary literature search did not yield any new viable comorbidity if the effects of EMH interventions on mental health outcomes
studies. in somatic inpatient groups were investigated (e.g., stress or anxiety
symptoms in cancer patients). Finally, the category return to work was used
Literature selection. We included research studies providing EMH for studies focussing on outcomes related to workplace reintegration fol-
interventions during inpatient treatment or aftercare following inpatient lowing inpatient care.
treatment, and studies investigating psychiatric symptoms co-occurring The variable type of therapy was classified according to the type of
in patients hospitalized for physical conditions (e.g., stress or depression therapy the EMH intervention was based on according to the authors. If no
symptoms in cancer patients). Cluster and pilot RCTs were included type of therapy was mentioned, the variable was valued as not available.
as well.
Studies were excluded if they 1) did not investigate the effect of EMH Data extraction
intervention or aftercare methods, 2) did not investigate inpatients (either Data was summarized on multiple variables: author, title, year, country, type
during or after inpatient intervention), 3) did not investigate mental health (aftercare, blended treatment), treated mental disorder, somatic illness (if
measures as treatment outcomes (e.g., only focusing on somatic symptoms present), digital method, type of therapy, type of control group (active,
or acceptability of the intervention; specific neuropsychological or cognitive passive), outcome measure, follow-up, sample sizes, and outcome results
outcomes like problem-solving skills were also excluded), 4) were not (means, standard deviations, odds ratios, effect sizes). Data was extracted by

npj Digital Medicine | (2024)7:253 6

https://doi.org/10.1038/s41746-024-01252-z Article

one rater and verified by two other independent raters. Study characteristics countries, 1990–2016: A systematic analysis for the global burden of
were tabulated according to the planned subgroup analyses. Studies with disease study 2016. Lancet 390, 1211–1259 (2017).
insufficient data were excluded from the (sub-)analyses. 4. Santomauro, D. F. et al. Global prevalence and burden of depressive
and anxiety disorders in 204 countries and territories in 2020 due to
Data transformation the COVID-19 pandemic. Lancet 398, 1700–1712 (2021).
Hedges’ g was used to report effect sizes as it outperforms Cohen’s d for small 5. Ahmed, N. et al. Mental health in Europe during the COVID-19 pandemic:
sample sizes. Cohen’s d effect sizes and variances were transformed to A systematic review. Lancet Psychiatry 10, 537–556 (2023).
Hedges’ g values and variances using the following formulas48: 6. Alonso, J. et al. Prevalence of mental disorders in Europe: Results

 from the European study of the epidemiology of Mental Disorders
3 (esemed) project. Acta Psychiatr. Scandinavica 109, 21–27
g ¼d 1 ð1Þ
4df  1 (2004).
7. Sacco, R., Camilleri, N., Eberhardt, J., Umla-Runge, K. & Newbury-


3 Birch, D. A systematic review and meta-analysis on the prevalence of
vg ¼ vd 1  2 ð2Þ mental disorders among children and adolescents in Europe.
4df  1
European Child & Adolescent Psychiatry https://doi.org/10.1007/
When a study reported odds ratio (OR) values, values were first s00787-022-02131-2 (2022).
transformed into Cohen’s d using the following formula48: 8. Wittchen, H. U. et al. The size and burden of mental disorders and
other disorders of the brain in Europe 2010. Eur.
pffiffiffi
3 Neuropsychopharmacol. 21, 655–679 (2011).
d ¼ logðORÞ ð3Þ
π 9. Zuberi, A. et al. Prevalence of mental disorders in the WHO Eastern
Mediterranean Region: A systematic review and meta-analysis. Front.
Cohen’s d values were then transformed into Hedges’ g according to Psychiatry 12, (2021).
Formula 1. 10. Jacobi, F. et al. Psychische Störungen in der Allgemeinbevölkerung.
Der Nervenarzt 85, 77–87 (2014).
Data analysis 11. Andrade, L. H. et al. Barriers to mental health treatment: Results from
Heterogeneity was tested and pre-assumed given the variety of setups in the WHO World Mental Health Surveys. Psychol. Med. 44,
research studies and subgroup analyses were therefore decided a priori. 1303–1317 (2013).
Fixed-random effects models were used with study as a random factor. 12. EU-Compass for action on Mental Health and well-being. Public
To assess the results’ robustness, the total effect is analysed two times, Health Available at: https://health.ec.europa.eu/non-communicable-
first using the whole range of data, and second using only outcomes that diseases/mental-health/eu-compass-action-mental-health-and-
are clinically relevant (limited to symptom severity and clinical out- well-being_en (Accessed: 21st May 2024).
comes). Effects’ certainty and confidence were assessed through risk of 13. Koopmanschap, M. A., Brouwer, W. B. F., Hakkaart-van Roijen, L. &
bias assessment according to Cochrane guidelines and by investigating van Exel, N. J. A. Influence of waiting time on cost-effectiveness. Soc.
publication bias using funnel plot and p-curve analyses. The meta- Sci. Med. 60, 2501–2504 (2005).
analysis was not preregistered. No protocol is available for the meta- 14. Reichert, A. & Jacobs, R. The impact of waiting time on patient
analysis. Confidence was assessed by calculating confidence intervals outcomes: Evidence from early intervention in psychosis services in
from standard errors. England. Health Econ. 27, 1772–1787 (2018).
Post-hoc analyses were decided after the data was analysed for the main 15. van Dijk, D. A. et al. Worse off by waiting for treatment? the impact of
hypotheses. Post-hoc analyses included the effect of EMH medium, the role waiting time on clinical course and treatment outcome for depression
of control group, and the effect of EMH on anxiety symptoms specifically. in routine care. J. Affect. Disord. 322, 205–211 (2023).
16. Sherman, M. L., Barnum, D. D., Buhman-Wiggs, A. & Nyberg, E.
Data availability Clinical intake of child and adolescent consumers in a rural community
Data including the complete list of searched literature, the included studies, mental health center: Does wait-time predict attendance? Community
extracted data, and risk assessment are publicly available at https://osf.io/ Ment. Health J. 45, 78–84 (2008).
bc59e. Thus, all data is provided to replicate assessment of literature 17. Williams, M. E., Latta, J. & Conversano, P. Eliminating the wait for
according to inclusion criteria and risk of bias, as well as all data necessary to Mental Health Services. J. Behav. Health Serv. Res. 35, 107–114 (2007).
replicate the analyses. 18. Köhnen, M., Dirmaier, J. & Härter, M. Potenziale und
Herausforderungen von E-mental-health-interventionen in der
Code availability Versorgung Psychischer störungen. Fortschr. der Neurologie ·
The R code for the analysis is publicly available at https://osf.io/bc59e. Main Psychiatr. 87, 160–164 (2019).
and subgroup analyses and visualization of results were conducted via 19. Weitzel, E. C. et al. E-mental health in Germany — what is the current
RStudio (ver. 2021.9.1.0, R version 4.1.2). The R packages dmetar and use and what are experiences of different types of health care
metafor were used for the analyses70,71. providers for patients with mental illnesses? Arch. Public Health 81,
1–6 (2023).
Received: 24 May 2024; Accepted: 3 September 2024; 20. Bolinski, F. et al. The effect of E-mental health interventions on
academic performance in university and college students: A meta-
analysis of randomized controlled trials. Internet Interventions 20,
References 100321 (2020).
1. Borges, G. et al. Twelve-month prevalence of and risk factors for 21. Firth, J. et al. Can smartphone mental health interventions reduce
suicide attempts in the World Health Organization World Mental symptoms of anxiety? A meta-analysis of randomized controlled
Health Surveys. J. Clin. Psychiatry 71, 1617–1628 (2010). trials. J. Affect. Disord. 218, 15–22 (2017).
2. Doran, C. M. & Kinchin, I. A review of the economic impact of mental 22. Linardon, J. et al. Current evidence on the efficacy of mental health
illness. Aust. Health Rev. 43, 43 (2019). smartphone apps for symptoms of depression and anxiety. A meta‐
3. Vos, T. et al. Global, regional, and national incidence, prevalence, and analysis of 176 randomized controlled trials. World Psychiatry 23,
years lived with disability for 328 diseases and injuries for 195 139–149 (2024).

npj Digital Medicine | (2024)7:253 7

https://doi.org/10.1038/s41746-024-01252-z Article

23. Linardon, J., Shatte, A., Messer, M., Firth, J. & Fuller-Tyszkiewicz, M. 41. Jacobi, C. et al. Web-based aftercare for women with bulimia nervosa
E-mental health interventions for the treatment and prevention of following inpatient treatment: Randomized controlled efficacy trial. J.
eating disorders: An updated systematic review and meta-analysis. J. Med. Internet Res. 19, e321 (2017).
Consulting Clin. Psychol. 88, 994–1007 (2020). 42. Gallinat, C. et al. Feasibility of an intervention delivered via mobile
24. Simblett, S., Birch, J., Matcham, F., Yaguez, L. & Morris, R. A systematic phone and internet to improve the continuity of care in schizophrenia:
review and meta-analysis of E-mental health interventions to treat A randomized controlled pilot study. Int. J. Environ. Res. Public Health
symptoms of posttraumatic stress. JMIR Mental Health 4, (2017). 18, 12391 (2021).
25. Stratton, E. et al. Effectiveness of ehealth interventions for reducing 43. Nolte, S. et al. Do sociodemographic variables moderate effects of an
mental health conditions in employees: A systematic review and internet intervention for mild to moderate depressive symptoms? an
meta-analysis. PLOS ONE 12, (2017). exploratory analysis of a randomised controlled trial (evident)
26. Löbner, M. et al. What comes after the trial? an observational study of including 1013 participants. BMJ Open 11, e041389
the real-world uptake of an e-mental health intervention by general (2021).
practitioners to reduce depressive symptoms in their patients. Int. J. 44. Abadi, M. et al. Achieving whole health: A preliminary study of TCMLH,
Environ. Res. Public Health 19, 6203 (2022). a group-based program promoting self-care and empowerment
27. Rost, T. et al. User acceptance of computerized cognitive behavioral among veterans. Health Educ. Behav. 49, 347–357 (2021).
therapy for depression: Systematic review. Journal of Medical Internet 45. Becker, J., Kreis, A., Beutel, M. E. & Zwerenz, R. Wirksamkeit der
Research 19, (2017). Internetbasierten, Berufsbezogenen Nachsorge GSA-online im
28. Wangler, J. & Jansky, M. How can primary care benefit from Digital Anschluss an die stationäre psychosomatische Rehabilitation:
Health Applications? – a quantitative, explorative survey on attitudes Ergebnisse einer randomisiert Kontrollierten Studie. Die Rehabilitation
and experiences of General Practitioners in Germany. BMC Digital 61, 276–286 (2022).
Health 2, 1–14 (2024). 46. Gulec, H. et al. A randomized controlled trial of an internet-based
29. Zipfel, S., Herzog, W., Kruse, J. & Henningsen, P. Psychosomatic posttreatment care for patients with eating disorders. Telemed.
medicine in Germany: More timely than ever. Psychother. Psychosom. e-Health 20, 916–922 (2014).
85, 262–269 (2016). 47. Bruhns, A., Lüdtke, T., Moritz, S. & Bücker, L. A mobile-based
30. Franz, M. et al. Stationäre Tiefenpsychologisch Orientierte intervention to increase self-esteem in students with depressive
psychotherapie bei Depressiven Störungen (stop-D) - erste befunde symptoms: Randomized controlled trial. JMIR mHealth uHealth 9,
einer naturalistischen, multizentrischen Wirksamkeitsstudie. Z. f.ür. e26498 (2021).
Psychosomatische Med. und Psychotherapie 61, 19–35 (2015). 48. Borenstein, M. Computing effect sizes for meta-analysis. (Wiley-
31. Gönner, S., Bischoff, C., Ehrhardt, M. & Limbacher, K. Effekte Blackwell, 2011).
therapiezielorientierter Kognitiv-Verhaltenstherapeutischer 49. Philippe, T. J. et al. Digital Health Interventions for delivery of mental
Nachsorgemaßnahmen auf den therapietransfer im Anschluss an eine health care: Systematic and comprehensive meta-review. JMIR Ment.
Stationäre psychosomatische Rehabilitationsbehandlung. Die Health 9, e35159 (2022).
Rehabilitation 45, 369–376 (2006). 50. Välimäki, M., Kannisto, K. A., Vahlberg, T., Hätönen, H. & Adams, C. E.
32. Zeeck, A., Wietersheim, Jvon, Weiss, H., Beutel, M. & Hartmann, A. Short text messages to encourage adherence to medication and
The INDDEP Study: Inpatient and day hospital treatment for follow-up for people with psychosis (mobile.net): Randomized
depression – symptom course and predictors of change. BMC controlled trial in Finland. J. Med. Internet Res. 19, e245 (2017).
Psychiatry 13, 100 (2013). 51. Alvarez‐Jimenez, M. et al. The Horyzons Project: A randomized
33. Giel, K. E. et al. Efficacy of post-inpatient aftercare treatments for controlled trial of a novel online social therapy to maintain treatment
anorexia nervosa: A systematic review of randomized controlled trials. effects from specialist first‐episode psychosis services. World
J. Eat. Disord. 9, 129 (2021). Psychiatry 20, 233–243 (2021).
34. Hegedüs, A., Kozel, B., Richter, D. & Behrens, J. Effectiveness of 52. Clarke, S., Hanna, D., Mulholland, C., Shannon, C. & Urquhart, C. A
transitional interventions in improving patient outcomes and service systematic review and meta-analysis of digital health technologies
use after discharge from psychiatric inpatient care: A systematic effects on psychotic symptoms in adults with psychosis. Psychosis
review and meta-analysis. Front. Psychiatry 10, 969 11, 362–373 (2019).
(2020). 53. Bauer, S., Okon, E., Meermann, R. & Kordy, H. Technology-enhanced
35. Vittengl, J. R., Clark, L. A., Dunn, T. W. & Jarrett, R. B. Reducing maintenance of treatment gains in eating disorders: Efficacy of an
relapse and recurrence in Unipolar Depression: A comparative meta- intervention delivered via text messaging. J. Consulting Clin. Psychol.
analysis of cognitive-behavioral therapy’s effects. J. Consulting Clin. 80, 700–706 (2012).
Psychol. 75, 475–488 (2007). 54. Bauer, S., Okon, E., Meermann, R. & Kordy, H. SMS-Nachsorge:
36. Sharma, G. et al. Brief app-based cognitive behavioral therapy for Sektorenübergreifende Versorgung für patientinnen mit bulimia
anxiety symptoms in psychiatric inpatients: Feasibility randomized nervosa. Verhaltenstherapie 23, 204–209 (2013).
controlled trial. JMIR Formative Res. 6, e38460 (2022). 55. Fichter, M. M., Quadflieg, N. & Lindner, S. Internet-based relapse
37. Zwerenz, R. et al. Transdiagnostic, psychodynamic web-based self- prevention for anorexia nervosa: Nine- month follow-up. J. Eat.
help intervention following inpatient psychotherapy: Results of a Disord. 1, 23 (2013).
feasibility study and randomized controlled trial. JMIR Ment. Health 4, 56. Neumayr, C., Voderholzer, U., Tregarthen, J. & Schlegl, S. Improving
e41 (2017). aftercare with technology for anorexia nervosa after intensive inpatient
38. Hennemann, S., Farnsteiner, S. & Sander, L. Internet- and Mobile- treatment: A pilot randomized controlled trial with a therapist‐guided
based aftercare and relapse prevention in mental disorders: A smartphone app. Int. J. Eat. Disord. 52, 1191–1201 (2019).
systematic review and recommendations for Future Research. 57. Fichter, M. M. et al. Does internet-based prevention reduce the risk
Internet Interventions 14, 1–17 (2018). of relapse for anorexia nervosa? Behav. Res. Ther. 50, 180–190
39. Cochrane Handbook for Systematic Reviews of interventions. (2012).
(Cochrane, 2019). 58. Ebert, D., Tarnowski, T., Gollwitzer, M., Sieland, B. & Berking, M. A
40. Kordy, H. et al. Internet-delivered disease management for recurrent transdiagnostic internet-based maintenance treatment enhances the
depression: A multicenter randomized controlled trial. Psychother. stability of outcome after inpatient cognitive behavioral therapy: A
Psychosom. 85, 91–98 (2016). randomized controlled trial. Psychother. Psychosom. 82, 246–256 (2013).

npj Digital Medicine | (2024)7:253 8

https://doi.org/10.1038/s41746-024-01252-z Article

59. Harrington, K. F. et al. Web-based smoking cessation intervention 77. Schlicker, S., Ebert, D. D., Middendorf, T., Titzler, I. & Berking, M.
that transitions from inpatient to outpatient: Study protocol for a Evaluation of a text-message-based maintenance intervention for
randomized controlled trial. Trials 13, 123 (2012). major depressive disorder after inpatient cognitive behavioral therapy.
60. Kernebeck, S., Busse, T. S., Ehlers, J. P. & Vollmar, H. C. Adhärenz J. Affect. Disord. 227, 305–312 (2018).
digitaler Interventionen im Gesundheitswesen: Definitionen, 78. Zwerenz, R. et al. Improving the Course of Depressive Symptoms
Methoden und offene Fragen [Adherence to digital health After Inpatient Psychotherapy Using Adjunct Web-Based Self-Help:
interventions: definitions, methods, and open questions]. Follow-Up Results of a Randomized Controlled Trial. J. Med. Internet
Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz Res. 21, e13655 (2019).
64, 1278–1284 (2021). OctGermanEpub 2021 Sep 24. PMID: 79. Shaygan, M., Yazdani, Z. & Valibeygi, A. The effect of online
34559252; PMCID: PMC8492574. multimedia psychoeducational interventions on the resilience and
61. Sieverink, F., Kelders, S. M. & van Gemert-Pijnen, J. E. Clarifying the perceived stress of hospitalized patients with COVID-19: A pilot
Concept of Adherence to eHealth Technology: Systematic Review on cluster randomized parallel-controlled trial. BMC Psychiatry 21,
When Usage Becomes Adherence. J. Med Internet Res 19, e402 93 (2021).
(2017). Dec 6PMID: 29212630; PMCID: PMC5738543. 80. Levis, M. et al. An implementation and effectiveness study evaluating
62. Beatty, L. & Binnion, C. A Systematic Review of Predictors of, and Conflict Analysis in VA residential substance abuse services: Whole
Reasons for, Adherence to Online Psychological Interventions. Int J. health informed self-guided online care. EXPLORE 18, 688–697
Behav. Med 23, 776–794 (2016). DecPMID: 26957109. (2022).
63. Eysenbach, G. The law of attrition. J. Med. Internet Res. 7, e11 (2005).
Mar 31PMID: 15829473; PMCID: PMC1550631. Author contributions
64. Norlund, F. et al. Internet-based cognitive behavioral therapy for A.D. drafted the manuscript, conducted study search and selection,
symptoms of depression and anxiety among patients with a recent analysed the data, and revised the manuscript. I.S. and A.L.F. revised the
myocardial infarction: The U-CARE heart randomized controlled trial. manuscript and validated study search and selection. C.J., A.B., M.T., A.R.,
J. Med. Internet Res. 20, e88 (2018). and G.G.D. revised the manuscript. M.T. acquired resources and funding.
65. Holländare, F. et al. Randomized trial of internet-based relapse
prevention for partially remitted depression. Acta Psychiatr. Funding
Scandinavica 124, 285–294 (2011). Open Access funding enabled and organized by Projekt DEAL.
66. Zwerenz, R. et al. Online self-help as an add-on to inpatient
psychotherapy: Efficacy of a new blended treatment approach. Competing interests
Psychother. Psychosom. 86, 341–350 (2017). The authors declare no competing interests.
67. Ebert, D. D. et al. Internet- and mobile-based psychological
interventions: Applications, efficacy, and potential for improving Additional information
mental health. Eur. Psychologist 23, 167–187 (2018). Supplementary information The online version contains
68. Weitzel, E. C. et al. E-mental-health und Digitale supplementary material available at
Gesundheitsanwendungen in Deutschland. Der Nervenarzt 92, https://doi.org/10.1038/s41746-024-01252-z.
1121–1129 (2021).
69. Page, M. J. et al. The Prisma 2020 statement: An updated guideline for Correspondence and requests for materials should be addressed to
reporting systematic reviews. BMJ. https://doi.org/10.1136/bmj.n71 Alexander Diel.
(2021).
70. Harrer, M., Cuijpers, P., Furukawa, T. A. & Ebert, D. D. Doing meta- Reprints and permissions information is available at
analysis with R https://doi.org/10.1201/9781003107347 (2021). http://www.nature.com/reprints
71. Viechtbauer, W. Conducting meta-analyses inrwith
themetaforpackage. J. Stat. Softw. 36, (2010). Publisher’s note Springer Nature remains neutral with regard to
72. Bischoff, C. et al. Wirksamkeit von Handheld-Gestütztem jurisdictional claims in published maps and institutional affiliations.
Selbstmanagement (e-coaching) in der rehabilitationsnachsorge.
Verhaltenstherapie 23, 243–251 (2013). Open Access This article is licensed under a Creative Commons
73. Ebert, D. et al. Web-basierte rehabilitationsnachsorge nach Attribution 4.0 International License, which permits use, sharing,
stationärer Psychosomatischer Therapie (W-rena). Die Rehab. 52, adaptation, distribution and reproduction in any medium or format, as long
164–172 (2013). as you give appropriate credit to the original author(s) and the source,
74. Schmädeke, S. & Bischoff, C. Wirkungen smartphonegestützter provide a link to the Creative Commons licence, and indicate if changes
Psychosomatischer Rehabilitationsnachsorge (eATROS) Bei were made. The images or other third party material in this article are
depressiven Patienten. Verhaltenstherapie 25, 277–286 (2015). included in the article’s Creative Commons licence, unless indicated
75. Willems, R. A. et al. Short‐term effectiveness of a web‐based tailored otherwise in a credit line to the material. If material is not included in the
intervention for cancer survivors on quality of life, anxiety, depression, and article’s Creative Commons licence and your intended use is not permitted
fatigue: Randomized controlled trial. Psycho-Oncol. 26, 222–230 by statutory regulation or exceeds the permitted use, you will need to
(2016). obtain permission directly from the copyright holder. To view a copy of this
76. Zwerenz, R. et al. Evaluation of a transdiagnostic psychodynamic licence, visit http://creativecommons.org/licenses/by/4.0/.
online intervention to support return to work: A randomized controlled
trial. PLOS ONE 12, e0176513 (2017). © The Author(s) 2024

npj Digital Medicine | (2024)7:253 9