Journal of Psychiatric Research 103 (2018) 91–96

Contents lists available at ScienceDirect

Journal of Psychiatric Research

journal homepage: www.elsevier.com/locate/jpsychires

Screening of adjustment disorder: Scale based on the ICD-11 and the T

Adjustment Disorder New Module
Menachem Ben-Ezraa,∗, Michal Mahat-Shamira, Louisa Lorenzb, Osnat Lavendaa,
Andreas Maerckerb
a
School of Social Work, Ariel University, Ariel 40700, Israel
b
Department of Psychology, Psychopathology and Clinical Intervention, University of Zürich, Zurich, Switzerland

A R T I C LE I N FO A B S T R A C T

Keywords: In line with ICD-11 new conceptualization of Adjustment disorder (AjD), a self-report Adjustment Disorder–New
Adjustment disorder Module (ADNM) was developed and validated. Nevertheless, the ADNM-20 is a long research tool and poten-
ICD-11 tially problematic in the use in epidemiological and clinical studies. The present study introduces the brief
Screening ADNM-8 and the ultra-brief ADNM-4, examines their validity and establishes cut-off scores for their clinical use.
Stress-related disorders
The study used a representative national sample of 1003 Israelis who reported on the ICD-11 stress spectrum
ranging from AjD, PTSD, complex PTSD and complicated grief. Construct validity was assessed via confirmatory
factor analysis and cut-off scores were established through ROC analysis. The original and brief instruments were
highly correlated (r > 0.918 or better). Cronbach's Alpha for the Brief ADNM-8 and the Ultra-Brief ADNM-4
were above 0.800. Correlations with stress related conditions indicated a good convergent and construct validity
for both instruments as well. The ultra-brief ADNM-4 was found to have a very good fit with the data.
These findings indicate that the brief ADNM-8 and the ultra-brief ADNM-4 can serve as a brief screening tools
for assessing AjD symptoms according to the ICD-11 definition.

1. Introduction reaction to the stressor is characterized by symptoms of preoccupation

with the stressor, such as excessive worry, recurrent and distressing
Up until the diagnosis of adjustment disorder (AjD) was established, thoughts about the stressor or constant rumination about its implica-
in the absence of a specific diagnostic criteria was subject to significant tions. There is a failure to adapt, which is manifested in symptoms in-
controversy was associated with it, in spite of its frequent use by clin- terfering with everyday functioning, such as difficulties concentrating
icians (Evans et al., 2013; Reed et al., 2011). AjD was one of the most or sleep disturbances. The symptoms can also be associated with loss of
ill-defined mental disorders, often described as the “wastebasket” of the interest in work, social life, caring for others, leisure activities resulting
psychiatric classification scheme (Casey and Bailey, 2011; Strain and in impairment in social or occupational functioning (restriction of social
Diefenbacher, 2008). network, conflicts in family, absenteeism and so on). However, if the
The revision of the International Classification of Diseases and definitional requirements are met for another disorder, then that dis-
Related Health Problems (ICD-11) has provided an opportunity for the order should be diagnosed instead of AjD (Maercker et al., 2013).
World Health Organization (WHO) to revisit these issues and devise a In line with this new conceptualization of AjD, Einsle, Kollner,
classification whose aim is to improve clinical utility and global ap- Dannemann, and Maercker (Einsle et al., 2010) developed and initially
plicability (First et al., 2015; International Advisory Group for the validated a self-report assessment, the Adjustment Disorder–New
Revision of ICD 10 Mental and Behavioral Disorders, 2011; Reed, Module (ADNM). There are two established versions of the ADNM: the
2010). original 29-item version, and a 20-item short version as well as the
According to ICD-11, AjD is a maladaptive reaction to a stressful validation of an 8-item brief version (consists of only two subscales of
event, to ongoing psychosocial difficulties or to a combination of the ADNM-20). Several validation studies of the former two versions
stressful life situations that usually emerges within a month of the oc- indicated satisfying psychometric properties for the long versions
currence of a stressor and tends to resolve within 6 months, unless the (Einsle et al., 2010; Glaesmer et al., 2015; Lorenz et al., 2016) and one
stressor persists for a longer duration (Maercker et al., 2013). The study for the short version (Kazlauskas et al., 2018).

∗
Corresponding author.
E-mail address: menbe@ariel.ac.il (M. Ben-Ezra).

https://doi.org/10.1016/j.jpsychires.2018.05.011
Received 16 March 2018; Received in revised form 8 May 2018; Accepted 12 May 2018
0022-3956/ © 2018 Elsevier Ltd. All rights reserved.
M. Ben-Ezra et al. Journal of Psychiatric Research 103 (2018) 91–96

Nevertheless, the ADNM-20 is a long research tool and therefore Participants were asked to report all stressors that occurred during the
potentially problematic for use in epidemiological and clinical studies, last two years (Einsle et al., 2010; Lorenz et al., 2016), disregarding the
which often require the application of several research tools at the same amount of subjective distress each event caused. Subsequently, the
time. Thus, there remains a need for a fast, structured assessment of AjD participants were asked to specify which three events caused them the
symptoms (Casey, 2014). In fact, the importance of a shorter version of most subjective distress. For the following calculations, the stressor
the ADNM is in line with the recent trend towards shorter instruments mentioned first was considered to be the most distressing event. For
(Rammstedt and Rammsayer, 2002). Examples of this trend toward acute stressors, participants indicated the date of the event (year/
minimal measurements are the single-item self-esteem scale (Robins month). For chronic stressors participants indicated the beginning and
et al., 2001), single-item ability ratings (Rammstedt and Rammsayer, ending date of the event (month/year).
2002), and even a 10-item measure of the Big Five (Gosling et al., Adjustment Disorder symptom criteria consisted of 19 items asses-
2003). As Burisch (1997) predicted, many of these super-short instru- sing the different symptoms of adjustment disorder in accordance with
ments show respectable psychometric characteristics. In addition, the Adjustment Disorder ICD-11 proposal. Additionally, one item as-
shorter versions of instruments reveal core elements in the diagnosis of sessing the criterion of functional impairment was included: “The
the phenomena. Defining the core elements will allow to eliminate il- symptoms cause clinically significant impairment in social, occupa-
lusory comorbidity of psychiatric conditions and at the same time will tional, or other important areas of functioning” (possible responses
eliminate the need to relate to cultural differences in the diagnosis. The were 1(none) – 4 (most)). Moreover, participants indicated the amount
need to address this issue in diagnosing psychiatric conditions is a of time passed since the symptom has occurred (0–1 month, 1–6
known concern in the professional literature (Maercker et al., 2013). months, 6–24 months). Symptom criteria endorsement and onset of
Recent findings indicating that the ADNM-20 items represent a uni- symptoms were assessed for the most distressing Adjustment Disorder
dimensional underlying latent construct suggest that a shorter version life event specified before. Two core symptom clusters (“preoccupa-
of the ADNM may be feasible (Glaesmer et al., 2015; Lorenz et al., tions” composed of 4 items and “failure to adapt” composed of 4 items,
2017). including the impairment item) and four associated feature clusters
Based on the ICD-11 definition of AjD and the ADNM-20, and while (“avoidance” composed of 4 items; “depression” composed of 3 items;
looking broadly at the stress-syndromes spectrum, the current study “anxiety” composed of 2 items; and “impulsivity” composed of 3 items;
aims at: a) further establishing the ICD-11 stress syndromes spectrum, see Table 1) were assessed. Participants indicated the frequency of all
b) presenting, validating and replicating findings regarding the brief symptoms on a 4-point Likert scale (1 = never, 2 = rarely, 3 = some-
ADNM-8 and c) presenting the ultra-brief ADNM-4. In particular, the times, 4 = often).
present study examines the short versions’ psychometric properties as Previous study showed the ADNM-20 has good psychometric
screening tools and establishing cut off scores for potential clinical use. properties (α = 0.94) for the total scale and for the different subscales
(α = 0.80–0.90) (Lorenz et al., 2016). The internal consistency, of the
2. Methods ADNM-20 used in the present sample for diagnostic purposes was sa-
tisfactory (α = 0.95).
2.1. Participants and procedure The ADNM-20 items are also used for identification of people with
high risk for AjD diagnosis. This is based on the following algorithm
The study sample was composed of 1003 Israelis obtained via an that meets the ICD-11 Adjustment Disorder criteria. The algorithm is
internet panel of over 130,000 Israelis. In order to maintain its re- based on the core symptoms of AjD. High risk is defined as one item
presentativeness, the panel undergoes dynamic changes according to rated ≥3 and at least two items rated ≥2 in both core symptom
changes in the Israeli census, based on data from the Israel Bureau of clusters and a rating ≥3 on the impairment criterion (item 20: “All in
Statistics (Bodas et al., 2017). The sample was drawn from the panel all, the situation causes serious impairment in my social life or occu-
using stratified and random sampling methods in order to obtain a pational life, my leisure activities or other important areas of func-
sample that is a close approximation to the general Israeli population. tioning”) (Lorenz et al., 2016).
Following the approval of Ethic committee of the researchers’ uni-
versity, potential participants were invited to participate in the study 2.2.2. Post-Traumatic Stress Disorder (PTSD) and complex PTSD
via email. Each participant signed an electronic informed consent be- Post-Traumatic Stress Disorder (PTSD) and Complex PTSD were
fore accessing the questionnaire. Eligibility for participating in the measure by the ICD-11 Trauma Questionnaire (ICD-TQ; Version 1.2)
study necessitated Israeli identity, an age of at least 18 year old and (Cloitre et al., 2014). The ICD-TQ is a 22-item self-report measure for
fluent Hebrew. The mean age of the sample was 40.6 years (SD = 14.5; the screening of ICD-11 PTSD and CPTSD symptomatology. Six items
range 18–70) with a slightly higher representation for women (51.7%). represent the three clusters of PTSD including Re-experiencing (RE)
All the participants were born in Israel and most of them (82.3%) re- (items P1-P2), Avoidance (AV) (items P3-P4), and Sense of Threat (Th)
ported living in urban areas. Most of the participants (82.7%) reported that is manifested by increased arousal and hypervigilance (items P5-
being employed either full time (61.8%) or part time (20.9%), and P6). CPTSD includes PTSD as well as three clusters reflecting `Dis-
68.4% reported having a college/university degree. About two thirds turbances in Self-Organization’ (DSO). Sixteen items represent the three
(70.5%) reported being in a committed relationship. The average DSO clusters including affective dysregulation (AD, items C1-C9), ne-
number of children was 1.8 (SD = 1.7; range 0–11). gative self-concept (NSC, items C10-C13), and disturbances in re-
lationships (DR, items C14-C16). Symptom endorsement is scored on a
2.2. Measurements Likert scale, indicating how much a symptom has been bothersome in
the past month, with scores ranging from 0 (not at all) to 4 (extremely).
2.2.1. Adjustment disorder The scale can be used to estimate a self-report ICD-11 PTSD or CPTSD
Adjustment disorder was measured by the Adjustment Disorder New diagnosis by recoding the Likert scores into six binary variables re-
Module (ADNM-20) (Einsle et al., 2010; Lorenz et al., 2016). The presenting each of the 3 PTSD and DSO symptom clusters based on the
module includes two components: a) Adjustment Disorder stressor list following cut-off scores. A score of ≥2 is considered presence of a
and b) Adjustment Disorder symptom criteria. The stressor list is symptom. A diagnosis of PTSD requires a score of ≥2 for at least one
composed of seven types of acute events (e.g. divorce, moving) and nine symptom in each of its three clusters. A diagnosis of CPTSD requires
types of chronic stressors (e.g. conflict with neighbors, serious illness). PTSD and the following scores for each of the three DSO clusters. The
Additionally, there are three open-ended questions regarding events proposed algorithm for each DSO cluster requires a sum that is half of
that are not included in the list (classified as “Other stressors”). the total possible score. AD requires a score ≥10 on items 1–5 (hyper-

92
M. Ben-Ezra et al. Journal of Psychiatric Research 103 (2018) 91–96

activation) or a score of ≥8 on items 6–9 (deactivation). For the 4 NSC (“I have to think about the stressful situation repeatedly” and “My
items, a score ≥8, and for the 3 DR items a score ≥6 are required. thoughts often revolve around anything related to the stressful situa-
Cronbach's alpha reliability estimates for the PTSD indicators were tion”) were discarded as they are not AjD-specific and can be evoked as
good (RE = 0.80, AV = 0.87, and Th = 0.86), but higher for the DSO a result of any other stress related condition. The other two items – 4 &
indicators (AD = 0.88, NSC = 0.93, and DR = 0.91). The internal 13 (“I have to think about the stressful situation a lot and this is a great
consistency of the six PTSD items used in the present study for diag- burden to me” and “I constantly get memories of the stressful situation
nostic purposes was satisfactory (α = 0.89), as well as of the DSO items and can't do anything to stop them”) emphasize the fundamental dif-
(α = 0.94). The ICD-TQ was previously used in several studies (Hyland ficulty to adjust, as a result of a stressful event, very clearly and ex-
et al., 2017; Karatzias et al., 2016, 2017). plicitly. In regard to Failure to adapt, items 19 & 20 (“Since the stressful
situation, I can no longer sleep properly” and “All in all, the situation
2.2.3. ICD-11 prolonged grief disorder (ICD-11) causes serious impairment in my social or occupational life, my leisure
ICD-11 prolonged grief disorder (ICD-11) was measured by the time, and other important areas of functioning”) are also very general
adapted Inventory of Complicated Grief- Revised (ICG-Revised) ICG-R and meet other stress-related conditions' symptoms and therefore were
items for ICD-11 PGD (Maciejewski et al., 2016). Based on prior works eliminated from the Ultra-brief instrument, while items 10 & 17 (“Since
(Prigerson et al., 1999) a symptom was considered present if rated “4” the stressful situation, I find it difficult to concentrate on certain things”
or “5”, and absent if rated “1”, “2” or “3” on its five-point scale. Taken and “Since the stressful situation, I do not like going to work or carrying
from previous work (Maciejewski et al., 2016), the “ICD-11 version” of out the necessary tasks in everyday life”) depict the difficulty to adjust
the Prolonged Grief Disorder (PGD) symptom-diagnostic test was con- and adapt. These items were selected as they are stress specific on the
structed based on a narrative proposal for the diagnostic assessment of stress syndromes spectrum in the ICD-11. Moreover, the concentration
PGD for ICD-11(8). This narrative proposal includes seven items that item is not a part of PTSD/CPTSD diagnosis contrary to the DSM-5 and
are represented directly in the ICG-R and that have been found to be it is specific and stressor related in comparison to other psychiatric
informative and unbiased in the empirical evaluation of items presented conditions that are depicted by poor concentration (a global aspect).
by Prigerson et al. (Prigerson et al., 2009). The internal consistency of Therefore, the items selected for the ultra-brief ADNM-4 were 4, 13, 10
the seven ICG-R items used in the present study for diagnostic purposes & 17.
was satisfactory (α = 0.83). Furthermore, an additional item that
measures functional impairment was used: “I believe that my grief has 2.3.2. Validation of the short instruments
resulted in impairment in my social, occupational or other areas of The structure of the brief ADNM-8 and the ultra-brief ADNM-4 were
functioning”. Responses ranged from ‘No functional impairment’ (1) to tested using confirmatory factor analysis (CFA) based on responses to
‘Completely functionally impaired’ (5). Adding the eighth item did not the full pool of 20 items.
changed the internal consistency of the ICG-R to (α = 0.83). The di- Four models were tested with various combinations of items, based
agnostic algorithm for the ICD-11 prolonged grief disorder includes on the number of items composing each, as described in the previous
seven items (two category A [ICG-R1 and ICG-R2] and five category B section. First, two models depict the brief tool: the first model was a
[ICG-R3 to ICG-R7]). Category A items capture the essence of the two-factor model comprised of core symptoms alone: preoccupation
syndrome and category B include items that collectively capture the and failure to adapt. This model included seven items as the eighth item
severity of the syndrome. Based on a previous study (Prigerson et al., measures functional impairment and therefore was discarded from
2009), a positive “ICD-11” test indicates endorsement of at least one of analysis (Kazlauskas et al., 2018; Lorenz et al., 2017). The second
two category A items and at least three of five category B items model was a one factor model depicting a general factor of core
(Maciejewski et al., 2016). symptoms including the seven items of Model 1. The third and the
fourth models examined the ultra-brief instrument, including the four
2.2.4. World Health Organization Well-Being Index items selected from the items of the brief ADNM-8. The third model was
Psychological wellbeing was measured using the World Health a two-factor model comprised of two items from each one of the core
Organization Well-Being Index (WHO-5) (World Health Organization, symptoms clusters: preoccupations presented by items 4 & 13; Failure
1998). The WHO-5 is a widely used, internationally-validated measure to adapt represented by items 10 & 17 from the ADNM-20, and finally,
of positive mental health. A recent review of 213 international studies the fourth model was a one factor model based on the four items pre-
supported the reliability and validity of the scale (Topp et al., 2015). sented in the third model. The CFA was conducted using AMOS 23.
Respondents are asked to indicate how they have been feeling over the Overall goodness of fit for each sample was assessed by a variety of
past two weeks, in response to each positively-phrased statement along indices: goodness of fit index (GFI, recommended result should be
a six-point Likert scale ranging from ‘At no time’ (0) to ‘All of the time’ higher than 0.90); adjusted goodness of fit index (AGFI, recommended
(5). Scores range from 0 to 25 with higher scores reflecting greater result should be higher than 0.85); normed fit index (NFI, re-
psychological wellbeing. Scores ≤13 are indicative of poor mental commended result should be higher than 0.90); Tucker–Lewis index
health and the possible presence of a psychiatric disorder (Awata et al., (TLI, recommended result should be higher than 0.95); comparative fit
2007). The internal consistency of the WHO-5 in the current sample index (CFI, recommended result should be higher than 0.95); incre-
was satisfactory (α = 0.93). mental fit index (IFI, recommended result should be higher than 0.95);
the root mean square error of approximation (RMSEA, recommended
2.3. Statistical analysis result should be lower than 0.08), Akaike Information Criterion (AIC)
and the Bayesian Information Criterion (BIC) were also obtained (for
2.3.1. Selection of items for the brief and ultra-brief instruments more information about the goodness of fit cutoffs, see: Bentler, 1990;
The brief ADNM-8 is composed of the eight items measuring core Burke et al., 1989; Hu and Bentler, 1995; Hu and Bentler, 1999; Marsh
AjD symptoms (ADNM-8): four items for measuring preoccupation, et al., 1988).
three items for measuring failure to adapt and 1 item measuring func- The correlations between the sum of score of the Brief ADNM-8 (8
tion impairment (see Appendix 1). This short instrument was developed items; models 1 & 2) and Ultra brief ADNM-4 (4 items; models 3 & 4)
and validated in a recent study conducted by the developers of the were examined.
original ADNM measure (Kazlauskas et al., 2018). Among the items In addition, construct and convergent validation were conducted by
composing each symptom, two items were identified statistically and correlation coefficients that were calculated between the various ADNM
content-wise as more AjD-specific that better fit the meaning of ad- instruments and PTSD, CPTSD and complicated Grief. These stress re-
justment disorder. Among the four items of preoccupation, items 2 & 15 lated conditions are known to be related to AjD (Maercker et al., 2008).

93
M. Ben-Ezra et al. Journal of Psychiatric Research 103 (2018) 91–96

Finally, in order to examine the reliability of the various ADNM PTSD. Both alternative models were positively and significantly asso-
instruments measuring Adjustment Disorder, internal consistency was ciated with stress related conditions (i.e. prolonged Grief disorder,
calculated for each instrument. PTSD, and CPTSD), and negatively and significantly associated with
psychological well-being. As indicated in Table 3, the correlations be-
2.3.3. Divergent validity of the ADNM-20, ADNM-8 and ADNM-4 tween the different models and PTSD were of a medium size
The WHO-5 scale was used as a different and unrelated construct in (0.362 < r < 0.387), whereas the associations with CPTSD
order to conduct divergent validity. The association between the ADNM (0.218 < r < 0.233) and with prolonged grief disorder
scales and the WHO-5 was examined. (0.123 < r < 0.127) were small to medium. The negative correlations
between the ADNM scales andpsychological wellbeing were of a
2.3.4. Establishing cut off points for clinical use medium effect size (−0.293 < r < −0.313).
In order to establish cut off points with potential clinical usefulness, Finally, ROC Analyses were conducted for each short instrument to
Receiver Operating Characteristic (ROC) analysis was conducted, in propose potential cut-off scores for clinical use. In the present sample,
which the state variable was the binary option for each endorsement 17.4% of the participants have met the criteria for a tentative diagnosis
(0 = not meeting ICD-11 AjD criteria vs. 1 = meeting ICD-11 AjD cri- of ICD-11 AjD as described before.
teria). The test variable was the sum of scores of the ADNM-8 and the The ROC analysis that tested the ADNM-8 sum score against the
ADNM-4 scales. Next, Youden's index was obtained in order to learn theoretical algorithm for AjD diagnosis in the ICD-11 indicated an ex-
what the maximal Youden value is for the proposed ADNM-8 and cellent accuracy of diagnosis (AUC = 0.972; 95% C.I. 0.964–0.981;
ADNM-4 cut-off scores. Finally, ADNM-8 and ADNM-4 included ana- P < 0.001). The Youden's index reached its maximum at a cut-off
lysis of sensitivity, specificity, positive predictive value, negative pre- value of 18.5 indicating that the diagnosis of AjD is very likely in every
dictive value and accuracy. person scoring 18.5 or higher on the ADNM-8 scale. The ADNM-8
sensitivity was 97.14% (95% C.I 93.46%–99.07%), specificity was
86.23% (95% C.I 83.70%–88.51%), positive predictive value was
3. Results
59.86% (95% C.I 55.66%–63.92%), negative predictive value was
99.30% (95% C.I 98.37%–99.71%), and accuracy was 88.14% (95% C.I
The results of the Confirmatory Factor Analysis are presented in
85.97%–90.07%). Hence, the recommended ADNM-8 cutoff is 18.5 or
Table 1. For the ADNM-8, Model 1, the model with 7 items representing
higher for AjD diagnosis.
two factors (preoccupation and failure to adapt), was found to have
The ROC analysis that tested the ADNM-4 sum score against the
good fit with the data across the majority of indices. For the ultra-brief
theoretical algorithm for AjD diagnosis in the ICD-11 indicated an ex-
ADNM-4, Model 3, the two-factor model for two symptoms from each
cellent accuracy of diagnosis (AUC = 0.944; 95% C.I. 0.929–0.959;
one of the core symptoms cluster was found to have a very good fit with
P < 0.001). The Youden's index reached its maximum at a cut-off
the data. For more information, see the supplemental file.
value of 8.5 indicating that the diagnosis of AjD is very likely in every
The calculation of Cronbach's alpha measures indicated a high in-
person scoring 8.5 or higher on the ADNM-4 scale. The ADNM-4 sen-
ternal consistency for both instruments. Cronbach's Alpha, for the Brief
sitivity was 94.86% (95% C.I 90.46%–97.62%), specificity was 79.23%
ADNM-8 was 0.91 and for the Ultra Brief ADNM-4 it was 0.81. In ad-
(95% C.I 76.30%–81.94%), positive predictive value was 49.11% (95%
dition to the internal consistency of the alternative instruments, cor-
C.I 45.69%–52.55%), negative predictive value was 98.65% (95% C.I
relations between the various versions of the measure were tested, i.e.
97.47%–99.28%), and accuracy was 81.95% (95% C.I
ADNM-20, ADNM-20 without the functional impairment item, ADNM-
79.43%–84.29%). Hence, the recommended ADNM-4 cutoff is 8.5 or
8, ADNM-8 without the functional impairment item and ADNM-4. As
higher for AjD diagnosis.
indicated in Table 2, the three versions of instrument are highly cor-
In sum, negative predictive value and accuracy are important as
related (r > 0.928).
they show the ADNM-8 and ADNM-4 will most likely determinate if a
Table 3 summarizes the correlations found between the alternative
participant is lacking AjD diagnosis. However, a positive answer will be
models and stress related psychiatric conditions, i.e. ICD-11 prolonged
best suited for a preliminary and cautionary diagnosis of AjD along with
grief, WHO-5 well-being index, ICD-11 PTSD, and ICD-11 Complex
further clinical inquiry.
Table 1
Fit statistics for the alternative models of Adjustment Disorder Symptoms 4. Discussion
(n = 1003).
The results revealed that both the brief ADNM-8 and the ultra-brief
Fit indices Recommended Value ADNM-8 ADNM-4
ADNM-4 have good psychometric properties. The shared variance (R2)
Model 1 Model 2 Model 3 Model 4 based on the correlations in Table 2 between the ADNM-8 and the
ADNM-20 was 91%–92%, and between the ADNM-4 and the ADNM-20
Maximum Likelihood (standardized regression weight) was 84%–85%. Consequently, these substitutes can be used instead of
Χ2 N/A 214.654 329.394 0.998 29.152
the ADNM-20 as they largely target the same construct of adjustment
Df N/A 13 15 1 2
Χ2/df ≤3.00 16.512 21.960 0.998 14.576 disorders. Meaning, the brief ADNM-8 and the ultra-brief ADNM-4 can
GFI ≥.90 0.936 0.907 1.000 0.985 serve as a brief screening for assessing AjD symptoms according to the
AGFI ≥.85 0.867 0.827 1.000 0.925 ICD-11 definition (Casey et al., 2001; Maercker et al., 2013; Maercker
NFI ≥.90 0.940 0.908 0.999 0.977 et al., 2007). According to the study's findings, the recommended
TLI ≥.95 0.909 0.877 1.000 0.936
CFI ≥.95 0.944 0.912 1.000 0.979
ADNM-8 cutoff is 18.5 or higher for AjD diagnosis while the re-
IFI ≥.95 0.944 0.912 1.000 0.979 commended ADNM-4 cutoff is 8.5 or higher for AjD diagnosis.
RMSEA ≤.08 0.112 0.145 0.000 0.116 The findings, showing the psychometric characteristics for both
AIC 244.654 355.394 18.998 45.152 short versions of the instrument (ADNM-8, ADNM-4), support their
BIC 318.315 419.234 63.195 84.438
validity and imply that these short versions may be used for assessing
Note. ML = Maximum likelihood; GFI = goodness-of-fit index; AGFI = adjusted the general population, and may also be useful in times when re-
goodness-of-fit index; NFI = normed fit index; TLI = Tucker–Lewis index; searchers face limited assessment time or research settings, in which
CFI = comparative fit index; IFI = incremental fit index; RMSEA = root-mean- participant time is limited.
square error of approximation; AIC = Akaike Information Criterion; BIC = Moreover, our findings demonstrate that the brief ADNM-8 and the
Bayesian Information Criterion. ultra-brief ADNM-4 subscales (preoccupation and failure to adapt)

94
M. Ben-Ezra et al. Journal of Psychiatric Research 103 (2018) 91–96

Table 2
Correlations between the ADNM versions.
ADNM-20 ADNM-20 without Function impairment ADNM-8 ADNM-8 ADNM-4
without Function impairment

ADNM-20 0.999*** 0.952*** 0.947*** 0.928***

ADNM-20 without Function impairment 0.945*** 0.945*** 0.926***
ADNM-8 0.994*** 0.963***
ADNM-8 without Function impairment 0.968***
ADNM-4

***Correlation is significant at the 0.001 level (2-tailed).

Table 3 conclusions regarding the test-retest reliability of the brief and ultra-
Correlation between the ADNM and ICD-PTSD/CPTSD/Complicated Grief/ brief ADNM-4. Future studies should strive to collect data using these
WHO-5/Psychological distress/Death anxiety. short measures over time. In order to examine the distinguishing power
ICD-11 Complicated WHO-5 ICD 11 ICD 11 of the Brief- and Ultra-brief measures one must use them with targeted
Grief Disorder PTSD CPTSD population, such as sample experiencing PTSD versus sample experi-
encing Adjustment Disorder. Future studies should strive to collect data
ADNM-20 .124*** -.313*** .387*** .223***
with these brief measures from various populations.
ADNM-20 without .124*** -.307*** .384*** .219***
Function Despite these limitations, this study showed evidence for the diag-
impairment nostic utility of the brief ADNM-8 and the ultra-brief ADNM-4 in the
ADNM-8 .123*** -.308*** .370*** .233*** assessment of AjD symptoms. Based on the present results, both short
ADNM-8 without .123*** -.293*** .362*** .224*** instruments can be recommended in research and clinical practice to
Function
impairment
evaluate a person's current state of AjD symptomatology. There are
ADNM-4 .127*** -.298*** .368*** .218*** several important implications for the present findings: First, the use of
short instruments in clinical and research settings will allow for the use
***Correlation is significant at the 0.001 level (2-tailed). of other clinical tools in conjunction, reducing the overload and ex-
haustion of the participants. Second, using a short instrument with high
identify subgroups at risk for AjD and potentially other conditions. negative predictive value will allow to differentiate those who don't
These results demonstrate the diagnostic validity of the brief ADNM-8 meet the ICD-11 AjD diagnostic criteria from those who are more likely
and the ultra-brief ADNM-4. These findings replicate previous studies’ to do so. Third, short instruments will easily allow to examine cross
findings addressing the core criteria for AjD, i.e. preoccupation and cultural differences in AjD diagnosis and reduce the possibility of
failure to adapt, as conceptually different from other psychiatric con- translation error that may lead to “loss in translation”. Finally, a concise
ditions such as major depression, generalized anxiety and others measure will improve practitioners' ability to assess the improvement of
(Maercker et al., 2013; Lorenz et al., 2016, 2017). This distinction was symptoms across time in comparison to full scales that might increase
not addressed in previous versions of diagnostic manuals such as DSM-5 the likelihood of refusal.
(American Psychiatric Association, 2013) or ICD-10 (World Health
Organization, 1998). These distinct core criteria are reflected in the
Conflicts of interest
short versions of the instrument allowing for a fast yet more accurate
screening of the disorder.
We declare no conflict of interest or otherwise.
Although the present study's findings are of great value for re-
searchers and practitioners, its limitations should not be overlooked.
First, although the sample was close to a representative one, it does not Acknowledgment
represent those who are older than 70 and those who don't have a
computer at home. Moreover, the study described in the present The study was funded by an internal research grant awarded to
manuscript is cross-sectional, which prevents us from drawing Professor Ben-Ezra from Ariel University RA1700000037.

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jpsychires.2018.05.011.

Appendix 1

The Adjustment Disorder – New Module (Brief ADNM-8)

Please think back upon the most stressful event in last year and answer the following questions in an honest and sincere way. The events can have
numerous consequences for our wellbeing and behavior. Below you will find various statements about which reactions these types of event can
trigger. Please indicate how often the respective statement applies to you (“never” to “often”).

Never Rarely Sometimes Often

(1) (2) (3) (4)

1. I have to think about the stressful situation repeatedly. ☐ ☐ ☐ ☐

2. I have to think about the stressful situation a lot and this is a great burden to me. ☐ ☐ ☐ ☐
3. Since the stressful situation, I find it difficult to concentrate on certain things. ☐ ☐ ☐ ☐
4. I constantly reminded of the stressful situation and can't do anything to stop them. ☐ ☐ ☐ ☐

95
M. Ben-Ezra et al. Journal of Psychiatric Research 103 (2018) 91–96

5. My thoughts often revolve around anything related to the stressful situation. ☐ ☐ ☐ ☐

6. Since the stressful situation, I do not like going to work or carrying out the necessary tasks in everyday ☐ ☐ ☐ ☐
life.
7. Since the stressful situation, I can no longer sleep properly. ☐ ☐ ☐ ☐
8. All in all, the situation causes serious impairment in my social or occupational life, my leisure time, and ☐ ☐ ☐ ☐
other important areas of functioning.
Scoring: sum all the marking for each statement (ranged from 8 to 32). A score of 18.5 or higher is indicative of Adjustment Disorder.

The Adjustment Disorder – New Module (Ultra Brief ADNM-4)

Please think back upon the most stressful event in last year and answer the following questions in an honest and sincere way. The events can have
numerous consequences for our wellbeing and behavior. Below you will find various statements about which reactions these types of event can
trigger. Please indicate how often the respective statement applies to you (“never” to “often”).

Never Rarely Sometimes Often

(1) (2) (3) (4)

1. I have to think about the stressful situation a lot and this is a great burden to me. ☐ ☐ ☐ ☐
2. Since the stressful situation, I find it difficult to concentrate on certain things. ☐ ☐ ☐ ☐
3. I constantly reminded of the stressful situation and can't do anything to stop them. ☐ ☐ ☐ ☐
4. Since the stressful situation, I do not like going to work or carrying out the necessary tasks in ☐ ☐ ☐ ☐
everyday life.
Scoring: sum all the markings for each statement (ranged from 4 to 16). A score of 8.5 or higher is indicative of Adjustment Disorder.

References An initial psychometric assessment of an ICD-11 based measure of PTSD and complex
PTSD (ICD-TQ): evidence of construct validity. J. Anxiety Disord. 44, 73–79.
Karatzias, T., Shevlin, M., Fyvie, C., Hyland, P., Efthymiadou, E., Wilson, D., et al., 2017.
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Evidence of distinct profiles of posttraumatic stress disorder (PTSD) and complex
Disorders, fifth ed. American Psychiatric Publishing. posttraumatic stress disorder (CPTSD) based on the new ICD-11 trauma questionnaire
Awata, S., Bech, P., Koizumi, Y., Seki, T., Kuriyama, S., Hozawa, A., et al., 2007. Validity (ICD-TQ). J affect disorders 207, 181–187.
and utility of the Japanese version of the WHO-Five Well-Being Index in the context Kazlauskas, E., Gegieckaite, G., Eimontas, J., Zelviene, P., Maercker, A., 2018. A brief
of detecting suicidal ideation in elderly community residents. Int. Psychogeriatr. 19, measure of ICD-11 adjustment disorder: investigation of psychometric properties in
77–88. adult help-seeking sample. Psychopathology 51, 10–15.
Bentler, P.M., 1990. Comparative fit indexes in structural models. Psychol. Bull. 107, Lorenz, L., Bachem, R.C., Maercker, A., 2016. The Adjustment Disorder–New Module 20
238–246. https://doi.org/10.1037/0033-2909.107.2.238. as a screening instrument: cluster analysis and cut-off values. Int. J. Occup. Environ.
Bodas, M., Siman-Tov, M., Kreitler, S., Peleg, K., 2017. The role of victimization in Med. 7, 215–220.
shaping households preparedness for armed conflicts in Israel. Disaster Med. Public Lorenz, L., Hyland, P., Perkonigg, A., Maercker, A., 2017. Is adjustment disorder uni-
Health Prep. 27, 1–9. dimensional or multidimensional? Implications for ICD-11. Int. J. Meth. Psychiatr.
Burisch, M., 1997. Test length and validity revisited. Eur. J. Pers. 11, 303–315. Res. http://dx.doi.org/10.1002/mpr.1591. e1591.
Burke, M.J., Brief, A., Atieh, J., Roberson, L., Webster, J., 1989. Measuring affect at work: Maciejewski, P.K., Maercker, A., Boelen, P.A., Prigerson, H.G., 2016. “Prolonged grief
confirmatory analyses of competing mood structures with conceptual linkage to disorder” and “persistent complex bereavement disorder”, but not “complicated
cortical regulatory systems. J. Pers. Soc. Psychol. 57, 1091–1102. grief”, are one and the same diagnostic entity: an analysis of data from the Yale
Casey, P., 2014. Adjustment disorder: new developments. Curr. Psychiatr. Rep. 16, 1–8. Bereavement Study. World Psychiatr. 15 (3), 266–275.
Casey, P., Bailey, S., 2011. Adjustment Disorder: the state of the art. World Psychiatr. 10 Maercker, A., Brewin, C.R., Bryant, R.A., Cloitre, M., van Ommeren, M., Jones, L.M.,
(1), 11–18. et al., 2013. Diagnosis and classification of disorders specifically associated with
Casey, P., Dowick, C., Wilkinson, G., 2001. Adjustment disorders: fault line in the psy- stress: proposals for ICD-11. World Psychiatr. 12, 198–206.
chiatric glossary. Br. J. Psychiatry 179, 479–481. Maercker, A., Einsle, F., Kollner, V., 2007. Adjustment disorders as stress response syn-
Cloitre, M., Garvert, D.W., Weiss, B., Carlson, E.B., Bryant, R.A., 2014. Distinguishing dromes: a new diagnostic concept and its exploration in a medical sample.
PTSD, complex PTSD, and borderline personality disorder: a latent class analysis. Eur. Psychopathology 40, 135–146.
J. Psychotraumatol. 5, 25097. https://doi.org/10.3402/ejpt.v5.25097. Maercker, A., Forstmeier, S., Enzler, A., et al., 2008. Adjustment disorders, posttraumatic
Einsle, F., Kollner, V., Dannemann, S., Maercker, A., 2010. Development and validation of stress disorder, and depressive disorders in old age: findings from a community
a self-report for the assessment of adjustment disorders. Psychol. Health Med. 15 (5), survey. Compr. Psychiatr. 49, 113–120.
584–595. Marsh, H.W., Balla, J.R., McDonald, R.P., 1988. Goodness of fit indexes in confirmatory
Evans, S.C., Reed, G.M., Roberts, M.C., Esparza, P., Watts, A.D., Correia, J.M., et al., 2013. factor analysis: the effect of sample size. Psychol. Bull. 103, 391–410.
Psychologists' perspectives on the diagnostic classification of mental disorders: results Prigerson, H.G., Horowitz, M.J., Jacobs, S.C., Prigerson, H.G., Horowitz, M.J., Jacobs,
from the WHO-IUPsyS Global Survey. Int. J. Psychol. 48 (3), 177–193. S.C., et al., 2009. Prolonged grief disorder: psychometric validation of criteria pro-
First, M.B., Reed, G.M., Hyman, S.E., Saxena, S., 2015. The development of the ICD-11 posed for DSM-V and ICD-11. PLoS Med. 6 e1000121.
clinical descriptions and diagnostic guidelines for mental and behavioral disorders. Prigerson, H.G., Shear, M.K., Jacobs, S.C., Reynolds 3rd, C.F., Maciejewski, P.K.,
World Psychiatr. 14 (1), 82–90. Davidson, J.R., et al., 1999. Consensus criteria for traumatic grief: a preliminary
Glaesmer, H., Romppel, M., Brahler, E., Hinz, A., Maercker, A., 2015. Adjustment disorder empirical test. Br. J. Psychiatr. 174, 67–73.
as proposed for ICD-11: dimensionality and symptom differentiation. Psychiatr. Res. Rammstedt, B., Rammsayer, T., 2002. Gender differences in self-estimated intelligence
229, 940–948. and their relation to gender-role orientation. Eur. J. Pers. 16, 369–382.
Gosling, S.D., Rentfrow, P.J., Swann Jr., W.B., 2003. A very brief measure of the Big-Five Reed, G.M., Correia, J.M., Esparza, P., Saxena, S., Maj, M., 2011. The WPA-WHO global
personality domains. J. Res. Pers. 37, 504–528. survey of psychiatrists' attitudes towards mental disorders classification. World
Hu, Y.J., Bentler, P.M., 1995. Evaluating model fit. In: Hoyle, R.H. (Ed.), Structural Psychiatr. 10 (2), 118–131.
Equation Modeling. Concepts, Issues and Applications. Sage Publications, London, Reed, G.M., 2010. Toward ICD-11: improving the clinical utility of WHO's International
pp. 76–99. Classification of mental disorders. Prof Psychol - Res PR 41 (6), 457.
Hu, Y.J., Bentler, P.M., 1999. Cutoff criteria for fit indices in covariance structure ana- Robins, R.W., Hendin, H.M., Trzesniewski, K.H., 2001. Measuring global self-esteem:
lysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6 (1), 1–55. construct validation of a single-item measure and the Rosenberg self-esteem scale.
Hyland, P., Shevlin, M., Brewin, C.R., Cloitre, M., Downes, A.J., Jumbe, S., et al., 2017. Pers. Soc. Psychol. Bull. 27, 151–161.
Validation of post-traumatic stress disorder (PTSD) and complex PTSD using the Strain, J., Diefenbacher, A., 2008. The adjustment disorders: the conundrums of the di-
International Trauma Questionnaire. Acta Psychiatr. Scand. 136 (3), 313–322. agnoses. Compr. Psychiatr. 49, 121–130.
International Advisory Group for the Revision of ICD-10 Mental and Behavioural Topp, C.W., Østergaard, S.D., Søndergaard, S., Bech, P., 2015. The WHO-5 Well-Being
Disorders, 2011. A conceptual framework for the revision of the ICD-10 classification Index: a systematic review of the literature. Psychother. Psychosom. 84, 167–176.
of mental and behavioral disorders. World Psychiatr. 10 (2), 86–92. World Health Organization, 1998. WHO (Five) Well-being Index (1998 Version).
Karatzias, T., Shevlin, M., Fyvie, C., Hyland, P., Efthymiadou, E., Wilson, D., et al., 2016. Available at: www.who-5.org.

96