Psychiatry Research ] (]]]]) ]]]–]]] Contents lists available at SciVerse ScienceDirect Psychiatry Research journal homepage: www.elsevier.com/locate/psychres The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample Antonio Semerari a,n, Michele Cucchi b, Giancarlo Dimaggio a, Daniele Cavadini b, Antonino Carcione a, Vittoria Battelli b, Giuseppe Nicolo a, Roberto Pedone a,c, Tomaso Siccardi b, Stefania D0 Angerio a, Paolo Ronchi b, Cesare Maffei b, Enrico Smeraldi b a Terzo Centro di Psicoterapia Cognitiva—Scuola di Psicoterapia Cognitiva (SPC), Training School in Cognitive Psychotherapy, via Ravenna 9/c, 00161 Rome, Italy Department of Clinical Neurosciences, Scientific Institute and University Vita-Salute San Raffaele, Via Stamira D0 Ancona 20, 20127 Milan, Italy c Department of Psychology, University of Naples II, Italy b a r t i c l e i n f o abstract Article history: Received 7 June 2011 Received in revised form 25 June 2012 Accepted 14 July 2012 Background: Metacognition is a multi-facet psychological construct; deficits in metacognitive abilities are associated to low social functioning, low quality of life, psychopathology, and symptoms. The aim of this study was to describe and develop a valid and reliable interview for assessing metacognition. Methods: The semi-structured interview, based on the author’s theory model of the metacognition construct, is described. The Metacognition Assessment Interview (MAI) is an adaptation of the Metacognition Assessment Scale (MAS) and evaluates how the subject is interviewed used metacognition during his own real life experiences elicited by the interviewer. A user manual was developed to assist the interview and scoring procedure. Results: Exploratory factor analysis and confirmatory factor analysis revealed preliminary evidence of a two factor-hierarchical structure, with two lower-order scales, representing the two main theoretical domains of the metacognitive function, ‘‘the Self’’ and ‘‘the Other’’, and one single higher-order scale that we labelled metacognition. Contrary to the authors’ prediction the existence of the four distinct dimensions under the two domains was not confirmed. The MAI and its two domains demonstrated acceptable levels of inter-rater reliability and internal consistency. Conclusions: The MAI appears to be a promising instrument for assessing metacognition. Future psychometric validation steps and clinical directions are discussed. & 2012 Elsevier Ireland Ltd. All rights reserved. Keywords: Metacognition Metacognition assessment Interview Reliability Factor structure 1. Introduction Research investigating the metacognitive construct has advanced rapidly over the past decade. Metacognition, as formulated by Semerari (Semerari et al. 2003; Carcione et al. 2008), refers to a broad set of cognitive and affective skills which allow people to identify mental states, reasoning about them, and ascribing them to themselves and others. These skills allow us to recognize the reason why a person reacts psychologically according to some regularities and constructs personal meaning over their lifespan. Several authors refer to the same concept with different meanings. Wells (2000), for example, views metacognition as a set of beliefs about one’s own mental content that helps people to regulate their attentive processes and that, in some cases, could induce the maintenance of dysfunctional attentive processes such n Corresponding author. E-mail address: semerari@terzocentro.it (A. Semerari). as worry. There is a terminological confusion over metacognition and this is also due to the fact that mind-reading abilities have been traditionally investigated by researchers belonging to different theoretical backgrounds and research fields, each of them with their own lexicon (Flavell, 1976; Fonagy, 1991; BaronCohen, 1995; Frith and Happé, 1999; Frith and Frith, 2006). Metacognition, as considered by Semerari et al. (2003), partially overlaps with similar constructs such as theory of mind (ToM) (Baron-Cohen et al., 1985) and mentalization (Bateman and Fonagy, 2004; Allen et al., 2008), but with some differences. Compared to ToM, as described below, metacognition usually includes more complex mental functions. Additionally, metacognition also refers to emotional understanding, while ToM mainly focuses on cognitive attribution. Compared to the general definition of mentalization given by Bateman and Fonagy (2004), Semerari’s concept differs since it considers mind-reading to be a general ability created by different subfunctions that interact with each other and that can be selectively impaired. Dysfunctions in metacognition are associated with low social functioning, low quality of life, psychopathology and symptoms of several psychiatric and personality disorders, and 0165-1781/$ - see front matter & 2012 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.psychres.2012.07.015 Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015 2 A. Semerari et al. / Psychiatry Research ] (]]]]) ]]]–]]] seem to predict worst treatment response (Semerari et al., 2007; Lysaker et al., 2010a, 2010b; Carcione et al., 2011; Lysaker et al., 2011a; Ogrodniczuk et al., 2011). However, the lack of a reliable and valid instrument for measuring metacognition has limited progress in this field, especially for developing clinical applications. Assessments of psychological constructs such as social cognition mostly rely on self-report instruments, laboratory tasks and structured interviews, and each has its own pros and cons. Interviews have unique advantages such as avoiding bias in self-ratings, especially if one is called upon to use a skill such as self-reflection, which in itself is supposed to be compromised; they also ask the person to reason about mental states in the context of personally relevant matters, i.e. the ones in which it is most important to be able to fully and swiftly understand the mental states of oneself and others. Therefore it is likely that metacognition as measured from interviews provides more clinician-friendly information that can directly affect treatment protocols (Lysaker et al., 2010a, 2011b). The aim of the present study was to investigate the reliability, internal consistency and structure of a semi-structured interview for assessing metacognition: the Metacognition Assessment Interview (MAI). The MAI is an adaptation of the Metacognitive Assessment Scale (MAS) (Semerari et al., 2003; Carcione et al., 2008; Carcione et al., unpublished), a rating scale assessing the construct as manifest in individual verbalization in psychotherapy transcripts. The common theoretical framework which underlines both the MAS and the MAI is that metacognition is made of specific and relatively independent subfunctions, each of them likely to be selectively damaged in clinical populations. The MAS is therefore divided into three scales: understanding one’s own mind, understanding others’ minds and mastery. Each scale is further composed of different subfunctions. The subfunction scales of understanding one’s own mind and understanding others’ minds are included in the MAI and they will be described in detail in Section 2. Work on an interview to investigate the third scale, mastery, is still in progress. The MAS and its subscales demonstrated acceptable levels of factorial validity, inter-rater agreement, internal validity and test–retest stability (Semerari et al., 2003, 2005; Lysaker et al., 2005; Carcione et al., 2008; Dimaggio et al., 2009; Lysaker et al., 2010b, 2011a, 2011b) and were related to executive functions and treatment outcomes (Lysaker et al., 2005; Semerari et al., 2005; Carcione et al., 2008; Lysaker et al., 2008). Studies on patients with personality disorders showed different profiles of metacognitive impairments in patients with different diagnoses (Semerari et al., 2003, 2005, 2007; Dimaggio et al., 2009). Furthermore, specific function impairment correlated with different symptoms in patients with schizophrenia (Lysaker et al., 2005). However, the MAS does not allow metacognitive functions to be directly stimulated through specific questions. Therefore, when the therapist does not directly investigate one specific subfunction, perhaps during a psychotherapy session, it is impossible to define whether the lack of use of this subfunction is due to impairment of it or simply to a lack of use of it in that specific circumstance. However, the direct investigation of a specific subfunction through specific questions, along with the weak answers that patients might give, highlights more clearly specific impairments in the metacognitive domains. For this reason, we aimed to develop a new tool that is capable of directly measuring metacognition and is less time-consuming than the MAS: the MAI (Semerari et al., 2008). Our hypotheses were (a) that metacognition could be elicited by such an interview and reliably scored; (b) that metacognition has a two-factor structure, with two separate domains, one for understanding the mental states of the self and one for understanding the mental states of others. 2. Methods 2.1. Construct description The MAI refers to the description of emotions and cognitions, and assesses how people are able to identify their own and others’ recurrent patterns of thinking, feeling and dealing with social problems. The interview evaluates two main functional skill domains of metacognition, ‘the Self’ and ‘the Other’, each one composed of two dimensions: monitoring and integrating for the Self, differentiating and decentring for the Other. To identify the 16 basic facets of which the dimensions are composed (four facets for each dimension) the authors took into account the clinical literature that describes deficit in the ability to know and regulate mental states, theoretically based on the literature on mentalization and attachment theories (Fonagy, 1991; Main, 1991; Fonagy and Target, 1997, 2006; Allen et al., 2008), theory of mind (Baron-Cohen et al., 1985; Premack and Woodruff, 1978a, 1978b, 1978c; Wellman, 1990), metacognition (Wellman, 1990; Wells, 2000) and, more generally, metarepresentation (Frith, 1992; Sperber, 2000). The Self domain comprises the ‘monitoring’ and ‘integrating’ dimensions and describes the way in which a person has explicit access to his own mental states (cognitive and emotional) in relation to behaviour. ‘Monitoring’ (MON) is referred to as the ability to distinguish, recognize and define one’s own inner states (emotions and cognitions) and following behaviours during the ‘here and now’ of the described real-life episode. MON is composed of four facets: (a) the ability to recognize one’s own representations (thoughts and beliefs); (b) the ability to recognize and verbalize one’s own emotions; (c) the ability to establish relations among the separate components of a mental state; and (d) the ability to establish relations between the components of mental states and behaviour. MON evaluates how a subject explains his/her own behaviour in terms of causes and/or motivations. If there is a deficit, he/she is unable to discern the reasons for his/ her behaviour, and he/she cannot recognize or verbalize emotions or other mental states. Examples of questions stimulating MON are ‘What do you feel?’, ‘What do you think?’ and ‘What was your aim?’. ‘Integrating’ (INT) is the second dimension of Self domain and involves the ability to produce coherent descriptions of people’s mental processes and states over time. INT refers to the ability to reflect on mental states and contents, putting them in a logical order and ranking them by relevance. Using INT abilities the subject is able to understand the link between his/her own mental states and different behaviours in different situations, decoding his/her mental, functional and dysfunctional habits and forming a consistent account of how his/her mental life has changed over his/her lifespan. INT comprises four skills: (a) the ability to describe understandable and coherent links among thoughts, events, actions and behaviours; (b) the ability to describe transitions among different mental states and explain the reasons why; (c) the ability to form generalized representations of his/her mental functioning, taking into account continuity over time of patterns of thinking and feeling; and (d) the ability to reconstruct and describe to the interviewer one’s own mental functioning, providing enough information, without giving irrelevant and out-offocus details, and giving a sense of order and coherence to the discourse. Examples of questions stimulating INT are ‘So, you have found yourself reacting byy [the interviewer refers to the described behaviour], and feelingy [the interviewer refers to the mentioned emotion]. Does feeling/thinking and behaving like this happen frequently to you?’ ‘You might also react in a different way, with different emotions or thoughts, in circumstances like the one described. Can you remember a specific episode?’ The Other metacognition domain comprises the ‘differentiating’ and the ‘decentring’ dimensions. Differentiating (DIF) concerns the ability to recognize the representational nature of one’s own and other individuals’ thoughts, the ability to differentiate between classes of representations, such as imagination, evaluation and expectation, and to distinguish between representation and reality. Using DIF abilities, the interviewee is able to consider his/her own opinion as a hypothesis and not as a matter of fact; DIF abilities allow one to consider representations as mental phenomena, separate from but related to reality. Good DIF functioning makes people flexible in formulating opinions and points of view, and causes mental states to change depending on the communicative acts and on the availability of salient information. DIF comprises four skills: (a) the ability to consider one’s own representation of the world as subjective and questionable; (b) the ability to give plausible interpretations of events; (c) the ability to reflect on and evaluate events (as opposed to a tendency to act impulsively); and (d) the ability to distinguish between different modes of thoughts such as dreaming, fantasizing and imagining. Examples of questions stimulating DIF are ‘You said you have thought thaty [the interview refers to the reported episode]. In that moment, how did you subjectively believe it and how did you consider other options?’ ‘Did you take into consideration alternative interpretations of the events?’ ‘During the episode how much did you feel confused or clear-headed?’ Have you ever experienced such levels of confusion, or not been able to remember whether something really happened, or felt dreamy, unreal, like brain fog?’. ‘Decentring’ (DEC) refers to the ability to infer others’ mental states in a plausible manner and adopt their perspective, recognizing that it is distinct from Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015 A. Semerari et al. / Psychiatry Research ] (]]]]) ]]]–]]] our own. DEC leads to the realization that other people’s behaviours are understandable on the basis of their own aims, beliefs, values and principles, which could be different from ours and independent of the relationship a person has with the subject. It involves the ability to describe others’ psychology in a plausible, clear way, without using stereotypes or cliché. DEC also includes the ability to realize that basically we are not the centre of others’ intentions and goals. DEC comprises four skills: (a) the ability to recognize, define and verbalize other people’s emotional inner state; (b) the ability to recognize, define and verbalize other people’s cognitive inner state; (c) the ability to establish relations among the separate components of others’ mental state; and (d) the ability to establish relations between the components of others’ mental state and their behaviour. Examples of questions stimulating DEC are ‘How did you think the other person would react emotionally during the episode?’ ‘What did you think he/she thought?’ ‘Why did you think he/she thought that?’ and ‘What reasons did he/ she have?’. 2.2. Interview description The MAI begins with a standardized overview about its purpose — the aim of the interview — and collects basic demographic information. Afterwards, the interviewee is encouraged to describe an autobiographical episode about the worst psychological situation that he/she has experienced in the last six months. The episode has to be autobiographical and private. In order to evaluate the comprehension of others’ mental state, the episode has to include interaction with another person. We decided to investigate the worst episode of the last six months in order to be able to evaluate metacognitive function in critical circumstances with the prospect of applying it to clinical populations. We chose six months as the time interval to make it possible to test–retest, avoiding recalling biases, and for evaluating change during psychotherapy. This is explicitly aimed at eliciting affect-laden and personally relevant information. The MAI continues with four modules, each specific to the evaluation of one metacognitive dimension as described before. For each dimension of the metacognitive construct, the interviewer has to ask a structured list of questions. 2.3. Participants The study protocol was approved by the local Ethical Committee. Participants were recruited via advertisements posted around the hospital’s department of psychiatry. Those who answered the advertisement were screened and provided with information about the study, invited to sign the informed-consent form and were interviewed by an expert clinician. Axis I and II diagnoses were defined according to DSM-IV criteria (APA, 2000), using the Italian version of the Structured Clinical Interview for DSM-IV Axis I and II disorders (First et al., 1997). Exclusion criteria were current or past psychiatric diagnoses on DSM-IV Axis I or II, a history of psychiatric or psychological treatment, or a history of severe head trauma or substance-related disorders. None of the participants were taking psychotropic drugs, or they had used them occasionally during the month preceding the study. All six of the interviewers/raters conducted interviews and scored (rated) the 16 questions during the course of their interview, which lasted approximately 45 min. A sample comprising 175 non-clinical individuals (60 men and 115 women) was recruited from the community. The mean ages for women and men were 30.69 years (S.D. ¼ 13.51) and 30.68 years (S.D. ¼ 13.35), respectively, while the mean years of education were 14.88 (S.D. ¼2.52) for women and 14.64 years (S.D. ¼3.02) for men. 2.4. Measurement 2.4.1. Item development, interviewer training and preliminary inter-rater reliability evaluation As a first step, the authors formulated the structured questions that enable to describe how the interviewee employed metacognition during the recounting of the real-life episode. Guided by a previous theoretical work, nine questions were written for each of the four related dimensions (MON, DIF, INT and DEC); the interview therefore comprises 36 questions. Each question was discussed within the research group and approved for its inclusion in the interview. For each of the 36 questions, the interviewer could use adjunctive, rather than just structured questions to complete a more accurate information collection, by formulating questions to fit the person’s understanding, and asking additional details to clarify previous requests. The use of those free questions should be considered as probes and are divided into ‘helps’ and ‘specifications’, depending on the nature of the adjunctive question. If the interviewee needs to clarify an already detailed answer provided to a structured question, the interviewer will call it ‘specification’; these questions do not affect scoring. If the interviewee seems not to use well the metacognitive facet under evaluation, the interviewer will give a ‘help question’ to suggest the correct way to perform that task; the use of the ‘help’ will influence the scoring procedure, determining a reduction of the score for that specific facet. On the basis of the interviewee’s responses, the interviewer should determine the scores for each one of the 16 facets. The output of the MAI is made of 16 items. 3 For each facet the interviewer will assign a score ranging from one to five, on the basis of a Likert scale, to describe how well the interviewee employed that aspect of the metacognitive function during the real-life episode; a score of one stands for ‘always deficit functioning, while a score of five stands for ‘always functioning well’. For each question, a set of prompts and/or probes was written to elicit information assisting in the accurate scoring of the item. After the item selection procedure, 20 audiotaped interviews were conducted by the two senior interviewers after obtaining written informed consent from the interviewees. These interviews were then discussed in plenary sessions, creating the basis for the practical part of the user’s manual, which contains instructions for administration and scoring. After completing the user’s manual, a preliminary inter-rater reliability evaluation was carried out. The MAI was administered to 11 outpatients (f ¼5 and m¼ 6) and 3 healthy control subjects (f ¼ 2 and m¼ 1), recruited consecutively. The subjects were volunteers and fulfilled DSM-IV criteria for different disorder categories. The 11 interviews were conducted by the two senior interviewers, while all six of the staff rated each interview, using audio tapes. The interviewers and the raters were blinded to the clinical characteristics of the sample recruited. 2.5. Statistical analysis 2.5.1. Preliminary inter-rater reliability A blind rating procedure (blinded for diagnosis and others’ rating) was used in an ANOVA design. In order to estimate the correlation for every single facet rated by different judges, the Intraclass Correlation Coefficient (ICC) was used. A twoway mixed absolute agreement model was applied to carry out the Intraclass Correlation Coefficients for each dimension of the metacognitive construct, evaluated through the MAI. 2.5.2. Explorative factor analysis Firstly, we explored the factorial structure of the MAI with a factor analysis (based on the principal component extraction procedure) carried out using the Statistica 7.0 software package for Windows (StatSoft Inc., Tulsa, Oklahoma, USA). We used the scree test, the eigenvalue criteria (i.e. 41.00) and additional ‘rules of thumb’ for determining the number of factors for extraction and rotation (Horn, 1965; Cattel, 1966; Gorsuch, 1983; Floyd and Widaman, 1995). 2.5.3. Confirmatory factor analysis (CFA) and model comparisons We used CFA techniques in order to examine a number of different models that could fit the matrix of the 16 items of the interview and that could be considered conceptually and theoretically plausible, clarifying which latent structure best represents the data. On the basis of the results from the explorative factor analysis, we considered three competitive models: a two-factor nonhierarchical model, a two-factor hierarchical model and a one-factor model. Multiple statistical fit indices were used in order to assess goodness of fit of the proposed models: absolute fit indices (Chi Square/d.f. ratio and Goodness of Fit Index [GFI]); relative fit indices (Comparative Fit Index [CFI] and Non-Normed Fit Index [NNFI]); non-centrality based index (Root Mean Square Error of Approximation [RMSEA]). A good-fitting model produces consistent results on many different indices, according to statistical criteria reported elsewhere by different authors (Ullman, 1996; Kline, 1998; Netemeyer et al., 2004). Comparisons between adequate models with different degrees of freedom have been tested with the Chi Square test. The goodness-of-fit summary statistics from the CFA is displayed in Table 2. 2.5.4. Mean scores and reliability estimate and intercorrelations of the MAI and its scales Descriptive statistics for the MAI dimensions and total score were calculated; differences between men and women were explored using a series of T-tests. In order to analyze the internal consistency of the scale, we carried out Cronbach0 s alphas, average inter-item correlations (AICs) and corrected item-total correlations (CITCs) for the two domains and the global metacognitive function as measured with the MAI and Pearson correlations among the MAI scales. 3. Results 3.1. Preliminary inter-rater reliability The ICC for M facets ranges from 0.54 to 0.69; for DIF facets from 0.44 to 0.76; for I facets from 0.59 to 0.64; and for D facets from 0.41 to 0.57. The significance level is p o0.0001 for all analyses. Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015 4 A. Semerari et al. / Psychiatry Research ] (]]]]) ]]]–]]] Table 1 Results from the factor analysis of the 16-item set (two-factor solution). M3 DIF4 I3 I4 I1 M1 DEC3 DIF2 I2 DEC1 DIF3 M2 M4 DEC2 DIF1 DEC4 Eigenvalues Variance (%) Cumulative variance (%) Factor 1 Factor 2 Communalities 0.23  0.00 0.23 0.15 0.24 0.33 0.83 0.63 0.55 0.86 0.29 0.29 0.40 0.82 0.69 0.81 7.02 43.90 43.90 0.69 0.52 0.65 0.72 0.74 0.54 0.22 0.29 0.37 0.17 0.51 0.67 0.63 0.19 0.23 0.20 1.74 10.92 54.83 0.53 0.27 0.48 0.55 0.61 0.40 0.74 0.48 0.44 0.77 0.35 0.54 0.56 0.71 0.54 0.71 3.3. Confirmatory factor analysis and model comparisons Factor loadings higher than 0.40 are marked. M1 ¼Monitoring, item 1; M2¼Monitoring, item 2; M3 ¼ Monitoring, item 3; M4 ¼ Monitoring, item 4; I1 ¼Integrating, item 1; I2 ¼ Integrating, item 2; I3 ¼ Integrating, item 3; I4 ¼Integrating, item 4; DIF1¼ Differentiating, item 1; DIF2 ¼Differentiating, item 2; DIF3 ¼ Differentiating, item 3; DIF4 ¼ Differentiating, item 4; DEC1 ¼Decentring, item 1; DEC2 ¼Decentring, item 2; DEC3 ¼ Decentring, item 3; DEC4 ¼Decentring, item 4. Table 2 CFA goodness of fit results for various factor structure models for the 16-item set. Models Goodness of fit statistics w2 (d.f.) w2/d.f. GFI self-oriented metacognitive domain. The ‘describes transitions across different mental states and explains the reasons’ item (integration) has a less clear conceptual attribution, because it concerns one0 s own mental state but loads on the other-oriented factor. CFI NNFI RMSEA ratio Two-factor models 2a 2b 198.48 (103) 123.36 (87) 1.93 1.42 0.87 0.91 0.92 0.97 0.91 0.96 0.07 0.05 One-factor model 1a 394.68 (104) 3.80 0.70 0.78 0.74 0.16 3.2. Explorative factor analysis The first solution, obtained without imposing any constriction on the analysis, comprised three factors. The third factor, however, comprised only two significant loading items (i.e. 40.40) (Netemeyer et al., 2004) and was considered trivial (Gorsuch, 1983). The two-factor solution comprised at least three significant loading items, with each extracted factor explaining more than 5% of the unrotated variance (see Table 1) and the whole model explaining 54% of the total variance, as recommended by Netemeyer et al. (2004). We interpret this solution as providing interesting information about the theoretical hypothesis concerning the organization of the metacognitive function. All decentring items, two differentiation items (‘considers his/ her own representation of reality as subjective and questionable’ and ‘makes plausible interpretations of events’) and one integration item (‘describes transitions across different mental states and explains the reasons’) load on the first factor; the remaining items—all monitoring, two differentiating (‘evaluates and reflects on events’ and ‘distinguishes between different kinds of mental representations’) and three integrating (‘demonstrates a comprehensible and coherent link between thoughts, emotions and behaviours’, ‘generalizes on mental functioning and detects aspects of continuity in his/her own way of thinking and feeling’ and ‘reconstructs and describes relevant aspects of his/her narration’) items—loaded on the second one. These two dimensions could be conceptualized respectively as an other-oriented and a Model 2a was a two-factor, non-hierarchical model constituted by two independent factors. NNFI and CFI show adequate fit (0.92 and 0.93 respectively) as well as RMSEA (0.076), while GFI is 0.87, consistent with an inadequate fit. The Chi Square/d.f. ratio is consistent with a good fit (1.93). Model 2b was a two-factor hierarchical model constituted by two partially dependent factors, nested under a single higher-order factor. NNFI and CFI show good fit (0.96 and 0.97 respectively) for this model, while GFI is 0.92, consistent with an adequate fit as well as RMSEA (0.051). The Chi Square/d.f. ratio is consistent with a good fit (1.42). Model 1a was a one-factor model, with all items loading on a single factor. GFI and relative fit indices produce inconsistent results (o0.90) as well as RMSEA (0.164) while the Chi Square/d.f. ratio is consistent with a just sufficient adequate fit (3.80). Because models 2a and 2b have both shown consistent results on most or all fit indices, they have been compared with the Chi Square test of difference. Model 2b produced better fit than model 2a: model 2a vs. model 2b, Chi Square (16)¼ 75.126, po0.00000. Based on these results, we could conclude that a two-factor hierarchical model is the best way to describe the 16-item matrix. In particular, the MAI seems to be composed of two lower-order scales, representing the two main domains of the metacognitive function, which has the Self and the Other as object of its processes, and one single higher-order scale that we labelled ‘metacognition’. Parameter estimates for model 2b are displayed in Table 3. 3.4. Mean scores and reliability estimate and intercorrelations of the MAI and its scales Table 4 reports Cronbach’s alphas, AICs and CITCs for the two domains and the global metacognitive function as measured with the MAI. Cronbach’s alphas range between 0.85 of the second Table 3 Item-to-facet parameter estimates from the CFA. Two-factor hierarchical model (n ¼175). Factor/item Parameter estimate Prob. level (2)-1-4[M3] (2)-2-4[DIF4] (2)-3-4[I3] (2)-4-4[I4] (2)-5-4[I1] (2)-6-4[M1] (2)-7-4[DIF3] (2)-8-4[M2] (2)-9-4[M4] (1)-10- 4[DEC3] (1)-11-4[DIF2] (1)-12-4[I2] (1)-13-4[DEC1] (1)-14-4[DEC2] (1)-15-4[DIF1] (1)-16-4[DEC4] 0.38 0.42 0.28 0.57 0.48 0.34 0.36 0.38 0.30 0.62 0.37 0.10 0.62 0.68 0.40 0.61 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.18 0.00 0.00 0.00 0.00 M1¼Monitoring. item 1; M2 ¼Monitoring. item 2; M3 ¼Monitoring. item 3; M4¼Monitoring. item 4; I1 ¼Integrating. item 1; I2 ¼Integrating. item 2; I3 ¼Integrating. item 3; I4 ¼Integrating. item 4; DIF1¼ Differentiating. item 1; DIF2¼ Differentiating. item 2; DIF3¼ Differentiating. item 3; DIF4¼ Differentiating. item 4; DEC1¼ Decentring. item 1; DEC2¼ Decentring. item 2; DEC3¼ Decentring. item 3; DEC4 ¼ Decentring. item 4. Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015 A. Semerari et al. / Psychiatry Research ] (]]]]) ]]]–]]] Table 4 CITCs. Cronbach’s a and AICs for the M.A.I. scales. Factor 1 M3 DIF4 I3 I4 I1 M1 DIF3 M2 M4 DEC3 DIF2 I2 DEC1 DEC2 DIF1 DEC4 a/AIC 0.80 0.61 0.54 0.80 0.76 0.65 0.76 0.90/0.57 Factor 2 Total 0.61 0.38 0.58 0.60 0.67 0.55 0.50 0.63 0.65 0.57 0.30 0.55 0.53 0.62 0.55 0.71 0.60 0.59 0.70 0.51 0.61 0.66 0.68 0.61 0.69 0.91/0.40 0.85/0.40 M1¼ Monitoring. item 1; M2 ¼ Monitoring. item 2; M3 ¼ Monitoring. item 3; M4¼ Monitoring. item 4; I1 ¼ Integrating. item 1; I2 ¼ Integrating. item 2; I3 ¼ Integrating. item 3; I4 ¼ Integrating. item 4; DIF1 ¼Differentiating. item 1; DIF2 ¼Differentiating. item 2; DIF3 ¼Differentiating. item 3; DIF4 ¼Differentiating. item 4; DEC1 ¼Decentring. item 1; DEC2 ¼Decentring. item 2; DEC3 ¼Decentring. item 3; DEC4 ¼Decentring. item 4. Table 5 Pearson correlations among the MAI scales. Factor 1 Factor 2 Total Factor 1 Factor 2 Total – 0.62 0.90 0.62 – 0.89 0.90 0.89 – All po 0.001. factor (self-oriented metacognition) and 0.91 of the global scale (total metacognition), exceeding the 0.70 criterion (Clark, 1995; Ullman, 1996; Kline, 1998; Netemeyer et al., 2004). For the three scales, no item, if deleted, produces higher Cronbach’s alphas. CITCs for all the items are largely higher than 0.20 (Clark, 1995; Ullman, 1996; Kline, 1998; Netemeyer et al., 2004). Pearson correlations among the MAI scales are reported in Table 5; all correlations are statistically significant. Taken together, these results are consistent with the hypothesis of a homogeneous, multidimensional construct. 4. Discussion Our study aimed first to investigate the chance of successfully and reliably assessing metacognitive skills through the use of a semi-structured interview about a real-life episode. Results regarding the preliminary statistical steps for the validation of a psychometric instrument are encouraging. The MAI can provide good inter-rater reliability, factorial validity and internal consistency. Our second aim was to test the idea that two partially different and separate domains, i.e. self-reflection and understanding of others’ minds, can be identified in the overall human capacity to detect and reason about mental states. Findings only partially supported the hypothesis. Specifically, the hypothesized domains were self-reflection and understanding others’ minds. The CFA yielded a two-factor solution but did not confirm the fit of the 16 facets into the domains as theoretically expected. On Factor 1, which could resemble the Other domain, were loaded all four items of the DEC dimension, two items of the DIF dimension, 5 ‘consider one’s own representation of the world as subjective and questionable’ and ‘give plausible interpretations of events’, and one item from the INT dimension, ‘describes transitions among different mental states and explains the reasons why’. On Factor 2, which could resemble the Self domain, were loaded: all four items of the monitoring domain and three items of the integration domain. A possible explanation for the items of the differentiation domain is that this subfunction measures two different levels of metacognitive processing. One level concerns the ability to distinguish between imagination or fantasy and reality. As observed by Bateman and Fonagy (2004), when this function is damaged, imagination and fantasy tend to be considered as real, reducing consequent reflective functioning and increasing the tendency for action. The ability to discriminate between fantasy and reality seems to be strongly related to the monitoring in the Self domain (as ‘I am aware’, ‘I am imagining’) and its dysfunction will come along with a tendency to act out. The two items of differentiation which fall into others’ domain could be considered as a manifestation of the false belief knowledge; in other words, they require some awareness that behaviour could be guided by beliefs, which are also sometimes wrong. Such awareness needs to take into account that others might hold beliefs and perspectives and make evaluations different from ours. Therefore it is not surprising that these two items are located in the Other domain. If these results are confirmed by further studies, a revision and distinction between the two types of differentiation would also be necessary for the MAS. All the D facets were loaded on the same factor, as well as two facets of the DIF dimension. These results fit into out theoretical model. With regard to the integration facet unexpectedly loaded on Factor 1 — ‘describes transitions among different mental states and explains the reasons why’ — a possible explanation is that this ability requires more complex inferential load in order to integrate individual mental representations with the representations of others. The reason why this item falls into the Other domain could be that the understanding of others’ mental states, without a direct access to these states as in monitoring, requires more inferential effort. Even the loading of all the MON facets and the three of the four I facets on factor 2 fits into our theoretical model. These data are still consistent with the idea that, in order to be skilfully aware of ourselves, on one hand we need to describe what we feel and think in order to form a consistent picture of ourselves as time and context change, and on the other hand we need to successfully form a coherent self-narrative that helps us identify basic aspects of our experience and realize whether we acted consistently or not with our recurrent patterns of behaviour (Dimaggio et al., 2008). In conclusion as theoretically hypothesized, the metacognitive construct, as measured by the MAI, seems to be supported by one multifaceted latent variable. Our results confirm that two higherorder factors (domains), the Self and the Other, exist but some elements of self-reflection are deeply intertwined with reflection in the mind of the others. These findings may yield relevant information for clinical practice, as they are likely to inform the clinician where to target the interventions as different aspects of the metacognitive system are compromised. We think that the initial findings from the present study are promising. In this study we found that in a sample of healthy subjects the MAI showed good inter-rater reliability, factorial validity and internal consistency. The MAI takes approximately 40–50 min, as well as lots of clinical interviews; with its prompts and probes to elicit further information and to appraise and judge any metacognitive deficits, the MAI may help to overcome specific problems of self-report measurements for assessing ability Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015 6 A. Semerari et al. / Psychiatry Research ] (]]]]) ]]]–]]] regarding social cognition such as not being aware of deficits in one’s own mental state. Moreover, the interview evaluates how the subject actually uses metacognition or fails to cope with stress, because of metacognitive deficits, within troubling real-life situations. Thanks to this feature the MAI resembles a psychometric tool that helpfully supports clinical practice. There are many limitations to our study. For example, further examinations of the psychometric properties are required, and in particular concurrent validity, test–retest stability, inter-rater reliability with a larger sample and predictive validity should be tested. Samples of 200 and larger are typically recommended for factor analysis procedures, and results from CFA have to be considered provisional for the same reason. Data about the concurrent validity of the instrument with the above listed variables in a non-clinical sample are now under evaluation. Furthermore, additional studies are needed to further confirm the construct validity of the MAI in other samples, particularly those from psychiatric populations. References Allen, J.G., Bateman, A.W., Fonagy, P., 2008. Mentalizing in Clinical Practice. American Psychiatric Publishing Inc., Washington DC. American Psychiatric Association, 2000. Diagnostic and Statistical Manual of Mental Disorders. Text Revision. DSM IV—TR, 4th ed. APA Press, Washington, DC. Baron-Cohen, S., 1995. Mindblindness. An Essay on Autism and Theory of Mind, Cambridge MA. MIT Press. Baron-Cohen, S., Leslie, A.M., Frith, U., 1985. Does the autistic child have a ‘theory of mind’? Cognition 21, 37–46. Bateman, A., Fonagy, P., 2004. Psychotherapy for Borderline Personality Disorder. Oxford University Press, Oxford, UK. Carcione, A., Dimaggio, G., Fiore, D., Nicolo, G., Procacci, M., Semerari, A., 2008. An intensive case analysis of client metacognition in a good-outcome psychotherapy: Lisa’s case. Psychotherapy Research 18 (6), 667–676.  G., Fiore, D., Procacci, M., Semerari, A. Carcione, A., Dimaggio, G., Conti, L., Nicolo, Metacognition Assessment Scale (MAS) V.4.0-Manual. Terzocentro, Rome, unpublished.  G., Pedone, R., Popolo, R., Conti, L., Fiore, D., Procacci, M., Carcione, A., Nicolo, Semerari, A., Dimaggio, G., 2011. Metacognitive mastery dysfunctions in personality disorder psychotherapy. Psychiatry Research 190 (1), 60–71. Cattel, R.B., 1966. The scree test for the number of factors. Multivariate Behavioural Research 1, 245–276. Clark, L.A., Watson, D., 1995. Constructing validity: basic issues in objective scale development. Psychological Assessment 7, 309–319.  G., Semerari, A., 2008. Know Dimaggio, G., Lysaker, P.H., Carcione, A., Nicolo, yourself and you shall know the othery to a certain extent: multiple paths of influence of self-reflection on mindreading. Consciousness and Cognition 17 (3), 778–789.  G., Conti, L., Fiore, D., Pedone, R., 2009. Impaired Dimaggio, G., Carcione, A., Nicolo, decentration in personality disorder: a series of single cases analysed with the metacognition assessment scale. Clinical Psychology and Psychotherapy 16 (5), 450–462. First, M.B., Gibbon, M., Spitzer, R.L., Williams, J.B.W., Benjamin, L.S., 1997. Structured Clinical Interview for DSM-IV Axis II Personality Disorders, (SCIDII). American Psychiatric Press Inc, Washington DC. Flavell, J.H., 1976. Metacognition aspects of problem solving. In: Resnick, L. (Ed.), The Nature of Intelligence. Erlbaum, Hillsdale, NJ. Floyd, F.J., Widaman, K.F., 1995. Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment 7, 286–299. Fonagy, P., 1991. Thinking about thinking: some clinical and theoretical considerations in the treatment of a borderline patient. International Journal of Psychoanalysis 72, 639–656. Fonagy, P., Target, M., 1997. Attachment and reflective function: their role in selforganization. Development and Psychopathology 9 (4), 679–700. Fonagy, P., Target, M., 2006. The mentalization-focused approach to self pathology. Journal of Personality Disorders 20 (6), 544–576. Frith, C., 1992. The Cognitive Neuropsychology of Schizophrenia. Erlbaum, Hillsdale, NJ. Frith, C., Frith, U., 2006. The neural basis of mentalizing. Neuron 50, 531–534. Frith, U., Happé, F., 1999. Theory of mind and self consciousness: what is it like to be autistic? Mind & Language 4, 1–22. Gorsuch, R.L., 1983. Factor Analysis. Lawrence Erlbaum, Hillside. Horn, J.L., 1965. A rationale and test for the number of factors in factor analysis. Psychometrika 30, 179–185. Kline, R.B., 1998. Principles and Practice of Structural Equation Modeling. Guilford Press, New York.  G., Procacci, M., Lysaker, P.H., Carcione, A., Dimaggio, G., Johannesen, J.K., Nicolo, 2005. Metacognition amidst narratives of self and illness in schizophrenia: associations with neurocognition, symptoms, insight and quality of life. Acta Psychiatrica Scandinavica 112, 64–71.  G., 2010a. MetacogniLysaker, P.H., Dimaggio, G., Buck, K.D., Carcione, A., Nicolo, tion and schizophrenia: the capacity for self-reflectivity and prospective assessments of work performance over six months. Schizophrenia Research 122, 124–130. Lysaker, P.H., Dimaggio, G., Daroyanni, P., Buck, K.D., LaRocco, V.A., Carcione, A., 2010b. Assessing metacognition in schizophrenia with the metacognition assessment scale: associations with the social cognition and object relations scale. Psychology and Psychotherapy 83, 303–315. Lysaker, P.H., Warman, D.M., Dimaggio, G., Procacci, M., LaRocco, V.A., Clark, L.K., 2008. Metacognition in schizophrenia associations with multiple assessments of executive function. Journal of Nervous and Mental Disease 196 (5), 384–389. Lysaker, P.H., Erickson, M.A., Buck, B., Buck, K.D., Olesek, K., Grant, M., 2011a. Metacognition and social function in schizophrenia: associations over a period of five months. Cognitive Neuropsychiatry 16 (3), 241–255. Lysaker, P.H., Kyle, L., Olesek, L., Warman, D.M., Martin, J.M., Salzman, A.K., 2011b. Metacognition in schizophrenia: correlates and stability of deficits in theory of mind and self-reflectivity. Psychiatry Research 190 (1), 18–22. Main, M., 1991. Metacognitive knowledge, metacognitive monitoring, and singular (coherent) vs. multiple (incoherent) models of attachment: findings and directions for future research. In: Stevenson-Hinde, J., Marris, P. (Eds.), Attachment Across the Life Cycle. Routledge, pp. 127–159. Netemeyer, R.G., Bearden, W.O., Sharma, S., 2004. Scaling Procedures: Issues and Applications. Sage Publications, Thousand Oaks. Ogrodniczuk, J.S., Piper, W.E., Joyce, A.S., 2011. Effect of alexithymia on the process and outcome of psychotherapy: a programmatic review. Psychiatry Research 190 (1), 43–48. Premack, D., Woodruff, G., 1978a. Chimpanzee problem-solving—test for comprehension. Science 202 (4367), 532–535. Premack, D., Woodruff, G., 1978b. Does chimpanzee have a theory of mind. Behavioral and Brain Sciences 1 (4), 515–526. Premack, D., Woodruff, G., 1978c. Cognition and consciousness in nonhuman species—reply. Behavioral and Brain Sciences 1 (4), 616–628.  G., Procacci, M., 2007. UnderSemerari, A., Carcione, A., Dimaggio, G., Nicolo, standing minds: different functions and different disorders? The contribution of psychotherapy research. Psychotherapy Research 17 (1), 106–119.  G., Procacci, M., 2003. Semerari, A., Carcione, A., Dimaggio, G., Falcone, M., Nicolo, How to evaluate metacognitive functioning in psychotherapy? The metacognition assessment scale and its applications. Clinical Psychology & Psychotherapy 10, 238–261.  G., Pedone, R., Procacci, M., 2005. Semerari, A., Carcione, A., Dimaggio, G., Nicolo, Metarepresentative functions in borderline personality disorder. Journal of Personality Disorders 19 (6), 609–710. Semerari, A., D0 Angerio, S., Popolo, R., Cucchi, M., Ronchi, P., Maffei, C., Dimaggio, G., Nicoló, G., Carcione, A., 2008. L0 Intervista per la Valutazione della Metacognizione (IVaM): Descrizione dello Strumento [The Metacognition Assessment Inerview (MAI): descriptionn of the instrument]. Cognitivismo Clinico 5, 174–192. Sperber, D., 2000. Metarepresentations. Oxford University Press, Oxford. Ullman, J., 1996. Structural equation modeling. In: Tabachnik, B., Fidell, L. (Eds.), Using Multivariate Statistics. Harper Collins, New York, pp. 709–811. Wellman, H.M., 1990. The Child’s Theory of Mind. Bradford Books. MIT Press, Cambridge, MA.. Wells, A., 2000. Emotional Disorders and Metacognition, Innovative cognitive therapy. John Wiley & Sons Ltd., UK. Please cite this article as: Semerari, A., et al., The development of the Metacognition Assessment Interview: Instrument description, factor structure and reliability in a non-clinical sample. Psychiatry Research (2012), http://dx.doi.org/10.1016/j.psychres.2012.07.015