Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? C... more Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? Consistencyof the p Factor Across Samples by Holly F. Levin-Aspenson, David Watson, Lee Anna Clark and Mark Zimmerman in Assessment
Historically, researchers have proposed higher-order factors to explicate the structure of psycho... more Historically, researchers have proposed higher-order factors to explicate the structure of psychopathology, including Externalizing, Internalizing, Fear, Distress, Thought Disorder, and a general factor. Despite extensive research in this domain, the underlying structure of psychopathology remains unresolved. Herein, we examine several issues in adjudicating among structural models of psychopathology. Using simulations and analyses of the extant literature, we contrast the model-based reliability of alternative structural models of psychopathology and highlight shortcomings of conventional model fit indices for such adjudication. We propose alternative criteria for evaluating and contrasting competing structural models, including various model characteristics (e.g., the magnitude and consistency of factor loadings and their precision), the consistency and sensitivity of factors to their constituent indicators, and the variance explained in and patterns of associations with relevant ...
There have been proposals to expand definitions for categorical disorders and dimensionally-conce... more There have been proposals to expand definitions for categorical disorders and dimensionally-conceptualized syndromes (e.g., psychopathy) to include negative mood lability and dysregulation (NMD). Factor analytic results are often presented in support of these proposals, and we provide factor analytic demonstrations across clinically-oriented samples showing that NMD indicators load strongly onto factors with a range of psychopathology. This is unsurprising from a transdiagnostic perspective but shows that factor analysis could potentially be used to justify expanding definitions for specific constructs even though NMD indicators show strong, nonspecific loadings on other psychopathology factors ranging widely in nature. Expanding construct definitions and assessment approaches to emphasize NMD also may negatively impact discriminant validity. Acknowledging differing viewpoints on expanding construct definitions, we agree that targeting NMD is essential for comprehensive assessment. ...
The challenges observed in health service psychology (HSP) training during COVID-19 revealed syst... more The challenges observed in health service psychology (HSP) training during COVID-19 revealed systemic and philosophical issues that preexisted the pandemic, but became more visible during the global health crisis. In a position paper written by 23 trainees across different sites and training specializations, the authors use lessons learned from COVID-19 as a touchstone for a call to action in HSP training. Historically, trainee voices have been conspicuously absent from literature about clinical training. We describe longstanding dilemmas in HSP training that were exacerbated by the pandemic and will continue to require resolution after the pandemic has subsided. The authors make recommendations for systems-level changes that would advance equity and sustainability in HSP training. This article advances the conversation about HSP training by including the perspective of trainees as essential stakeholders.
The challenges observed in health-service-psychology (HSP) training during COVID-19 revealed syst... more The challenges observed in health-service-psychology (HSP) training during COVID-19 revealed systemic and philosophical issues that preexisted the pandemic but became more visible during the global health crisis. In a position article written by 23 trainees across different sites and training specializations, we use lessons learned from COVID-19 as a touchstone for a call to action in HSP training. Historically, trainee voices have been conspicuously absent from literature about clinical training. We describe long-standing dilemmas in HSP training that were exacerbated by the pandemic and will continue to require resolution after the pandemic has subsided. We make recommendations for systems-level changes that would advance equity and sustainability in HSP training. This article advances the conversation about HSP training by including the perspective of trainees as essential stakeholders.
Sudden gains commonly occur among patients receiving psychotherapy for depression, and have been ... more Sudden gains commonly occur among patients receiving psychotherapy for depression, and have been found to consistently predict better treatment outcomes. However, the majority of prior research has examined sudden gains primarily in weekly or biweekly treatment settings. Individuals were divided into two groups; those who experienced at least one sudden gain and those who did not. Rates of sudden gain occurrence, pre-treatment factors, and posttreatment outcomes were examined between the two groups. Over 60% of this sample experienced at least one sudden gain, the majority of which occurred during the first three days of treatment. Sudden gains were associated with significantly lower baseline depression and anxiety severity. Patients who experienced sudden gains reported significantly greater improvement in depressive and anxiety symptoms, coping skills, functioning, positive mental health, and well-being at treatment termination. This study was conducted in a single location with a relatively homogeneous sample. Due to a lack of follow-up data, we were unable to determine if treatment outcomes were sustained after treatment termination. The assessment timeline of the depressive symptoms differs between baseline and daily scales, which may have affected the number of observed sudden gains after the initial treatment day. The proportion of sudden gains in this study is higher than those found in outpatient settings, demonstrating that this phenomenon may commonly occur among depressed patients in acute treatment. These results suggest that the mechanisms by which sudden gains occur may be reinforced by daily, intensive treatment.
The development of factor analysis is uniquely situated within psychology, and the development of... more The development of factor analysis is uniquely situated within psychology, and the development of many psychological theories and measures are likewise tethered to the common use of factor analysis. In this paper, we review modern methodological controversies and developments through concrete demonstrations of how to use factor analytic methods across the exploratory-confirmatory continuum. We illustrate best practices for working through common challenges in personality disorders research. To help researchers conduct risker tests of their theory-implied models, we review what factor analysis is and is not, as well as some dos and don’ts for engaging in the process of model evaluation and selection. Throughout, we also emphasize the need for closer alignment between factor models and our theories, as well as clearer statements about which criteria would support or refute the theories being tested. Consideration of these themes appears promising in terms of advances in theory, resear...
Supplemental material, sj-pdf-1-asm-10.1177_10731911211003976 for The Development of Preliminary ... more Supplemental material, sj-pdf-1-asm-10.1177_10731911211003976 for The Development of Preliminary HiTOP Internalizing Spectrum Scales by David Watson, Miriam K. Forbes, Holly F. Levin-Aspenson, Camilo J. Ruggero, Yuliya Kotelnikova, Shereen Khoo, R. Michael Bagby, Matthew Sunderland, Praveetha Patalay and Roman Kotov in Assessment
Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? C... more Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? Consistencyof the p Factor Across Samples by Holly F. Levin-Aspenson, David Watson, Lee Anna Clark and Mark Zimmerman in Assessment
Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When ... more Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When shortened screening tools are used, it is desirable to maintain equivalent diagnostic accuracy to full-length forms. This manuscript presents a case study that illustrates how external data and individual participant data meta-analysis can be used to assess the equivalence in diagnostic accuracy between a shortened and full-length form. This case study compares the Patient Health Questionnaire-9 (PHQ-9) and a 4-item shortened version (PHQ-Dep-4) that was previously developed using optimal test assembly methods. Using a large database of 75 primary studies (34,698 participants, 3,392 major depression cases), we evaluated whether the PHQ-Dep-4 cutoff of ≥ 4 maintained equivalent diagnostic accuracy to a PHQ-9 cutoff of ≥ 10. Using this external validation dataset, a PHQ-Dep-4 cutoff of ≥ 4 maximized the sum of sensitivity and specificity, with a sensitivity of 0.88 (95% CI 0.81, 0.93), 0.68 (95% CI 0.56, 0.78), and 0.80 (95% CI 0.73, 0.85) for the semi-structured, fully structured, and MINI reference standard categories, respectively, and a specificity of 0.79 (95% CI 0.74, 0.83), 0.85 (95% CI 0.78, 0.90), and 0.83 (95% CI 0.80, 0.86) for the semi-structured, fully structured, and MINI reference standard categories, respectively. While equivalence with a PHQ-9 cutoff of ≥ 10 was not established, we found the sensitivity of the PHQ-Dep-4 to be non-inferior to that of the PHQ-9, and the specificity of the PHQ-Dep-4 to be marginally smaller than the PHQ-9.
In the present report from the Rhode Island Methods to Improve Diagnostic Assessment and Services... more In the present report from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we describe the development and validation of the Clinically Useful Anger Outcome Scale (CUANGOS). Current anger measures vary in their psychometric quality, clinical utility, and clinically relevant content coverage, and no one scale addresses all three of these key considerations. We aimed to develop a brief, clinically useful anger scale that (a) assesses clinically relevant aspects of anger, (b) performs well across different patient populations, and (c) is brief, accessible, and easy to use in routine clinical practice either as a stand-alone measure or as part of an assessment battery. Analyses included data from 2710 psychiatric outpatients and 1397 partial hospitalization patients. We used data from randomly drawn subsamples to select items with good psychometric properties and sufficient distinction from measurements of other emotional dysfunction, resulting in a 5-item scale. In reliability and validity analyses using the remaining participants, CUANGOS scores showed high internal consistency and appropriate test-retest reliability, as well as excellent discriminant validity from measurements of depression and anxiety. CUANGOS scores converged strongly with clinician ratings of subjective and overt anger and differentiated across all or almost all levels of clinician-rated anger severity. CUANGOS scores were also significantly higher in patients with anger-related disorders versus patients with other psychiatric disorders. Results provide promising evidence for the CUANGOS as a reliable and valid measurement of anger in clinical populations. Moreover, the CUANGOS is brief and feasible to incorporate into routine clinical practice.
This study builds upon research indicating that focusing narrowly on model fit when evaluating fa... more This study builds upon research indicating that focusing narrowly on model fit when evaluating factor analytic models can lead to problematic inferences regarding the nature of item sets, as well as how models should be applied to inform measure development and validation. To advance research in this area, we present concrete examples relevant to researchers in clinical, personality, and related subfields highlighting two specific scenarios when an overreliance on model fit may be problematic. Specifically, we present data analytic examples showing that focusing narrowly on model fit may lead to (a) incorrect conclusions that heterogeneous item sets reflect narrower homogeneous constructs and (b) the retention of potentially problematic items when developing assessment measures. We use both interview data from adult outpatients (N = 2,149) and self-report data from adults recruited online (N = 547) to demonstrate the importance of these issues across sample types and assessment meth...
Self-report is a crucial part of psychological assessment, but the reliability and validity of se... more Self-report is a crucial part of psychological assessment, but the reliability and validity of self-report measures can be threatened by individual differences in item interpretation and response. Individuals higher in trait neuroticism were hypothesized to use a lower threshold to determine whether a given experience qualifies as symptomatic when completing self-report measures. This study investigated signal-detection processes to explain the association between trait neuroticism and overreport of psychological symptoms. A nondiagnosed sample of 280 college students were asked to determine the caseness of 34 target and foil vignettes, 22 of which describe negative, potentially symptomatic experiences. The main hypothesis was not supported by study findings. Exploratory findings and future directions are discussed.
As part of a broader project to create a comprehensive self-report measure for the Hierarchical T... more As part of a broader project to create a comprehensive self-report measure for the Hierarchical Taxonomy of Psychopathology consortium, we developed preliminary scales to assess internalizing symptoms. The item pool was created in four steps: (a) clarifying the range of content to be assessed, (b) identifying target constructs to guide item writing, (c) developing formal definitions for each construct, and (d) writing multiple items for each construct. This yielded 430 items assessing 57 target constructs. Responses from a heterogeneous scale development sample ( N = 1,870) were subjected to item-level factor analyses based on polychoric correlations. This resulted in 39 scales representing a total of 213 items. The psychometric properties of these scales replicated well across the development sample and an independent validation sample ( N = 496 adults). Internal consistency analyses established that most scales assess relatively narrow forms of psychopathology. Structural analyses...
The ICD-11 personality disorder model is the first fully dimensional assessment of personality pa... more The ICD-11 personality disorder model is the first fully dimensional assessment of personality pathology. It consists of a personality disorder (PD) dysfunction-severity dimension, which encompasses both self- and interpersonal dysfunction, and six optional qualifiers for five prominent personality traits—Negative Affectivity (NA), Detachment (DET), Dissociality (DSL), Disinhibition (DSN), and Anankastia (ANK)—plus a borderline pattern that is defined by the criteria of DSM-IV borderline PD. This article reports on the development of a new self-report measure to assess self- and interpersonal dysfunction and the five trait qualifiers. It is the first comprehensive measure of the ICD-11 PD model in that (a) it is the only one to include both PD dysfunction-severity as well as trait scales and because (b) it is based on the Clinical Description and Diagnostic Guidelines, which are more detailed than the “statistical” model description that is currently on the ICD-11 website. The autho...
Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? C... more Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? Consistencyof the p Factor Across Samples by Holly F. Levin-Aspenson, David Watson, Lee Anna Clark and Mark Zimmerman in Assessment
Historically, researchers have proposed higher-order factors to explicate the structure of psycho... more Historically, researchers have proposed higher-order factors to explicate the structure of psychopathology, including Externalizing, Internalizing, Fear, Distress, Thought Disorder, and a general factor. Despite extensive research in this domain, the underlying structure of psychopathology remains unresolved. Herein, we examine several issues in adjudicating among structural models of psychopathology. Using simulations and analyses of the extant literature, we contrast the model-based reliability of alternative structural models of psychopathology and highlight shortcomings of conventional model fit indices for such adjudication. We propose alternative criteria for evaluating and contrasting competing structural models, including various model characteristics (e.g., the magnitude and consistency of factor loadings and their precision), the consistency and sensitivity of factors to their constituent indicators, and the variance explained in and patterns of associations with relevant ...
There have been proposals to expand definitions for categorical disorders and dimensionally-conce... more There have been proposals to expand definitions for categorical disorders and dimensionally-conceptualized syndromes (e.g., psychopathy) to include negative mood lability and dysregulation (NMD). Factor analytic results are often presented in support of these proposals, and we provide factor analytic demonstrations across clinically-oriented samples showing that NMD indicators load strongly onto factors with a range of psychopathology. This is unsurprising from a transdiagnostic perspective but shows that factor analysis could potentially be used to justify expanding definitions for specific constructs even though NMD indicators show strong, nonspecific loadings on other psychopathology factors ranging widely in nature. Expanding construct definitions and assessment approaches to emphasize NMD also may negatively impact discriminant validity. Acknowledging differing viewpoints on expanding construct definitions, we agree that targeting NMD is essential for comprehensive assessment. ...
The challenges observed in health service psychology (HSP) training during COVID-19 revealed syst... more The challenges observed in health service psychology (HSP) training during COVID-19 revealed systemic and philosophical issues that preexisted the pandemic, but became more visible during the global health crisis. In a position paper written by 23 trainees across different sites and training specializations, the authors use lessons learned from COVID-19 as a touchstone for a call to action in HSP training. Historically, trainee voices have been conspicuously absent from literature about clinical training. We describe longstanding dilemmas in HSP training that were exacerbated by the pandemic and will continue to require resolution after the pandemic has subsided. The authors make recommendations for systems-level changes that would advance equity and sustainability in HSP training. This article advances the conversation about HSP training by including the perspective of trainees as essential stakeholders.
The challenges observed in health-service-psychology (HSP) training during COVID-19 revealed syst... more The challenges observed in health-service-psychology (HSP) training during COVID-19 revealed systemic and philosophical issues that preexisted the pandemic but became more visible during the global health crisis. In a position article written by 23 trainees across different sites and training specializations, we use lessons learned from COVID-19 as a touchstone for a call to action in HSP training. Historically, trainee voices have been conspicuously absent from literature about clinical training. We describe long-standing dilemmas in HSP training that were exacerbated by the pandemic and will continue to require resolution after the pandemic has subsided. We make recommendations for systems-level changes that would advance equity and sustainability in HSP training. This article advances the conversation about HSP training by including the perspective of trainees as essential stakeholders.
Sudden gains commonly occur among patients receiving psychotherapy for depression, and have been ... more Sudden gains commonly occur among patients receiving psychotherapy for depression, and have been found to consistently predict better treatment outcomes. However, the majority of prior research has examined sudden gains primarily in weekly or biweekly treatment settings. Individuals were divided into two groups; those who experienced at least one sudden gain and those who did not. Rates of sudden gain occurrence, pre-treatment factors, and posttreatment outcomes were examined between the two groups. Over 60% of this sample experienced at least one sudden gain, the majority of which occurred during the first three days of treatment. Sudden gains were associated with significantly lower baseline depression and anxiety severity. Patients who experienced sudden gains reported significantly greater improvement in depressive and anxiety symptoms, coping skills, functioning, positive mental health, and well-being at treatment termination. This study was conducted in a single location with a relatively homogeneous sample. Due to a lack of follow-up data, we were unable to determine if treatment outcomes were sustained after treatment termination. The assessment timeline of the depressive symptoms differs between baseline and daily scales, which may have affected the number of observed sudden gains after the initial treatment day. The proportion of sudden gains in this study is higher than those found in outpatient settings, demonstrating that this phenomenon may commonly occur among depressed patients in acute treatment. These results suggest that the mechanisms by which sudden gains occur may be reinforced by daily, intensive treatment.
The development of factor analysis is uniquely situated within psychology, and the development of... more The development of factor analysis is uniquely situated within psychology, and the development of many psychological theories and measures are likewise tethered to the common use of factor analysis. In this paper, we review modern methodological controversies and developments through concrete demonstrations of how to use factor analytic methods across the exploratory-confirmatory continuum. We illustrate best practices for working through common challenges in personality disorders research. To help researchers conduct risker tests of their theory-implied models, we review what factor analysis is and is not, as well as some dos and don’ts for engaging in the process of model evaluation and selection. Throughout, we also emphasize the need for closer alignment between factor models and our theories, as well as clearer statements about which criteria would support or refute the theories being tested. Consideration of these themes appears promising in terms of advances in theory, resear...
Supplemental material, sj-pdf-1-asm-10.1177_10731911211003976 for The Development of Preliminary ... more Supplemental material, sj-pdf-1-asm-10.1177_10731911211003976 for The Development of Preliminary HiTOP Internalizing Spectrum Scales by David Watson, Miriam K. Forbes, Holly F. Levin-Aspenson, Camilo J. Ruggero, Yuliya Kotelnikova, Shereen Khoo, R. Michael Bagby, Matthew Sunderland, Praveetha Patalay and Roman Kotov in Assessment
Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? C... more Supplemental material, Supplemental_Material for What Is the General Factor of Psychopathology? Consistencyof the p Factor Across Samples by Holly F. Levin-Aspenson, David Watson, Lee Anna Clark and Mark Zimmerman in Assessment
Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When ... more Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When shortened screening tools are used, it is desirable to maintain equivalent diagnostic accuracy to full-length forms. This manuscript presents a case study that illustrates how external data and individual participant data meta-analysis can be used to assess the equivalence in diagnostic accuracy between a shortened and full-length form. This case study compares the Patient Health Questionnaire-9 (PHQ-9) and a 4-item shortened version (PHQ-Dep-4) that was previously developed using optimal test assembly methods. Using a large database of 75 primary studies (34,698 participants, 3,392 major depression cases), we evaluated whether the PHQ-Dep-4 cutoff of ≥ 4 maintained equivalent diagnostic accuracy to a PHQ-9 cutoff of ≥ 10. Using this external validation dataset, a PHQ-Dep-4 cutoff of ≥ 4 maximized the sum of sensitivity and specificity, with a sensitivity of 0.88 (95% CI 0.81, 0.93), 0.68 (95% CI 0.56, 0.78), and 0.80 (95% CI 0.73, 0.85) for the semi-structured, fully structured, and MINI reference standard categories, respectively, and a specificity of 0.79 (95% CI 0.74, 0.83), 0.85 (95% CI 0.78, 0.90), and 0.83 (95% CI 0.80, 0.86) for the semi-structured, fully structured, and MINI reference standard categories, respectively. While equivalence with a PHQ-9 cutoff of ≥ 10 was not established, we found the sensitivity of the PHQ-Dep-4 to be non-inferior to that of the PHQ-9, and the specificity of the PHQ-Dep-4 to be marginally smaller than the PHQ-9.
In the present report from the Rhode Island Methods to Improve Diagnostic Assessment and Services... more In the present report from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we describe the development and validation of the Clinically Useful Anger Outcome Scale (CUANGOS). Current anger measures vary in their psychometric quality, clinical utility, and clinically relevant content coverage, and no one scale addresses all three of these key considerations. We aimed to develop a brief, clinically useful anger scale that (a) assesses clinically relevant aspects of anger, (b) performs well across different patient populations, and (c) is brief, accessible, and easy to use in routine clinical practice either as a stand-alone measure or as part of an assessment battery. Analyses included data from 2710 psychiatric outpatients and 1397 partial hospitalization patients. We used data from randomly drawn subsamples to select items with good psychometric properties and sufficient distinction from measurements of other emotional dysfunction, resulting in a 5-item scale. In reliability and validity analyses using the remaining participants, CUANGOS scores showed high internal consistency and appropriate test-retest reliability, as well as excellent discriminant validity from measurements of depression and anxiety. CUANGOS scores converged strongly with clinician ratings of subjective and overt anger and differentiated across all or almost all levels of clinician-rated anger severity. CUANGOS scores were also significantly higher in patients with anger-related disorders versus patients with other psychiatric disorders. Results provide promising evidence for the CUANGOS as a reliable and valid measurement of anger in clinical populations. Moreover, the CUANGOS is brief and feasible to incorporate into routine clinical practice.
This study builds upon research indicating that focusing narrowly on model fit when evaluating fa... more This study builds upon research indicating that focusing narrowly on model fit when evaluating factor analytic models can lead to problematic inferences regarding the nature of item sets, as well as how models should be applied to inform measure development and validation. To advance research in this area, we present concrete examples relevant to researchers in clinical, personality, and related subfields highlighting two specific scenarios when an overreliance on model fit may be problematic. Specifically, we present data analytic examples showing that focusing narrowly on model fit may lead to (a) incorrect conclusions that heterogeneous item sets reflect narrower homogeneous constructs and (b) the retention of potentially problematic items when developing assessment measures. We use both interview data from adult outpatients (N = 2,149) and self-report data from adults recruited online (N = 547) to demonstrate the importance of these issues across sample types and assessment meth...
Self-report is a crucial part of psychological assessment, but the reliability and validity of se... more Self-report is a crucial part of psychological assessment, but the reliability and validity of self-report measures can be threatened by individual differences in item interpretation and response. Individuals higher in trait neuroticism were hypothesized to use a lower threshold to determine whether a given experience qualifies as symptomatic when completing self-report measures. This study investigated signal-detection processes to explain the association between trait neuroticism and overreport of psychological symptoms. A nondiagnosed sample of 280 college students were asked to determine the caseness of 34 target and foil vignettes, 22 of which describe negative, potentially symptomatic experiences. The main hypothesis was not supported by study findings. Exploratory findings and future directions are discussed.
As part of a broader project to create a comprehensive self-report measure for the Hierarchical T... more As part of a broader project to create a comprehensive self-report measure for the Hierarchical Taxonomy of Psychopathology consortium, we developed preliminary scales to assess internalizing symptoms. The item pool was created in four steps: (a) clarifying the range of content to be assessed, (b) identifying target constructs to guide item writing, (c) developing formal definitions for each construct, and (d) writing multiple items for each construct. This yielded 430 items assessing 57 target constructs. Responses from a heterogeneous scale development sample ( N = 1,870) were subjected to item-level factor analyses based on polychoric correlations. This resulted in 39 scales representing a total of 213 items. The psychometric properties of these scales replicated well across the development sample and an independent validation sample ( N = 496 adults). Internal consistency analyses established that most scales assess relatively narrow forms of psychopathology. Structural analyses...
The ICD-11 personality disorder model is the first fully dimensional assessment of personality pa... more The ICD-11 personality disorder model is the first fully dimensional assessment of personality pathology. It consists of a personality disorder (PD) dysfunction-severity dimension, which encompasses both self- and interpersonal dysfunction, and six optional qualifiers for five prominent personality traits—Negative Affectivity (NA), Detachment (DET), Dissociality (DSL), Disinhibition (DSN), and Anankastia (ANK)—plus a borderline pattern that is defined by the criteria of DSM-IV borderline PD. This article reports on the development of a new self-report measure to assess self- and interpersonal dysfunction and the five trait qualifiers. It is the first comprehensive measure of the ICD-11 PD model in that (a) it is the only one to include both PD dysfunction-severity as well as trait scales and because (b) it is based on the Clinical Description and Diagnostic Guidelines, which are more detailed than the “statistical” model description that is currently on the ICD-11 website. The autho...
Uploads