Support for the Admissibility of Forensic Neuropsychological Evidence

Clinical Neuropsychology

This study on the state of the practice in forensic neuropsychological assessment provides key insight into accepted and well-validated instruments commonly used in forensic assessment. This is the bottom line of a recently published article in The Clinical Neuropsychologist. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | The Clinical Neuropsychologist | 2017, Online First Publication

Toward generally accepted forensic assessment practices among clinical neuropsychologists: a survey of professional practice and common test use

Authors

Casey LaDuke NYU-Langone Medical Center, Comprehensive Epilepsy Center, New York, NY, USA; Department of Psychiatry and Neurobehavioral Sciences, University of Virginia Health System, Charlottesville, VA, USA
William Barr NYU-Langone Medical Center, Comprehensive Epilepsy Center, New York, NY, USA
Donald L. Brodale Department of Sociology, Brooklyn College of The City University of New York, Brooklyn, NY, USA
Laura A. Rabin Department of Psychology, Brooklyn College and The Graduate Center of The City University of New York, Brooklyn, NY, USA

Abstract

Objective: This study investigated professional practice and common test use among clinical neuropsychologists engaging in forensic assessment.
Method: Doctorate-level psychologists active in the practice of neuropsychology and on the INS and NAN membership listings (n = 502) were surveyed about their demographics, professional practice, and common test use. Participants who reported engaging in forensic practice (n = 255) were further surveyed about their forensic practice.
Results: Forensic participants were more likely to be male and Caucasian, and reported higher ages, more years of professional experience, and a higher prevalence of board certification. While characteristics of their professional and forensic practice varied, forensic participants reported spending most of their professional time conducting neuropsychological assessments with adult clients in a private or group practice setting, focusing on civil referrals and civil legal questions involving older adult issues, developmental issues, head injury, and psychiatric issues. Common test use across neuropsychological assessment domains is presented for board-certified forensic participants (n = 77). An examination of these results reveals that the current pattern of test use is similar to the results of a more general survey of neuropsychological test use.
Conclusions: The findings provide insight into the practice of forensic neuropsychological assessment, and further establish the admissibility of neuropsychological evidence in the United States legal system. Results will be useful for clinical neuropsychologists, field leaders, and legal professionals hoping to gain insight into the role of clinical neuropsychology in civil and criminal legal decision-making.

Keywords

neuropsychological assessment; forensic assessment; test use; practice; admissibility

Summary of the Research

“Clinical neuropsychologists are psychological practitioners with specialized training and experience in the science of brain-behavior relationships… Prior survey research and professional commentaries highlight forensic neuropsychology as a rapidly developing subspecialty among clinical neuropsychologists. By definition, forensic neuropsychology is the application of neuropsychology to legal issues that arise in civil and criminal proceedings, and a forensic neuropsychologist is a neuropsychologist who provides opinions in a legal setting. Forensic practice is common among clinical neuropsychologists and accounts for a significant source of income.” (p. 1-2).

“Given the importance of general acceptance and the use of well-validated methods to the admissibility of expert evidence in the United States legal system (Daubert v. Merrell Dow Pharmaceuticals, Inc., 1993 [i.e. the Daubert standard]; Frye v. United States, 1923 [i.e. the Frye standard]), it is imperative to track common practices among neuropsychologists engaging in forensic assessment… The current study is a representative survey of clinical neuropsychologists engaging in forensic practice in North America, including items specific to their common test use.” (p. 2). “Participants in the current study sample therefore consisted of 502 doctorate-level psychologists active in the practice of neuropsychology and on the INS and NAN membership listings who responded whether they engaged in forensic practice” (p. 3).

“The current survey described the demographic characteristics of clinical neuropsychologists who engage in forensic practice. Compared to their non-forensic peers, forensic participants were more likely to be men. Consistent with their non-forensic peers, forensic participants were predominantly Caucasian. The reported disparities in gender and cultural identity among forensic participants are not surprising based on prior demographic data on neuropsychologists engaging in forensic practice. The current results reinforce the need to increase gender diversity in forensic neuropsychological practice specifically (particularly among board certified practitioners) and ethnic diversity in clinical neuropsychology generally, in part to ensure effective and ethical practice in forensic neuropsychological assessment.” (p. 12).

“Specific to their forensic practice, participants appeared to work with a higher proportion of civil referrals than criminal referrals, particularly among boarded forensic participants. Relatedly, civil legal questions were generally the most frequently reported forensic referral questions (i.e. personal injury, neurotoxic exposure, medical malpractice, testamentary capacity), followed by criminal legal questions (i.e. insanity, competence to stand trial or confess, and capital mitigation). These results generally support previously reported trends in forensic practice among clinical neuropsychologists… Common test use among practitioners board certified in neuropsychology and related fields was described across domains and methods relevant to contemporary neuropsychological assessment. The current results appeared broadly consistent with test use among general neuropsychologists described in prior research. This finding appears consistent with the guidance of both field experts and ethical authorities that practitioners should seek to apply the most valid and reliable assessment procedures and tests available, such that clinical practices need to change little in forensic assessment” (p. 16-17).

Translating Research into Practice

“Understanding common and well-validated practices plays a vital role in the development of forensic neuropsychology as a professional practice, and is a foundational aspect of the admissibility of neuropsychological evidence in civil and criminal legal proceedings. This study appears to be the first to investigate common test use among a representative sample of clinical neuropsychologists engaging in forensic practice. The results therefore represent an important step toward further understanding forensic assessment practices among clinical neuropsychologists, and further establishing the admissibility of neuropsychological evidence in the United States legal system.” (p. 10).

“The current findings appear to support the admissibility of neuropsychological evidence in legal decision-making. Specifically, the data highlight tests commonly used by neuropsychologists engaging in forensic assessments, which clearly speaks to the general acceptance criteria that is included in both Frye and Daubert standards and their various state and federal applications. Results also demonstrate the variety of tests used by neuropsychologists engaging in forensic assessments, which itself speaks to the core characteristic of forensic neuropsychological practice being based on a number of well-validated methods that appear to be implemented at both higher and lower frequencies. As such, the use of well-validated and lower-frequency tests in forensic neuropsychological assessments would also likely be admissible in jurisdictions applying either the Frye standard (i.e. general acceptance) or Daubert standard (i.e. general acceptance, falsifiability, peer-review, and known error rates). Overall, the current study further supports the admissibility of evidence obtained through clinical neuropsychological assessment in legal decision-making by providing data about generally accepted forensic assessment practices among clinical neuropsychologists” (p. 17).

Other Interesting Tidbits for Researchers and Clinicians

“Forensic and non-forensic participants appeared similar in their reported degrees and areas of specialization, though participants engaging in forensic practice reported more years of professional experience and a higher prevalence of board certification through a neuropsychology-related board. Given the legal standards of admissibility and increased scrutiny required in forensic assessments, it may not be surprising if neuropsychologists engage in forensic assessments after obtaining the level of mastery and fluency that comes from years of specialized training and experience in neuropsychology. Further, given the evolving conceptualization of board certification as the standard for competence to practice clinical neuropsychology and the standards set for the admissibility of neuropsychological evidence, it may not be surprising if neuropsychologists with board certification feel more capable or confident to engage in forensic practice. It will be important to track the prevalence of board certification among clinical neuropsychologists engaging in forensic practice, as this may come to represent a practice standard (real or perceived) among practitioners and legal professionals that could significantly impact the practice of forensic neuropsychological assessment” (p. 13).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Amanda Reed

Amanda L. Reed is a doctoral student in John Jay College of Criminal Justice’s clinical psychology program. She is the Lab Coordinator for the Forensic Training Academy. Amanda received her Bachelor’s degree in psychology from Wellesley College and a Master’s degree in Forensic Psychology from John Jay College of Criminal Justice. Her research interests include evaluator bias and training in forensic evaluation.

Assessing Sentence Comprehension May Reduce the Frequency of Invalid MMPI-2-RF Protocols

Assessing sentence comprehension, as opposed to word reading, may reduce the frequency of Invalid MMPI-2-RF Protocols in forensic evaluation. This is the bottom line of a recently published article in International Journal of Forensic Mental Health. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | International Journal of Forensic Mental Health | 2017, Vol. 16, No. 3, 239-248

Assessing Reading Ability for Psychological Testing in Forensic Assessments: An Investigation with the WRAT-4 AND MMPI-2-RF

Authors

Kiera Himsl, Patton State Hospital
Danielle Burchett, Loma Linda University
Anthony M. Tarescavage, California State University
David M. Glassmire, Kent State University

Abstract

This study examined the association between two measures of WRAT-4 reading ability—Word Reading and Sentence Comprehension—and two well-validated measures of inconsistent responding—MMPI-2-RF Variable Response Inconsistency (VRIN-r) and True Response Inconsistency (TRIN-r) among 136 forensic inpatients (90 men, 46 women). It was hypothesized that WRAT-4 Sentence Comprehension would demonstrate stronger associations with VRIN-r than WRAT-4 Word Reading. It was also hypothesized that there may be a minimal association between Sentence Comprehension and TRIN-r. Although WRAT-4 Word Reading was not significantly correlated with VRIN-r (rs = -.17, p = .07) or TRIN-r (rs = -.10, p =.31), Sentence Comprehension was significantly correlated with VRIN-r (rs = -.27, p = .01). A hierarchical regression predicting VRIN-r scores indicated that WRAT-4 Sentence Comprehension significantly accounted for an additional 5.4% of the variance in VRIN-r scores after accounting for self-reported education level and Word Reading (p = .03). However, Word Reading did not significantly account for any additional variance in VRIN-r after accounting for Education and Sentence Comprehension (incremental R2 = .001, p = .74). These results suggest that Sentence Comprehension (rather than Word Reading) should be assessed prior to administering psychological testing, especially in forensic settings.

Keywords

MMPI-2-RF, WTAT-4, reading level, consistency, forensic assessment

Summary of the Research

“One of the reasons for the widespread use of the MMPI [Minnesota Multiphasic Personality Inventory] instruments in forensic evaluations is their utility in measuring an examinee’s test-taking styles. Burchett and Ben-Porath (2010) found that the validity (as measured by external correlates) of MMPI-2-RF substantive scales was lower among examinees instructed to feign than among examinees who took the test under standard instructions. Therefore, forensic clinicians can be more confident in test scores when the reliability and validity of the examinee’s responses have been evaluated formally as part of the assessment” (p. 239).

“Two of the MMPI-2-RF Validity Scales—Variable Response Inconsistency (VRIN-r) and True Response Inconsistency (TRIN-r)—assess for inconsistent responding, which is most relevant to the current investigation. VRIN-r examines variable (i.e., random) responding, whereas TRIN-r assesses fixed responding by indicating whether examinees engage in acquiescent (i.e., fixed true) or counter-acquiescent (i.e., fixed false) responding styles” (p. 239).

“Because the MMPI-2-RF uses a subset of items from the MMPI-2 item pool, issues related to item-level reading comprehension are similar for the two instruments. Schinka and Borum (1993) calculated the grade level reading equivalent of MMPI-2 items using the formula for Flesch-Kincaid grade level, which utilizes word and syllable counts. The Flesch-Kincaid grade level of the MMPI-2 was roughly at the 4th to fifth-grade reading level, although some scales had grade level estimates at the sixth- to seventh-grade level” (p. 240).

Schinka and Borum cautioned that using an examinee’s completed grade level as a proxy for reading ability may result in overestimation of reading ability and concluded that individuals with less than an eighth-grade education may require additional assistance (e.g., synonyms for unrecognized words) due to the possibility of insufficient reading skills” (p. 240).

“Given these cautions, it is important to note that a large portion of adults in the United States have basic (“indicates skills necessary to perform simple and every- day literacy activities”) or below basic (“indicates no more than the most simple and concrete literacy skills”) literacy skills” (p. 240).

“MMPI-2-RF scales assessing important clinical constructs are likely to be less accurate among test protocols that were answered inconsistently and/or not pre-screened for consistency of responding” (p. 240).

Measures used: MMPI-2-RF and WRAT

“Archival MMPI-2 and MMPI-2-RF protocols were utilized from a larger data set of inpatients at a state-operated forensic psychiatric hospital […] All participants were primary English speakers” (p. 241).

“TRIN-r is first calculated by counting the raw number of TRIN-r True responses and TRIN-r False responses for each examinee. The raw count of False item pairs is subtracted from the raw count of True item pairs. Higher scores above 50T indicate an overall acquiescent response style, and lower scores indicate a counter-acquiescent response style” (p. 243).

“To determine whether WRAT-4 performance predicted inconsistent and acquiescent/counter-acquiescent responding above and beyond variance accounted for by self-reported grade level, hierarchical regression analyses were conducted.” (p. 243).

In addition “[A] series of hierarchical regression analyses [was conducted] in which the predictor of self-reported education was entered into the first block, WRAT-4 Word Reading was entered into the second block, and Sentence Comprehension was entered into the third block. This series of analyses was also conducted with education in the first block, Sentence Comprehension in the second block, and Word Reading in the third block.” (p. 244)

“In the second analysis in which self-reported education was entered into the first block, Sentence Comprehension into the second block, and Word Reading into the third block” (P. 244)

Results:
“[T]he WRAT-4 Word Reading subtest was not significantly correlated with VRIN-r or final TRIN-r; however, WRAT-4 Word Reading was significantly correlated with TRIN-r True. As expected, the WRAT-4 Sentence Comprehension subtest was significantly correlated with VRIN-r and TRIN-r True, but was not significantly associated with TRIN-r False or final TRIN-r. Both the WRAT-4 Word Reading and Sentence Comprehension subtests were significantly correlated with self-reported education attainment” (p. 244)

“It is important to note that a greater number of participants had Word Reading data than Sentence Comprehension data, which impacted the power for some analyses. Self-reported education was not significantly correlated with VRIN-r, final TRIN-r, TRIN-r True, or TRIN-r False” (p. 244).

“The results indicated that WRAT-4 Word Reading was associated with fixed acquiescent responding, but not with variable responding, whereas Sentence Comprehension was associated with variable responding and fixed acquiescent responding” (p. 245).

Moreover, from the hierarchical regression analyses “[T]he first block, which consisted of self-reported education, significantly accounted for 4.4% of the adjusted variance in VRIN-r scores. In the second block WRAT-4 Word Reading was entered, which did not significantly account for additional variance in VRIN-r; however, Sentence Comprehension significantly accounted for an additional 5.4% of the variance when entered into the final block” (P. 244)

“[W]ith Sentence Comprehension in the second block, rather than the third, Sentence Comprehension accounted for an addition 5.8% of the variance above and beyond self-reported education” (p. 244)

“The authors found that several MMPI-2-RF overreporting and underreporting scales were affected by random responding and acquiescent responding. Additionally, the authors found that MMPI-2-RF scales of overreporting produced fewer misclassifications when protocols were pre-screened with VRIN-r and TRIN-r before interpreting the overreporting scales” (p.240).

Translating Research into Practice

“Sentence Comprehension should be assessed prior to administering psychological testing like the MMPI-2-RF, particularly in cases where reading ability is questionable, as it appears to generalize better than Word Reading to the task of completing a self-report inventory” (p. 245).

“Examiners should be mindful that although word reading tasks are substantially more expedient than sentence comprehension tasks, the results of the current study suggest that word reading does not generalize as well as sentence comprehension to the consistent completion of self-report psychological tests” (p. 245).

“[I]n many cases any time saved by using a shorter word reading task might then lead the examiner to conduct lengthy self-report psychological testing that may be invalid in cases where examinees with low reading levels were not adequately screened for reading comprehension” (p. 245).

“[R]eading comprehension issues can impact performance on validity indicators, such as VRIN-r. In cases where reading comprehension is poor, administering the standardized audio version of the MMPI-2-RF is recommended” (p. 246).

Other Interesting Tidbits for Researchers and Clinicians

“Unlike the MMPI instruments and other multi-scale inventories such as the Personality Assessment Inventory, several other psychological tests do not have validity scales to identify if an individual was responding consistently to test items.” (p. 245).

In addition, “Limitations of this study include use of a relatively small sample and a lack of experimental control over the time between administration of the WRAT-4 and the MMPI-2-RF, due to the use of archival data” (p. 246).

“[R]eading ability is generally considered a test of premorbid functioning that is stable across time for adults, the amount of variability in reading ability over time is likely limited” (p. 246).

“In the second block WRAT-4 Word Reading was entered, which did not significantly account for additional variance in VRIN-r [and] it is important for future research to confirm if reading ability as assessed with the WRAT-4 is likely to remain stable for patients with psychotic spectrum diagnoses” (p. 246).

“It is recommended that future researchers consistently administer reading tests to all patients during the same testing session as the MMPI-2-RF. Future research with larger sample sizes would be useful for more fine-grained analyses of subgroups with different reading levels” (p. 246).

“[F]uture research examine the effect of language comprehension issues on validity indicators. Additionally, many individuals may have limited reading abilities in English due to English not being an individual’s primary language, which highlights the importance of assessing reading ability in English prior to testing” (p. 246).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Ahyun Go

Ahyun Go graduated from John Jay College of Criminal Justice with a BA in Forensic Psychology. She was also minoring in Police Studies. She plans to continue her studies in forensic psychology MA program in the near future. Her main research interests include cognitive biases and crime investigation.

Research should Include Consideration of Combined CST and MSO Evaluations

Combined evaluations of competency to stand trial (CST) and mental state at the time of the offense (MSO) should be incorporated into research methodology to more closely mirror clinical practice. This is the bottom line of a recently published article in Law and Human Behavior. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | Law and Human Behavior | 2017, Vol. 41, No. 3, 217-229

Combined Evaluations of Competency to Stand Trial and Mental State at the Time of the Offense: An Overlooked Methodological Consideration?

Authors

Lauren Kois, John Jay College and the Graduate Center, City University of New York
Preeti Chauhan, John Jay College and the Graduate Center, City University of New York
James M. Wellbeloved-Stone, University of Virginia
Janet I. Warren, University of Virginia

Abstract

Combined evaluations of competency to stand trial (CST; competency) and mental state at the time of the offense (MSO; sanity) frequently co-occur. However, most research examines the 2 as discrete constructs without considering 4 potential combined evaluation outcomes: competent-sane, incompetent- sane, competent-insane, and incompetent-insane. External validity can be improved if research more closely mirrored practice. It may be incorrect to assume incompetent defendants are similar across CST-only and combined evaluations, and insane defendants are similar across MSO-only and combined evaluations. Using a sample of 2,751 combined evaluations, we examined demographic, clinical, offense, evaluation, and psycholegal characteristics associated with evaluators’ combined evaluation opinions. Multinomial regression analyses revealed older defendants were more likely to be opined incompetent- insane. Defendants with psychotic disorders were more often opined insane, regardless of competency status. Affective diagnoses predicted competent-insane opinions. Developmental disorders were closely related to incompetence, regardless of sanity status. Defendants with organic disorders tended to have global psycholegal impairment, in that they were more often opined incompetent-insane, incompetent- sane, or competent-insane, relative to competent-sane. Prior hospitalization predicted competent-insane relative to competent-sane opinions. Defendants not under the influence of a substance during the offense or with no prior convictions were more likely to be opined insane, regardless of competency status. We interpret these findings in light of psycholegal theory and provide recommendations for research and practice. Collectively, results suggest incorporation of combined evaluations into CST and MSO research is an important methodological consideration not to be overlooked.

Keywords

forensic assessment, competency to stand trial, mental state at the time of the offense, insanity, combined evaluation

Summary of the Research

“Evaluators regularly conduct combined evaluations of competency to stand trial (CST, competency) and mental state at the time of the offense (MSO, sanity). In their survey of U.S. pretrial forensic services, Grisso, Cocozza, Steadman, Fisher, and Greer found referral and evaluation processes for CST and MSO were similar and evaluators frequently conducted evaluations of both referral questions simultaneously in most states. Researchers have documented the frequency of these combined evaluations, with estimates occurring up to 100% of the time in study samples. However, the research literature most often examines correlates of CST and MSO independently, instead of examining them in the context of the four potential combined evaluation opinions or outcomes— competent-sane, incompetent-sane, competent- insane, and incompetent-insane. This brings to question the external validity and generalizability of this body of research, in that research thus far does not usually mirror clinical practice.” (p. 217)

“CST and MSO are strongly associated, although the correlation is not perfect. Indeed, Chauhan et al. found correlates of CST and MSO differed across CST-only, MSO-only, or combined evaluations. Furthermore, forensic evaluators in their study more often opined defendants incompetent in CST-only evaluations (36.6%) compared to combined evaluations (13.8%) and more often opined defendants insane in combined evaluations (15.0%) relative to MSO-only evaluations (12.2%), although the latter effect size was small. Still, taken together, research suggests defendants, evaluation processes, and CST and MSO opinions may be fundamentally different across singular and combined evaluations, and we cannot assume base rates and correlates of CST and MSO remain consistent regardless of the referral question and clinical inquiry implemented to address them.” (p. 218)

“Our literature review suggests base rates and correlates of CST and MSO may vary according to a sample’s composition of singular and combined evaluations. This provides further evidence that, with combined evaluation samples, CST and MSO outcomes should be examined according to the four evaluation outcomes, rather than competent versus incompetent and sane versus insane. To shed light on this methodological consideration, we examined which demographic, clinical, offense, evaluation, and psycholegal characteristics are correlated with the four definitive outcomes of combined evaluations. Consistent with the literature, we hypothesized clinical and psycholegal impairment would demonstrate the most robust relationships with opinions of incompetent-sane, competent-insane, and incompetent-insane as compared to competent-sane.” (p. 219)

“At least 282 [evaluators] submitted 5,998 joint evaluations for this study between 1990 and 2005. Note that 1.7% (n =104) of cases were missing evaluator identification numbers, so the number of evaluators is likely larger. The number of evaluations conducted by each evaluator ranged from 1 to 922 with a mean of 21.00 (SD 80.03). Five evaluators contributed more than 260 cases (3 SDs above the mean) in our study. Therefore, we randomly selected the mean number of cases (n=21) for inclusion for each of these evaluators and removed all cases with missing evaluator identification. This brought our sample size to 3,371 joint evaluations.” (p. 220)

“Guided by our results, we believe combined evaluations of CST and MSO are unique relative to CST-only or MSO-only evaluations and should be examined accordingly. Study highlights include age predicted incompetentinsane outcomes; psychotic disorders were more often associated with insanity, while developmental disorders were more often associated with incompetence; organic disorders were linked to global psycholegal impairment; and defendants who were not under the influence at the time of the offense and who did not have prior convictions were more often opined insane. Importantly, study variables did not always predict consistent CST or MSO opinions across the four outcomes:
the only variable to do so was diagnosis of an organic disorder.” (p. 227)

Translating Research into Practice

“With respect to practice implications, we found evaluators are finding defendants incompetent but still opining on defendants’ MSO status on the FEIS, although we do not know whether or not evaluators included both opinions in their written reports for the court. Along with Bush, Connell, and Denney (2006), we recommend that when evaluating both psycholegal questions, evaluators
submit two separate reports to the court, accompanied by a message explaining the reports are kept separate so as not to violate defendants’ rights from self-incrimination. Combined evaluation reports may create important ethical and legal issues currently underaddressed in the literature. For instance, is it incriminating for an evaluator to note, in a CST report, whether or not defendants
were also evaluated for MSO? At present, there is no consensus or standard protocol for opining combined evaluation outcomes, and we are unsure of how evaluators typically proceed in combined evaluations when they believe a defendant is incompetent. Our missing opinion data could suggest evaluators vary in this process. According to the American Bar Association, “Existing professional ethics boards and committees should develop specific criteria and special review procedures designed to address the ethical questions that may arise when mental health professionals participate in the criminal process” (American Bar Association, 2016, Standard 7-1.6[b], “Joint professional obligations for improving the administration of justice in criminal cases involving individuals with mental disorders”). Combined evaluation issues require further contemplation by our field.” (p. 227)

“This research, in conjunction with others’ work, begins to illuminate the complexities of combined CST and MSO evaluations. As it stands, the nature of these evaluations is largely overlooked in the literature, and there are still many questions to pursue in order to develop a comprehensive understanding of the intricacies of CST and MSO research and practice. On a broader level, this study encourages researchers to carefully consider external valid-ity (meaning, true practice) when conceptualizing clinical study designs. Moving forward, we hope researchers and practitioners consider the methodological implications of combined evaluation structures in their work.” (p. 228)

Other Interesting Tidbits for Researchers and Clinicians

“Our findings indicate we cannot assume (1) defendants opined competent-sane in combined evaluations resemble those who are opined competent or sane in solitary CST or MSO evaluations (or similar patterns), and (2) there is no difference between defendants opined competent-sane and competent-insane, and no difference between defendants opined incompetent-sane and incompetent- insane. Assuming “pure” CST or MSO samples may set the stage for methodological confounds (e.g., evaluation type) that cloud the true nature of CST and MSO research findings. These findings highlight the need for further research to attend to the complexity of combined evaluations and the impact they may have on evaluation outcomes and the related research.” (p. 227)

“We encourage researchers to replicate our study across jurisdictions. We cannot assume our results are typical or if they will generalize to other forensic systems. Although Grisso et al. found more similarities than differences among individual states’ pretrial forensic services, divergence in referral base rates, legal criteria, and evaluator and defendant characteristics will likely influence evaluation outcomes. Past research that incorporated a subset of our data identified both inter- and intrastate differences in these domains.” (p. 227)

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Kenny Gonzalez

Kenny Gonzalez is currently a master’s student in the Forensic Psychology program at John Jay College. His main research interest include forensic assessment, specifically violence risk. In the future, Kenny hopes to obtain a Phd in clinical forensic psychology and pursue a career in academia and practice.

 

Computerized Assessments Could be a Viable Alternative to Traditional Assessment in Forensic Settings

Incarcerated participants reacted positively to computerized assessment formats in this randomized controlled experiment. This is the bottom line of a recently published article in Law and Human Behavior. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | Law and Human Behavior | 2017, Vol. 41, No. 5, 468-477

Tablet Computers and Forensic and Correctional Psychological Assessment: A Randomized Controlled Study

Authors

Christopher M. King Montclair State University
Kirk Heilbrun Drexel University
Na Young Kim Drexel University
Kellie McWilliams Drexel University
Sarah Phillips Drexel University
Jessie Barbera Drexel University
Ralph Fretz Community Education Centers, West Caldwell, New Jersey

Abstract

Mobile computing technology presents various possibilities and challenges for psychological assessment. Within forensic and correctional psychology, assessment of justice-involved persons facilitated by such technology has not been empirically examined. Accordingly, this randomized controlled experiment involved administering questionnaires about risk—needs, treatment readiness, and computerized technology opinions to a large (N 212) and diverse sample of individuals under custodial correctional supervision using either a tablet computer or traditional paper-and-pencil materials. Results revealed that participants in the paper-and-pencil condition completed the packet of questionnaires faster but omitted items more frequently. Older participants and those with lower levels of education tended to take longer to complete the tablet-administrated measures. The tablet format was rated as more usable irrespective of demographic and personal characteristics, and most participants across the 2 conditions indicated that they would prefer to use computerized technology to complete psychological testing. Administration format did not have a clear effect on attitudes toward correctional rehabilitation services. Noteworthy for researchers is the substantial time saved and absence of practical problems with the tablet condition. Implications for practitioners include the general usability of the devices, their appeal to incarcerated persons, and the potential for tablets to facilitate clinical and administrative tasks with corrections clients. Considering the novel nature of this study, its promising results, and its limitations, future research in this area is warranted.

Keywords

forensic assessment, correctional psychology, tablet computer, digital technology, data collection

Summary of the Research

“Compared with applied psychology in general, the specialty areas of forensic and correctional psychology may be advancing more slowly in the adoption of digital technologies to facilitate research and clinical work. This is apparent from the dearth of published scholarly literature on the topic of computerized technology applications for forensic or correctional psychology” (p. 468).

“Computer facilitations are available for certain psychological tests frequently used in forensic mental health assessment, as well as for some specialized forensic assessment instruments. However, only two studies could be located that empirically evaluated such technologies in correctional populations…there are simple questions that have not yet been systematically or thoroughly examined concerning computerized technology and forensic and correctional psychology. Do digitized administrations of self-report questionnaires save time? Do they reduce the number of skipped, missed, or otherwise unscorable items? Relevant to participant responsiveness to correctional rehabilitation services (e.g., motivation, engagement), do participants prefer computerized test administrations, and might attitudes toward assessment and treatment services improve following use of such technology? Do personal characteristics influence the answers to such questions? Second, the use of mobile computing technologies, such as tablet computers, has apparently not yet been empirically examined in a forensic mental health or correctional context. This contrasts with other areas of mental health practice, although the research base is underdeveloped throughout applied psychology generally” (p. 469).

“The present study involved recruiting and randomly assigning a large, diverse sample of individuals in a custodial community corrections setting to complete either a tablet computer or paper-and-pencil version of an 86-item packet of self-report questionnaires that covered demographic and index offense information, self-perceived criminogenic risk and needs, attitudes toward correctional treatment, and opinions about computing technologies in correctional rehabilitation services… The diverse composition of the sample also allowed an exploratory examination of relationships among demographic characteristics and dependent variables. Response patterns on the self-perceived criminogenic risk and needs measure, broken out by study condition, were also examined for exploratory purposes” (p. 469). In total, 212 participants from units comprised of “general population male state prisoners, general population female state prisoners, and male state prisoners with mental health treatment needs that could be met in a community correctional setting” were included in this study (p. 470). The measures used included the Risk Need Perception Survey (RNPS), Corrections Victoria Treatment Readiness Questionnaire (CVTRQ) Attitudes and Motivation subscale, and study-specific questions about the testing format and their opinions of computerized assessments for a total of 86 questions.

“Somewhat unexpectedly, participants in the paper-and-pencil condition completed the study materials faster than those in the tablet condition…It was hypothesized that participants completing the questionnaires via tablets would omit significantly fewer items, with the survey software ensuring that the paper-and-pencil condition would be contrasted with a mean of zero. Results were consistent with this hypothesis. Participants filling out the questionnaires with paper and pencil tended to accidentally omit or otherwise provide an uninterpretable marking for one or more items. The ability to ensure that participants do not unintentionally skip any items, and the unequivocal recording of responses, is a clear advantage of computer-facilitated assessment technology. It was also hypothesized that participants in the tablet condition would rate their administration format more favorably in terms of usability (easy to use, understandable, kept one’s attention, and enjoyable). This was supported. Those in the tablet condition rated their format much more favorably than did paper-and-pencil participants. Notably, among tablet participants, usability ratings did not vary based on demographic or personal characteristics, suggesting that tablet-facilitated assessment may be appropriate for offenders of differing ages, levels of education, ethnic backgrounds, sexes, and mental health functioning” (p. 474). “Most participants in both conditions preferred (or would have preferred) to complete the study questionnaires using the tablet devices. The same was true for the preference to use tablets or other computers for future assessment. In addition, most participants, irrespective of condition, recommended increased incorporation of computing technologies into correctional rehabilitation” (p. 475).

Translating Research into Practice

“For practitioners, it is noteworthy that participants took significantly longer to complete the questionnaires using the tablet computers, although this may have been attributable to the fact that participants were not required to use a bubble answer sheet. This seems likely [according to prior research]. However, the tablet software enabled us to ensure that no items would be omitted or unintentionally skipped. Although some older participants needed minor assistance in using the tablets, all tablet participants successfully completed the digital questionnaires. As a group, they gave the digital format higher ratings in terms of usability. Importantly, tablet usability ratings did not differ as a function of demographic characteristics and mental health status, nor did administration format relate to response patterns on the RNPS. Most participants also indicated that they would be willing to use computers more in correctional treatment services—indeed, they would prefer it. This suggests the potential for increasing engagement in correctional rehabilitation through the use of computerized technology. Furthermore, there are numerous tasks that would be facilitated by the portability and power of tablet computers. It is time to consider this seriously” (p. 475).

“We note that there were no problems in using this equipment. Administrators may be concerned about corrections clients harming valuable equipment, or using it inappropriately (e.g., to access the Internet). No equipment was harmed or stolen. It was also straightforward to disable the tablets so that they could not be used for purposes beyond the intended study, and to password protect the devices against having this functionality restored by participants” (p. 475).

“There may be numerous advantages to thoughtfully integrating digital technology into forensic mental health and correctional contexts, including increasing engagement, decreasing the time required for data collection, decreasing transfer errors of physical data to digital databases, and decreasing the use of paper. The potential for using digital technology in forensic and correctional psychology is far greater than has been its application to date. Thoughtful use of such technology has enormous promise for improving the quality of research, assessment, treatment, and documentation in work with justice-involved individuals” (p. 476).

Other Interesting Tidbits for Researchers and Clinicians

“During study debriefings, a few participants noted that it would have taken them longer to fill out the paper-and-pencil materials if they had been required to use a bubble answer sheet, rather than just circling or making a mark in the response boxes that had been provided for questions. Also, age was related to completion time only for the tablet condition (older participants tended to take longer), and level of education was only clearly related to completion time for the tablet condition (individuals with lower levels of education tended to take longer). These effects may be related to greater familiarity and comfort with tablet computers among younger persons with higher educational achievement” (p. 474).

“For researchers, we note that receiving approval to introduce tablets into the study facility for use with incarcerated individuals required only one additional step: signing a property damage liability release. The survey software allowed us to store results locally for later uploading to cloud storage once offsite. Facility residents seemed interested in participating because it offered the potential opportunity to use a tablet. Attempts to misuse the devices (e.g., trying to enable the Internet) were infrequent (and unsuccessful). Perhaps the most significant finding for researchers is the data entry and checking time we saved—a few minutes for the tablet condition versus more than 25 hr for the paper-and pencil condition. Thus, researchers should consider taking advantage of computerized technologies in their studies, as our experiences were certainly encouraging” (p. 475).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Amanda Reed

Amanda L. Reed is a doctoral student in John Jay College of Criminal Justice’s clinical psychology program. She is the Lab Coordinator for the Forensic Training Academy. Amanda received her Bachelor’s degree in psychology from Wellesley College and a Master’s degree in Forensic Psychology from John Jay College of Criminal Justice. Her research interests include evaluator bias and training in forensic evaluation.

Less than 20% Total Agreement between Independent Evaluators on Forensic Mental Health Evaluations

Total agreement between independent evaluators on forensic mental health evaluations was less than twenty percent. This is the bottom line of a recently published article in Psychological Assessment. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | Psychological Assessment, 2017, Vol. 29, No. 6, 692-700

Diagnostic Field Reliability in Forensic Mental Health Evaluations

Authors

W. Neil Gowensmith, University of Denver
Stephanie N. Sessarego, University of Denver
Meghan K. McKee, University of Denver
Samantha Horkott, University of Denver
Nine MacLean, University of Denver
Katherine E. McCallum, University of Denver

Abstract

How likely are multiple forensic evaluators to agree on defendants’ diagnoses in routine forensic mental health evaluations? A total of 720 evaluation reports were examined from 240 cases in which 3 evaluators, working independently, provided diagnoses for the same defendant. Results revealed perfect agreement across 6 independent diagnostic categories in 18.3% of cases. Agreement for individual diagnostic categories was higher, with all 3 evaluators agreeing on the separate presence of psychotic, mood, or substance disorders in more than 64.7% of cases and agreeing on the presence of cognitive or developmental disorders in more than 89.7% of cases. However, evaluators agreed about the combination of psychotic and substance-related diagnoses in only 46.5% of cases. Agreement was enhanced by diagnoses with low base rates, and it was suppressed in evaluations conducted in jails. Psychiatrists and contracted evaluators were more likely to provide dissenting diagnostic categories than psychologists and state-employed evaluators. These results are among the first to document diagnostic agreement among nonpartisan practitioners in forensic evaluations conducted in the field, and they allow for practice and policy recommendations for evaluators in routine forensic practice to be made.

Keywords

Diagnosis, forensic assessment, reliability competency, sanity

Summary of the Research

“Many studies have examined the diagnostic reliability of individual disorders, as well as the criterion validity of each DSM edition, but fewer have examined reliability in the field. A significant body of literature suggests that practitioners do not always demonstrate the same levels of reliability and validity in their daily practice as is found in laboratory conditions. This highlights the importance of diagnostic field reliability, or the reliability of diagnoses made by professionals on cases in the course of their regular practice.” (p. 692-693).

“Researchers in Australia, assessing agreement of psychiatric diagnosis in forensic cases, found good interrater reliability on diagnoses of brain injuries, substance-induced disorders, and intellectual disabilities, but they also reported poor to moderate agreement on diagnosis of depressive, personality, and anxiety disorders. Overall, diagnostic reliability for mental health disorders has been described as “substandard” (p. 693).

“Inherent in the psycholegal constructs of competency and sanity is the diagnostic category of the person under evaluation… An evaluator who misdiagnoses a defendant could run the risk of inaccurately opining that defendant CST or not guilty by reason of insanity (or vice versa). Accurate diagnostic categorization is key in forensic mental health evaluation.” (p. 963).

“One state’s forensic system is arranged in a manner that is ideal for a naturalistic study of evaluator agreement. Hawaii Revised Statute 704–404 states that in felony cases requiring FMHAs, “the court shall appoint three qualified examiners” to evaluate the defendant. These examiners are neutral (not retained by defense or prosecution) and certified to conduct forensic evaluations by the state’s department of health (DOH). In each case involving CST and legal insanity, the court requires that the evaluator provide a “diagnosis of the physical or mental condition” of the person being evaluated (Hawaii Revised Statutes, 2013). […] The present study investigates diagnostic reliability in forensic mental health evaluations conducted in the field.” (p. 694).

Measures used: A total of 257 evaluation reports were submitted by psychiatrists (35.7%) compared with 463 by psychologists (64.3%). A total of 7 psychologists working for the state of Hawaii’s DOH submitted 240 reports (33.3%), compared with 33 contracted evaluators in independent practice who collectively submitted 480 reports (66.67%) […] Diagnoses were coded into the following categories: psychotic, cognitive, mood, personality, substance-related disorders, and developmental disorders/mental retardation (DD/MR).” (p. 694).

Results: “In 44 (18.3%) of the 240 cases, all three evaluators agreed on every diagnostic category, even when the defendant was diagnosed with multiple disorders. Results showed an agreement rate of 71.8% for psychotic disorders (n = 173) 64.7% for substance abuse (n = 156), and 65.2% for mood disorders (n = 157). Agreement was highest for diagnoses of DD/MR (n = 231; 95.9%) and cognitive disorders (n = 216). Diagnostic reliability was lowest for personality disorders; the evaluators agreed on 149 (61.8%) of the 240 cases.” (p. 695).

Researchers “then calculated unanimous agreement rates on various combinations of psychotic, substance-related, and DD/MR diagnostic categories. Evaluators agreed unanimously on the presence or absence of psychotic and substance disorders in 46.5% (n = 109, ICC = .61) of cases, on psychotic and DD/MR disorder in 68.8% (n = 165, ICC = .52) of cases, and on substance-related and DD/MR in 62.0% (n = 150, ICC = .41) of cases. Evaluators agreed unanimously on all three diagnostic categories in 43.8% (n = 105, ICC =.26) of cases. Furthermore “of the 44 evaluations showing perfect categorical diagnostic agreement, 13 (29.5%) were conducted in an inpatient hospital, 12 (27.3%) were conducted in jail, and 19 (43.2%) were conducted in an outpatient office … [and] perfect categorical diagnostic agreement was found significantly more often in inpatient hospitals (29.5% vs. 18.3%) and less often in correctional facilities (27.3% vs. 40.7%.” (p. 695).

In addition, “psychiatrists were disproportionately and significantly more likely to offer the lone dissenting diagnostic category.” (p. 695).

Translating Research into Practice (L2, #2488CD)

“Evaluators showed perfect agreement as to the presence or absence of all six diagnostic categories in only 18.3% of all cases. This percentage is much higher than would be expected by chance. However, the cases in which evaluators agreed across all diagnostic categories were relatively “simple” cases; evaluators coded for the presence of approximately 1.3 disorders in those cases.

Furthermore, although the perfect agreement rate exceeded chance, the 18.3% rate was still low. In routine practice, evaluators agreed on a defendant’s entire diagnostic picture in fewer than one of five cases.” (p. 696).

[E]valuators agreed unanimously on the presence or absence of the combination of psychotic and substance-related disorders in only 46.5% of cases. This represents a substantial drop-off from the diagnostic agreements rates for psychotic and substance-related diagnostic categories on their own (71.8% and 64.7%, respectively). It appears that one of the biggest hurdles regarding diagnostic agreement in forensic evaluations is that evaluators show lower rates of agreement in cases involving the potential of psychosis and substance abuse; however, these two diagnostic categories are critically important in most pretrial forensic evaluations.” (p. 697)

“In terms of field reliability, this means that evaluators reach a consensus on the most pertinent diagnostic categories for pretrial evaluations in fewer than half of all pretrial cases. This low level of agreement is likely to have serious implications for the psycholegal opinions made by the evaluators, and, in turn, the ultimate judicial dispositions made by the court. Evaluators that disagree on the presence of psychosis or substance-related disorders are more likely to disagree on the psycholegal opinions of competency and sanity.” (p. 697).

“[C]onducting evaluations within the first 2 weeks is more likely to lead to disagreement on diagnoses and the psycholegal issues under consideration and that disagreement dissipates beyond the 15-day time frame. […] Defendants
evaluated later may present with more reliable, stable symptoms.” (p. 697).

Other Interesting Tidbits for Researchers and Clinicians

Overall, field reliability of most forensic evaluations is poor to mediocre (Fuger, Acklin, Nguyen, Ignacio, & Gowensmith, 2014; Robinson & Acklin, 2010). These authors have suggested that low reliability may partially stem from diagnostic uncertainty; however, this hypothesis has yet to be empirically tested.

Furthermore, some recent studies have hinted that diagnostic categories can affect the overall reliability of evaluator opinions in forensic cases. In cases involving the evaluation of legal sanity, evaluators were more likely to disagree on the defendant’s sanity in cases in which defendants were diagnosed with substance-related disorders (Gowensmith, Murrie, & Boccaccini, 2013b).

Conversely, in the same study, evaluators were more likely to agree on defendants’ legal sanity when defendants were diagnosed with psychotic disorders.” (p. 693-694).

“[D]espite the near certainty of some diagnostic disagreement, at some point the amount of disagreement becomes unacceptable. That level is not defined in forensic mental health and is probably fodder for setting standards of practice in the field; where that threshold will ultimately be set is unknown.” (p. 698)

“[U]nanimous agreement on all diagnostic categories was found in fewer than one case out of five. Work should be done to improve this low level of diagnostic field reliability.” (p. 698).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Ahyun Go

Ahyun Go graduated from John Jay College of Criminal Justice with a BA in Forensic Psychology. She was also minoring in Police Studies. She plans to continue her studies in forensic psychology MA program in the near future. Her main research interests include cognitive biases and crime investigation.

Conducting competent research on competency and sanity

Incorporating combined competency to stand trial (CST) and mental state at the time of the offense (MSTO) evaluations into CST and MSTO research is an important methodological consideration not to be overlooked. This is the bottom line of a recently published article in Law and Human Behavior. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | Law and Human Behavior | 2017, Vol. 41, No. 3, 217-229

Combined Evaluations of Competency to Stand Trial and Mental State at the Time of the Offense: An Overlooked Methodological Consideration?

Authors

Lauren Kois, John Jay College and the Graduate Center, City University of New York
Preeti Chauhan, John Jay College and the Graduate Center, City University of New York
James M. Wellbeloved-Stone, University of Virginia
Janet I. Warren, University of Virginia

Abstract

Combined evaluations of competency to stand trial (CST; competency) and mental state at the time of the offense (MSO; sanity) frequently co-occur. However, most research examines the 2 as discrete constructs without considering 4 potential combined evaluation outcomes: competent-sane, incompetent-sane, competent-insane, and incompetent-insane. External validity can be improved if research more closely mirrored practice. It may be incorrect to assume incompetent defendants are similar across CST-only and combined evaluations, and insane defendants are similar across MSO-only and combined evaluations. Using a sample of 2,751 combined evaluations, we examined demographic, clinical, offense, evaluation, and psycholegal characteristics associated with evaluators’ combined evaluation opinions. Multinomial regression analyses revealed older defendants were more likely to be opined incompetent-insane. Defendants with psychotic disorders were more often opined insane, regardless of competency status. Affective diagnoses predicted competent-insane opinions. Developmental disorders were closely related to incompetence, regardless of sanity status. Defendants with organic disorders tended to have global psycholegal impairment, in that they were more often opined incompetent-insane, incompetent-sane, or competent-insane, relative to competent-sane. Prior hospitalization predicted competent-insane relative to competent-sane opinions. Defendants not under the influence of a substance during the offense or with no prior convictions were more likely to be opined insane, regardless of competency status. We interpret these findings in light of psycholegal theory and provide recommendations for research and practice. Collectively, results suggest incorporation of combined evaluations into CST and MSO research is an important methodological consideration not to be overlooked.

Keywords

forensic assessment, competency to stand trial, mental state at the time of the offense, insanity, combined evaluation

Summary of the Research

“Evaluators regularly conduct combined evaluations of competency to stand trial (CST, competency) and mental state at the time of the offense (MSO, sanity). In their survey of U.S. pretrial forensic services, Grisso, Cocozza, Steadman, Fisher, and Greer (1994) found referral and evaluation processes for CST and MSO were similar and evaluators frequently conducted evaluations of both referral questions simultaneously in most states. Researchers have documented the frequency of these combined evaluations, with estimates occurring up to 100% of the time in study samples. However, the research literature most often examines correlates of CST and MSO independently, instead of examining them in the context of the four potential combined evaluation opinions or outcomes—competent-sane, incompetent-sane, competent-insane, and incompetent-insane. This brings to question the external validity and generalizability of this body of research, in that research thus far does not usually mirror clinical practice” (p. 217).

“We make two implicit assumptions when neglecting to identify this frequent pairing of referral questions, relative to CST-only or MSO-only evaluations. First, we assume defendants opined competent-sane in combined evaluations resemble those who are opined competent or sane in solitary CST or MSO evaluations, with the same pattern continuing for those opined incompetent-sane, competent-insane, or incompetent-insane. The second assumption is when combined evaluations are conducted in one assessment process, there are no differences between defendants opined competent-sane and competent-insane and no differences between defendants opined incompetent-sane and incompetent-insane. Stated differently, those found competent, regardless of their sanity status, are assumed to resemble each other; those found incompetent, regardless of their sanity status, are also assumed to resemble each other, and so on” (p. 217-218).

“Given CST and MSO embody different psycholegal concepts requiring different types of clinical inquiry, it may be inappropriate to consider singular and combined evaluations as mirrored reflections of one another, devoid of differences associated with referral or clinical processes” (p. 218).

“While they may be evaluated simultaneously, CST and MSO are distinct psycholegal constructs. CST evaluations focus on current functioning and assess two primary abilities: whether defendants understand the legal proceedings against them and whether they can work with their attorney in developing a defense” (p. 218).

“Whereas CST evaluations are present focused, MSO evaluations are retrospective inquiries. In order to be acquitted insane in most jurisdictions, defendants must demonstrate a major mental disease or defect substantially impaired their ability to understand the nature or consequences of the illegal act, their ability to distinguish right from wrong, or their ability to resist their illegal impulse” (p. 218).

“Our literature review suggests base rates and correlates of CST and MSO may vary according to a sample’s composition of singular and combined evaluations. This provides further evidence that, with combined evaluation samples, CST and MSO outcomes should be examined according to the four evaluation outcomes, rather than competent versus incompetent and sane versus insane. To shed light on this methodological consideration, we examined which demographic, clinical, offense, evaluation, and psycholegal characteristics are correlated with the four definitive outcomes of combined evaluations. Consistent with the literature, we hypothesized clinical and psycholegal impairment would demonstrate the most robust relationships with opinions of incompetent-sane, competent-insane, and incompetent-insane as compared to competent-sane” (p. 219).

“In line with our hypotheses, psychiatric diagnoses and psycholegal abilities were strongly associated with evaluators’ CST and MSO opinions. Furthermore, we found predictors of incompetency and insanity vary when considering all four combined evaluation outcomes. We can conclude those found competent, regardless of their sanity status, do not always resemble each other; those found incompetent, regardless of their sanity status, do not always resemble each other, and so on. As a result, our findings shed light on the complexity of opinions associated with combined evaluations” (p. 223).

Translating Research into Practice

“Along with Bush, Connell, and Denney (2006), we recommend that when evaluating both psycholegal questions, evaluators submit two separate reports to the court, accompanied by a message explaining the reports are kept separate so as not to violate defendants’ rights from self-incrimination. Combined evaluation reports may create important ethical and legal issues currently under addressed in the literature. For instance, is it incriminating for an evaluator to note, in a CST report, whether or not defendants were also evaluated for MSO? At present, there is no consensus or standard protocol for opining combined evaluation outcomes, and we are unsure of how evaluators typically proceed in combined evaluations when they believe a defendant is incompetent” (p. 227).

“According to the American Bar Association, ‘Existing professional ethics boards and committees should develop specific criteria and special review procedures designed to address the ethical questions that may arise when mental health professionals participate in the criminal process.’ Combined evaluation issues require further contemplation by our field” (p. 227).

Other Interesting Tidbits for Researchers and Clinicians

“Our findings indicate we cannot assume (1) defendants opined competent-sane in combined evaluations resemble those who are opined competent or sane in solitary CST or MSO evaluations (or similar patterns), and (2) there is no difference between defendants opined competent-sane and competent-insane, and no difference between defendants opined incompetent-sane and incompetent-insane. Assuming ‘pure’ CST or MSO samples may set the stage for methodological confounds (e.g., evaluation type) that cloud the true nature of CST and MSO research findings. These findings highlight the need for further research to attend to the complexity of combined evaluations and the impact they may have on evaluation outcomes and the related research” (p. 227).

“We encourage researchers to replicate our study across jurisdictions. We cannot assume our results are typical or if they will generalize to other forensic systems. Although Grisso et al. (1994) found more similarities than differences among individual states’ pretrial forensic services, divergence in referral base rates, legal criteria, and evaluator and defendant characteristics will likely influence evaluation outcomes. Past research that incorporated a subset of our data identified both inter- and intrastate differences in these domains” (p. 227).

“Our base rate of incompetency was much lower than that of Pirelli, Gottdiener, and Zapf (2011), but similar to research incorporating combined evaluations. It may be that, in general, base rates of incompetence are much higher when the only psycholegal question is a defendant’s CST. We suggest researchers begin to clearly state the source of their data (i.e., CST-only, MSO-only, or combined evaluations). In doing so, research will better mirror practice and enhance external validity” (p. 227).

“Like Crocker et al. (2002), we had small portions of evaluators provide no opinion for CST or MSO in our study. Pirelli, Zapf, and Gottdiener (2011) proposed that exploring the ‘gray areas’ (i.e., deferred opinions) of CST is an important pursuit. This is an important endeavor in MSO research as well. Deferred evaluation opinions could limit due process by stalling legal proceedings. Unfortunately, our sample sizes were too small to analyze ‘no opinions.’ While it may be difficult to locate a significant number of deferred opinions given their low base rates, researchers may oversample this group in future analyses” (p. 227).

“As it stands, the nature of these evaluations is largely overlooked in the literature, and there are still many questions to pursue in order to develop a comprehensive understanding of the intricacies of CST and MSO research and practice. On a broader level, this study encourages researchers to carefully consider external validity (meaning, true practice) when conceptualizing clinical study designs. Moving forward, we hope researchers and practitioners consider the methodological implications of combined evaluation structures in their work” (p.228).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

About the Author

Amanda Beltrani is a current graduate student in the Forensic Psychology Masters program at John Jay College of Criminal Justice in New York. Her professional interests include forensic assessments, specifically, criminal matter evaluations. Amanda plans to continue her studies in a doctoral program after completion of her Masters degree.