Understanding and Mitigating Bias in Forensic Evaluation

Minimizing Bias in Forensic Decision Making Training

A new article in press in the International Journal of Forensic Mental Health written by Professors Patricia Zapf & Itiel Dror examines the ways in which bias can interfere with forensic evaluation.

Zapf, P. A., & Dror, I. E. (in press). Understanding and mitigating bias in forensic evaluation: Lessons from forensic science. International Journal of Forensic Mental Health.

Abstract

Research and commentary have emerged in the last decade surrounding cognitive bias in forensic examinations, both with respect to various domains within forensic science as well as with respect to forensic psychology. Indeed, in 2009 the National Research Council (NRC) issued a 352-page report entitled, Strengthening Forensic Science in the United States: A Path Forward that delineated several weaknesses within the various forensic science domains and proposed a series of reforms to improve the issue of reliability within the forensic sciences. Since the NRC report various commentators have written about the impact of cognitive biases in the forensic sciences and have proposed solutions to mitigate the impact of these biases. The purpose of this paper is to examine and consider the various influences that can bias observations and inferences in forensic evaluation and to apply what we know from forensic science to propose possible solutions to these problems.

Training in Minimizing Bias in Forensic Decision Making


This self-paced, online training program is presented by Dr. Itiel Dror and focuses on Minimizing bias in Forensic Decision Making. The program covers brain and cognitive issues relating to bias and cognitive processing, and then connects the cognitive science issues to practical and specific issues in forensic decision making. In addition to knowledge about the cognitive factors in forensic decision making, the program also provides practical solutions to address weaknesses as well as best practices to enhance forensic practices.

 

Specific application to forensic mental health evaluation is provided through engaging discussions between Dr. Dror and Dr. Patricia Zapf, a forensic psychologist and expert in best practices in forensic mental health evaluation. In addition, Dr. Zapf provides elaboration on how the factors discussed by Dr. Dror are applicable to forensic mental health evaluation. 

Evaluators Do Not Always Uphold Culturally Competent Practice Guidelines

Self-reported culturally competent practices among evaluators suggest that evaluators do not always uphold practice guidelines. This is the bottom line of a recently published article in the International Journal of Forensic Mental Health. Below is a summary of the research and findings as well as a translation of this research into practice.

Featured Article | International Journal of Forensic Mental Health | 2016, Vol. 15, No. 4, 312-322

Forensic Evaluators’ Self-Reported Engagement in Culturally Competent Practices

Authors

Lauren Kois Department of Psychology, John Jay College of Criminal Justice and the Graduate Center, City University of New York, New York, New York, USA
Preeti Chauhan Department of Psychology, John Jay College of Criminal Justice and the Graduate Center, City University of New York, New York, New York, USA

Abstract

Cultural competence is a rising concern within the sub-specialty of forensic evaluation and will grow in need as the population diversifies. We surveyed 100 forensic evaluators to explore issues related to cultural competence. Overall, evaluators differed demographically from those they evaluate. Self reported culturally competent practices varied, suggesting that evaluators do not always uphold practice guidelines. Evaluators’ training variety was associated with an increased likelihood to address communication difficulties with evaluatees. Evaluators who saw more racially and linguistically diverse evaluatees were more likely to participate in culturally sensitive case formulation practices. We conclude with implications for practice at the individual and institutional levels and directions for research.

Keywords

Forensic mental health; evaluator; assessment; culture; competence

Summary of the Research

“Recent estimates indicate that about 41% of jail inmates are Asian, Black, or Hispanic; that 22% of federal inmates are foreign-born; and that immigration proceedings constitute 46% of all federal arrests. Considering these numbers in combination with projected population trends, it is likely that forensic evaluators will conduct evaluations of diverse evaluatees with increasing frequency” (p. 312).

“The Specialty Guidelines for Forensic Psychologists and forensic evaluation scholars have highlighted the need for cultural competence in forensic contexts and have provided a number of suggestions for practice. Still, compared to the broader cultural competence literature, research and discussion regarding multicultural considerations within the forensic context is scarce. Further, the forensic evaluator/evaluatee relationship (i.e., nontherapeutic) is a unique one, and we cannot assume that all practices recommended for general assessment and psychotherapy extend to forensic evaluations. Nonetheless, a combination of general cultural competence and forensic-specific guidelines can provide a “roadmap” of five domains (communication, clinical interview and collateral information, assessment, case formulation, and bounds of competence) that evaluators may consider when conducting forensic evaluations with diverse populations” (p. 312-313).

“Theory, research, and specialty guidelines identify a number of steps for conducting culturally competent forensic evaluations. However, there has been no formal investigation to determine if and how forensic evaluators apply these practices that are intended to develop or demonstrate cultural competence. We conducted a survey of forensic mental health evaluators to explore this issue. Through this project, we aimed to document evaluator and evaluatee characteristics, quantify evaluators’ cultural competence training and self-reported practices, and explore which evaluator and evaluate characteristics are associated with these practices” (p. 314).

Results from this survey indicated that evaluators reported engaging in communication practices (i.e., focused on verbal and nonverbal communication, considered linguistic concerns, and coordinated interpretation services) usually or always. Evaluators reported engaging in clinical interview and collateral information practices (i.e., discussing immigration status, gathering third party information from family and friends, and discussing religious/spiritual beliefs) half the time to usually. Most evaluators denoted that they usually or always consider level of acculturation and psychometric properties of assessments with evaluees from similar groups. When preparing cases and answering referral questions, evaluators reported considering the evaluee’s cultural context usually to always. Additionally, evaluators reported that they recognize bounds of competence (i.e., discussion with experience colleagues and referencing literature on belief, values and traditions) half the time to usually. 

“Variety of training experiences was associated with asking evaluatees’ level of comfort in talking with evaluators. Consideration of how evaluatees’ perceptions of evaluators’ racial, ethnic, or cultural backgrounds and how it may influence response style was associated with evaluatees’ Race and Language DIs. Evaluatees’ language DIs were also associated with consideration of cultural context when forming diagnoses. No other associations were significant at our conservative p level of <.006” (p. 318).

“Evaluators varied in their reported methods of practice. In general, evaluators usually or always adhered to practice guidelines in the domains of assessment, followed by communication and case formulation practices. Still, over half of the practices we assessed exhibited the full range of potential responses, with some evaluators indicating that they never engaged in 14 of the 25 practices included in our survey. This suggests that at least some evaluators do not consistently practice according to APA guidelines. On average, evaluators least often endorsed practices related to the clinical interview and collateral information and bounds of competence domains” (p. 319).

Translating Research into Practice

“Evaluators who engage in a variety of diversity-related training opportunities were more likely to ask evaluatees’ their level of comfort in speaking with them. It may be that a variety of training opportunities helps to enhance evaluators’ approach to this communication practice. Alternatively, evaluators who seek a diverse range of training opportunities may be more open to exploring evaluatees’ comfort with evaluations in general. All evaluators had attended a diversity-themed training, and so we could not explore practice differences between those who had or had not. Instead, we explored the relationship between workplace training and evaluators’ reported practices. Workplace training was not significantly associated with any practice variables. It may be that evaluators seek training opportunities that enhance their practice outside of the workplace. This is especially the case for those in private practice, whose employment setting does not necessarily “provide” training under the umbrella of a larger mental health organization. It may be that overall number of trainings, frequency of trainings, and trainings that followed APA practice guidelines may better capture evaluators’ training experiences and help to clarify links between training and practice”  (p. 319).

“Evaluators are at a loss when quality training is scarce. Of concern, almost three-fourths of evaluators reported lack of training opportunities as a barrier to cultural competence in the workplace, and nearly half cited funding limitations as problematic. Work by other researchers suggests that increased access to resources is related to increased self-perceived cultural competence. We recommend that employment sites and conference organizations allot adequate time and resources to training in line with APA practice guidelines for both practical and ethical reasons” (p. 320).

“We find it critical that evaluators stay abreast of discussion surrounding cultural competence and recommend that they do so via the aforementioned workplace training, APA resources, and research databases. This task will grow in importance as researchers generate new findings and cultural dynamics evolve over time” (p. 320).

Other Interesting Tidbits for Researchers and Clinicians

“We must strongly emphasize that our study explored self-reported practices, which may not provide an accurate picture of the frequency with which these culturally competent practices are actually employed. As found in Constantine and Ladany’s (2000) study of psychology professionals and trainees, self-reported cultural competence may not translate to true culturally competent

practice abilities. Then again, research has demonstrated that clinicians’ self-report of culturally competent practices can be associated with increased patient satisfaction and sharing of information, indicating that self-report may reflect clinical practice in at least some circumstances” (p. 319).

“Future research should explore which practices elucidate important cultural considerations during forensic evaluations. Once these practice benchmarks are established, researchers may utilize preand post-training assessments—ideally measured through observation, rather than self-report survey—to identify effective training methods for developing and maintaining cultural competence” (p. 320).

“It is also important to note that our findings do not provide a better understanding of intrinsic and/or extrinsic motivations behind evaluators’ self-perceptions and pursuit of culturally competent practice. Researchers have found that evaluator characteristics, such as diversity orientation and social desirability, may contribute to self-reported cultural competence. Incorporating these additional evaluator characteristics into future research may be informative. As the linguistic landscape diversifies, researchers should explore the nuances of working with interpreters.  Development and validation of assessments with diverse populations is also a pertinent direction for future work” (p. 320-321).

Join the Discussion

As always, please join the discussion below if you have thoughts or comments to add!

Authored by Amanda Beltrani

Amanda Beltrani is a current graduate student in the Forensic Psychology Masters program at John Jay College of Criminal Justice in New York. Her professional interests include forensic assessments, specifically, criminal matter evaluations. Amanda plans to continue her studies in a doctoral program after completion of her Masters degree.