Document Type

Article

Publication Date

2016

Abstract

What makes a study valid or invalid? In 2013, the Hastings Law Journal published a law review article by law professor Deborah Denno entitled What Real-World Cases Tell Us About Genetic Evidence. This article questioned the validity of an article that we published in Science: The Double Edged-Sword: Does Biomechanism Increase or Decrease Judges’ Sentencing of Psychopaths? Denno’s trenchant critique focused on our use of experimental, rather than archival, methodology, and revealed a misunderstanding of the diverse goals of empirical legal research. One study, which in our case investigated the impact of biological explanations of criminal behavior on sentencing, is not meant to answer the universe of potentially relevant questions. This is as true in science as it is in law. Rather, experimental and archival projects complement each other by asking and answering different questions aimed at different forms of validity. We describe archival and experimental research methods, and then explain how their design impacts external validity, including concerns of ecological validity, robustness, and generalizability; internal validity; and construct validity. We appreciate Denno’s questions about external validity in particular, specifically asking how and under what conditions a particular set of experimental effects might occur in real court cases. However, the questions she poses do not challenge the internal validity of our study—that is, its ability to identify particular causal factors that influence judges’ ratings and sentencing decisions in the particular set of conditions and case features we tested. By explaining the tradeoffs between different forms of validity, this brief article may serve as a helpful tool for scholars in law, psychology, and other social sciences, as well as attorneys and judges who rely on empirical legal research in their work.

Download

Share

COinS