Psychology is a questionnable science

Science is a process that begins with observation that leads to the formulation of a theory that is tested experimentally. Even if the result confirms the theory, there is another critically step that must be completed before the theory can be validated. The result must be independently reproduced. Reproducibility is a foundation of science. If an experimental result cannot be reproduced by an independent lab, the theory is unreliable and cannot be accepted as part of scientific knowledge.

Peer review is the process by which scientists review and evaluate the work of other scientists that is published in professional scientific journals.

A recent study published in the professional journal, Science, concluded that only 36% of 100 experimental results published in the top three professional journals in the field of Psychology could be independently reproduced. This result means that much of what psychologists assume to be true is not reliable.

In the Abstract of a research article titled, Estimating the Reproducibility of Psychological Science, the authors conclude,

No single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here.

Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.

The American Psychological Assocoation (APA) is already under fire for its continuing support of the use of torture enhanced interrogation techniques, even though the organization and the war criminals people who use these techniques have never been able to show that they have produced accurate information not already known. The results of this study cast further doubt on whether Psychology is a science or junk science.

There has been a lot of talk this year about the need to reform the criminal justice system. No doubt we will hear a lot more talk about the subject in the run-up to the election next year. Most of the discussion has focused on releasing nonviolent offenders from our overcrowded prisons. I agree that we should do that and I have also proposed that we officially end the failed War on Drugs and decriminalize the use and possession of all drugs, including heroin and cocaine, by following the successful model established by Portugal.

Based on my extensive experience as a criminal defense attorney specializing in death-penalty defense and forensics, I can assure my readers that we also have to tighten the foundational requirements for expert witnesses in Evidence Rule 702, as it relates to self-described mental health experts, particularly psychologists. One especially troublesome area is the prediction of future violence. Despite the absence of any evidence that a psychologist can more accurately predict whether a particular individual will commit a future act of violence compared to merely flipping a coin, prosecutors in death cases still put on their whore psychologist who predicts the defendant will commit future acts of violence. Jurors, who don’t know any better, respond with death verdicts. This crap needs to stop.

I will leave for another time a discussion about the woeful state of forensic laboratory work. Indeed, forensic fraud is one of the major causes of wrongful convictions. Just to provide a quick for example, Kentucky crime lab ‘experts’ still testify that a hair found on a victim’s body at a crime scene ‘matches’ a defendant’s known hair. The so-called match is based on a visual hair comparison using a stereoscopic microscope. They do this even though we have known for 20 years that the most can say about a visual hair comparison is that two hairs were contributed by two belonging to the same race. Further differentiation is impossible unless a nuclear DNA test (assuming a root is attached) or mitochondrial DNA (assuming a follicle with no root) is performed. Because the visual appearance of a hair differs substantially depending on what part of the hair is examined under the microscope. it’s impossible for an examiner to accurately conclude, unless he makes a lucky guess, that two segments of one hair follicle came from the same individual (assuming he does not know that the hair follicle was cut into two lengths).

10 Responses to Psychology is a questionnable science

  1. gblock says:

    As someone who has studied research psychology, I feel that I should comment.

    First, you should be aware that clinical psychology and research psychology are two different animals. Degrees are awarded not only through different programs, but programs in clinical psychology are often taught at completely different (and often, less known) universities.

    Second, there will be a certain number of people who will prostitute themselves any time there is money to be made by doing so.

    An important question is, why are so many research results not reproduced? Part of it is that psychology studies individuals who are each somewhat different from each other, in respects that may be relevant. There are many more things that can vary in psychology than in the physical sciences, so results won’t always come out the same. A different group may be different enough to give a different result.

    Another issue is that the way that the research is typically done, someone comes up with a plausible explanation and then creates an experiment to confirm or refute the hypothesis. So, someone could have gotten the result they wanted, but for the wrong reason. In many cases, a different researcher may later see a different reason for the result, and do a different experiment that may refute the first one.

    Another problem is that some research may just be sloppily done.

    The issue of novel results being more valued than trying to copy someone else’s experiment is one that should be addresses more extensively. The fact is, that in psychology, you can’t actually “know what you know” until a question has been addressed many times, in a variety of different ways.

    I’m happy that you have found some research psychologists that you respect. I think that you would find a lot more if you studied the field more extensively. Part of doing valuable research is being able to figure out what questions should be asked.

    • Thanks for providing us with more context. Because of the ease of self-publishing on the internet, I think we are seeing the results of more and more research published that way instead of the traditional way via referees who decide what to accept or reject for publication in professional journals. That’s not helping matters.

      I have used quite a few research psychologists for topics like the fallibility of eyewitness testimony and coerced confessions. I have also used clinical psychologists to administer psych tests, issue reports and testify in death penalty sentencings.

      • gblock says:

        Thanks for your comment. One additional issue is that clinical psychology has frequently been based on the individual clinician’s intuitions. There has been some discussion about trying to add more of a scientific basis to clinical psychology.

  2. Charles Ramsey says: There is a Yahoo group about this. I would like to see a lot more people on it.

  3. Bill Taylor says:

    majored in psychology and during college argued with the professors that it seemed we were being taught how to attain clients and make money rather than how to help people overcome their problems and be healed.

    • I do respect some psychologists like Elizabeth Lofthus and her ex-husband who did a lot of work demonstrating memory problems and the unreliability of eyewitness identifications.

      I also credit psychologists who demonstrated the unreliability of child memories and who explained how false confessions happen.

      Also I credit the work of some psychologists exposing the torturers and the horrors visited on GITMO prisoners.

      (Sorry, I’m blanking on their names right now)

  4. GB says:

    In the trial of Jodi Arias, the prosecution called a psychiatrist in the guilt phase, to make a questionable diagnosis of “Borderline Personality Disorder”, presumably in an attempt to shore up it’s very weak circumstantial case for premeditated murder ( even though there is no scientific evidence that suggests Borderlines are prone to premeditated murder ). It was bizarre.

    • Yep, borderline personality disorder has little to do with a person’s ability to form specific intent to kill, reflect on the decision to kill, and recommit to killing the targeted individual. That’s what it takes to prove premeditation. I suppose it’s possible to conclude that a person with that disorder might more easily premeditate to kill another person compared to ‘normal’ person’s thought process. On the other hand, I think someone could argue that someone with a borderline personality disorder would not even bother to reflect on the decision to kill and you can’t have premeditation without proof of reflection. Either way, the expert witness would be speculating. If I were the judge, I would have prohibited the witness from expressing an opinion regarding whether Arias premeditated or was capable of premeditating on the ground that, pursuant to Evidence Rule 702, speculation would not help the jury to decide whether Arias reflected on her decision to kill before deciding to ahead and kill him.

      Unless a defendant has confessed to premeditating a murder, the only way to prove it is via circumstantial evidence.

    • Malisha says:

      I found, during 18 years trying to help mothers whose children were being wrongfully taken away by mother-hating courts (most often to give them to fathers who had not paid their child support and who wanted that burden to go away) that “borderline personality disorder” is code, actually dog-whistle words for:

      “This is a woman you would hate if she were YOUR wife, so don’t worry about HER legal or constitutional rights; just call her some names and kick her case out.”

      “Forensic specialists” were on the market (many of them part of what was called “THE A TEAM”) who would charge huge amounts of money to make findings that these mothers had Borderline Personality Disorder. They would take one characteristic listed in the DSM and give one example of it; then they would take another example of something and compare it to something THEY claimed the mother had done or said or even THOUGHT, and presto-chango, they had a BPD case the judge could very easily say was an “unfit mother.” One psychiatrist at a conference in New Jersey announced that using all the data he had collected for ten years, he could now determine the CAUSE of borderline personality disorder: “In females, becoming involved in a custody dispute with a relatively wealthy man who can hire aggressive lawyers can give you BPD in less than six months, depending upon the court calendar.”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: