The title of the Aug. 28, 2015 article is “Estimating the reproducibility of psychological science.”
“Reproducibility is a core principle of scientific progress,” the article begins, then proceeds to destroy the believability of more than half of the tests of the older studies that were investigated.
From the abstract: “We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available.”
Researchers defined “the sampling frame as 2008 articles of three important psychology journals: Psychological Science (PSCI), Journal of Personality and Social Psychology (JPSP), and Journal of Experimental Psychology: Learning, Memory, and Cognition (JEP:LMC).”
Now, before you go to sleep, keep in mind that millions of education dollars are at stake, since wholesale changes in school curricula and advanced academic degrees are based on studies like these.
Furthermore, millions of dollars worth of psychoactive drug sales are also at stake, since new mental illnesses can be defined based on studies like these. No doubt more millions are at stake that we couldn’t determine, since no actual studies were described. Really.
“… publishing and analytic practices make it likely that more than half of research results are false and therefore irreproducible…,” was one conclusion.
After a long explanation of methods, much statistical measurement reporting, etc. – all aimed at heading of the inevitable barrage of attacks by alarmed academics, there was one clear conclusion:
A large portion of replications produced weaker evidence for the original findings (31) despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes.”
And here’s a sentence that would get the writer thrown out of any Tea Party convention:
“Because reproducibility is a hallmark of credible scientific evidence, it is tempting to think that maximum reproducibility of original results is important from the onset of a line of inquiry through its maturation.” Of course not, you silly goose!
“This is a mistake,” they say. “If initial ideas were always correct, then there would hardly be a reason to conduct research in the first place. A healthy discipline will have many false starts as it confronts the limits of present understanding.”
What a terrible gaggle of misleading sentences. The reason this notion is wrong is that we are not talking about the IDEAS, we are talking about the experimental RESULTS. The results can and should always be accurate. They may not substantiate the IDEA the researcher began with, but that’s what science is about.
In any event, this is the state of science today. The analysis above does not discuss political or financing issues, but it’s easy to extrapolate. Think Global Warming/ Change/Chaos, etc.
We’re betting that medical research is the least likely o experience these dismal results. To paraphrase what a friend said about another topic, “You can’t bullshit your way out of a patient death.”
Think about this the next time you hear, “The science is settled.”