Here’s a story on bad science that ran on the Inquirer's front page today (Tuesday). I got the idea from a fascinating paper in which researchers created a test case to show how dubious social science could pass acceptable standards of statistical significance. Others say the problem extends to medical research as well. Read the whole story here.
Would you believe a scientific paper that said listening to the Beatles song "When I'm 64" made people get younger?
It was tested by experiment, and the result came out with "statistical significance," which is the gold standard for incorporating new findings into the established scientific literature.
The point of the experiment was not really to test the youth-restoring effects of the song, but to show how too many dubious studies are getting published in respected journals. (If it were really true, Paul McCartney would get even richer.)
Wharton researcher Uri Simonsohn constructed the song experiment as a sort of test case, along with colleagues at the University of Pennsylvania and the University of California, Berkeley. They conducted a similar experiment showing that listening to the children's song "Hot Potato" made people feel older.
The researchers got both results using well-accepted practices for collecting, parsing, and analyzing statistical data, and both easily met qualifications for acceptance in peer-reviewed journals, said Simonsohn, a coauthor of the paper.
"It's unacceptably easy to publish statistically significant evidence consistent with any hypothesis," he said.
Read the rest here.