Advertisement

Exaggeration Nation

A new study finds behavioral researchers in the U.S. are prone to reporting extreme results.

By | August 27, 2013

WIKIMEDIA, ZINA DERETSKY, NATIONAL SCIENCE FOUNDATION, AND ANNA BAUER“Science is a struggle for truth against methodological, psychological, and sociological obstacles,” wrote Daniele Fanelli and John Ioannidis in a Proceedings of the National Academy of Sciences report published this week (August 26). The pair presented evidence for this struggle after analyzing nearly 1,200 studies in 82 meta-analyses on biological and behavioral research. Overall, Fanelli and Ioannidis found that behavioral researchers are more likely to report extreme effects, and those in the US are more likely to skew extreme when study results suit their initial expectations.

Fanelli, an evolutionary biologist at the University of Edinburgh, and Ioannidis, a Stanford University medical professor, called this tendency a “US effect.” They found that behavioral researchers from other countries might also end up with extreme findings, but not as often as those in the US. It’s not entirely clear why this observed bias might occur, nor why it seems more common in the US. “Whatever methodological choices are made, those made by researchers in the United States tend to yield subtly stronger supports for whatever hypothesis they test,” Fanelli told Nature. He and Ioannidis did not find similar patterns among studies in genetics meta-analyses.

The authors speculate that the culture of scientific publishing in the US nudges scientists to seek out methods that exaggerate their results. “Somehow the researchers there are subtly more pressured than elsewhere in the world to make strong discoveries,” Fanelli told The Guardian.

Chris Chambers, a psychologist at Cardiff University, was quoted in The Guardian as saying that he doesn’t believe US scientists are “actively engaging in mass fraud—instead, most of these questionable practices are unconscious. . . . It's easy to fool ourselves into thinking that a result which 'feels' right is in fact true. This problem is known as confirmation bias, which ironically was itself discovered by psychologists.”

One potential solution Chambers suggested centers on the idea of pre-registration, making researchers’ methods and hypotheses are laid out from the get-go. Fanelli proposed a culture shift, such that researchers are not rewarded just for publishing striking results in high-ranking journals, but for “research that is methodologically highly accurate,” he told The Guardian.

Advertisement

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: williamb

williamb

Posts: 1

August 28, 2013

I find that researchers from nations other than the United States tend to yield subtly stronger prejudice against research they don't hypothesis is true if it comes from US research.

 

Avatar of: Voduyan

Voduyan

Posts: 1

August 29, 2013

That is a much-needed reminder, in my opinion.

Avatar of: Roy Niles

Roy Niles

Posts: 56

August 29, 2013

Where's the evidence?  In my opinion, Americans are actually too quick to attribute remarkable powers to the accidental elements of nature, unwilling to accept that all natural systems and their functional elements are intelligently constructed - the remarkable ability for nature to intelligently evolve itself escapes them.

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Advertisement
Mettler Toledo
BD Biosciences
BD Biosciences