What prevents me from conducting a research privately as many times until I get the desired result? What can you even trust?
I got into a statistics course recently, and I've started to question the validity of research results. Related Veritasium's video.
Basically, is there anything stopping a researcher from doing this: Conduct a research --> get too big p-value --> conduct another research --> continue until you get low enough p-value. Publish result.
Then again, what prevents me from conducting a fake research and pull numbers straight out of my ass? Nothing.
I don't know how the structure around conducting and publishing researches work, but basically studies about important topics will be extensively reproduced, while in niche topics with few studies anything can be faked and published in a niche paper. Then again, these papers will get unreliable reputation in the research industry.
The only conclusion I've come to is that, the credibility of studies aren't based on facts, but reputation and trust in a set of research institutes that have majority support and hopefully aren't corrupt.
I don't believe that the main research scenes are that corrupt, mostly. However, on certain topics like political stuff, people get VERY riled up and eager to do anything to prove that their view is right. Thus I believe that there are a lot of researchers who are more motivated to prove their view than to conduct honest research to see how things are. So, it then becomes a war where each side tries to prove their view right, causing just bunch of contradictory results.
The problem is especially when there is a majority view vs minority view, the minority view just gets steamrolled and pushed out of reputable papers by the force of majority, regardless of what view is actually true. The validity of any research can't be verified. Major papers might just brush off minority views as fake research and not try to reproduce it, and the minority view will be left in the shadows unverified as unreliable information. Because, why waste resources on reproducing something that is clearly different from majority research (which has been conducted to support the majority view), and to risk the "wrong" view being true?
People see what they want to see, and with herd mentality, they won't even become aware of it. Just think of what kind of views people held few tens of years ago. Especially when it's about a majority view, I believe there is huge risk of biased research. Papers will accept and publish research results that they like.
What do you think? How to determine what research to trust, and how to determine what is actually true?
Posted May 24, 2019, 4:53 PM UTC