Information Processing: Is your research replicable?

Tuesday, March 31, 2015

Iamthep said...: see Mina Bissel's "The Risks of the Replication Drive".

Of course I disagree with her. The very fact that these experiments are so hard to replicate should be a huge red flag. Biology needs to change how it works, and I think it is happening. At least the interviews of George Church give me hope. Biochemistry is certainly getting tied more and more closely with data science.; 11:00 PM
Alex Lamb said...: Machine Learning has issues with replicating results, because it's sometimes easy to get good results on a fixed test set by tuning hyperparameters on the test set and then claiming that the hyperparameters were selected using some theory or intuition. The social sciences probably have a similar problem.

The community gets around this to some extent by running contests where contestants can't see the test data while they validate their models.; 3:33 PM
AG said...: Replicable result is only sure way to prove the truth. When the research result is not replicable, there is serious doubt. It could be original research at fault or simply imcompetent repeaters who are no better than `Myth buster show'. If Myth busters could not make an Atom Bomb, nuclear weapon is still real.
However, verification is fundamenlly important for scientific truth.; 4:18 PM

Information Processing