Significance

This past week (I am not certain how to define week anymore) the course focused on bivariate statistics. That is, the nature of the relationship of two variables. Or, a more precise "that is"- does a change in one variable lead to a change in the other? Does smoking more cigarettes lead to a higher chance of causing something (quantitative) and does being female result in being smarter (qualitative)?

This ties in nicely with statistical significance. Haaf et al, Retire significance, but still test hypotheses  really want to be done with significance. So much so, that they got 800 scientists to sign that they also want to be done with significance. Significance means that the results are more than just chance. Or so I thought.

In the article, they blame scientists for using significance as a kind of pass/fail criteria. As a result of significance being the end-goal, they claim scientists often massage data and cherry pick methods so as to prove significance. Finally, the arbitrary level of 0.05 shouldn't be used blindly.

The solutions they propose are: pre-register studies and print all results, don't be so confident and categorical but rather try to think through the data and realize uncertainty is built in (they reccomend "compatible intervals" rather than confidence intervals), calculate a p value accurately rather than blindly using 0.05 and be humble.

As scientists are known for their ability to be humble, I doubt the last recommendation should be difficult. And I really do agree that studies should be pre-registered. But the big problem is as follows: significance serves to add objectivity and steer clear from intuition. Of course it would be wonderful if people were thoughtful and clever and objective. And while many people can be thoughtful, really are clever, and try to be objective, I doubt significance is the problem. After all, if those who used significance incorrectly weren't able to learn what significance (a relatively not challenging term) means, why would they take the time to do more complicated statistics? Intuition is a big problem (as the reproducibility problem proves), and weeding out significance is just a cosmetic solution.

I realize non of this is physical and all of this is data... Oh well.