(2021-12-20) Alexander The Phrase "No Evidence" Is A Red Flag For Bad Science Communication

Scott Alexander: The Phrase "No Evidence" Is A Red Flag For Bad Science Communication. Every single one of these statements that there was “no evidence” for is currently considered true or at least pretty plausible.

I don’t think the scientists and journalists involved in these stories meant to shrug and say that no study has ever been done so we can’t be sure either way. I think they meant to express strong confidence this is false.

Science communicators are using the same term - “no evidence” - to mean:

• This thing is super plausible, and honestly very likely true, but we haven’t checked yet, so we can’t be sure.
• We have hard-and-fast evidence that this is false, stop repeating this utterly debunked lie.

This is utterly corrosive to anybody trusting science journalism.

Unfortunately, I don’t think this is just a matter of scientists and journalists using the wrong words sometimes. I think they are fundamentally confused about this.

In traditional science, you start with a “null hypothesis” along the lines of “this thing doesn’t happen and nothing about it is interesting”. Then you do your study

This is a perfectly fine statistical hack, but it doesn’t work in real life. In real life, there is no such thing as a state of “no evidence” and it’s impossible to even give the phrase a consistent meaning.

Real truth-seeking is Bayesian. You start with a prior for how unlikely something is. Then you update the prior as you gather evidence. If you gather a lot of strong evidence, maybe you update the prior to somewhere very far away from where you started, like that some really implausible thing is nevertheless true. Or that some dogma you held unquestioningly is in fact false. If you gather only a little evidence, you mostly stay where you started.

I think you have to go back to the basics of journalism: what story are you trying to cover?

Breaking an addiction to false certainty is as hard as breaking any other addiction. But the first step is admitting you have a problem.

I think the most virtuous way to write this is to actually investigate. If it’s worth writing a story about why there’s no evidence for something, probably it’s because some people believe there is evidence. What evidence do they believe in? Why is it wrong? How do you know?