(2023-06-08) Alexander Attempts To Put Statistics In Context Put Into Context

Scott Alexander: Attempts To Put Statistics In Context, Put Into Context. There are many statistics that are much higher than you would intuitively think, and many other statistics that are much lower than you would intuitively think. A dishonest person can use one of these for “context”, and then you will incorrectly think the effect is very high or very low.

In last week’s post on antidepressants, I wrote:... ((2023-05-31) Alexander All Medications Are Insignificant In The Eyes Of God And Traditional Effect Size Criteria)

But we can do worse. Studies find that IQ correlates with grades at about 0.54. Here are two ways to put that in context:

The first way makes it sound like IQ doesn’t matter that much; the second way makes it sound like it matters a lot.

Or suppose that you’re debating whether there’s such a thing as “general” intelligence, eg whether students who are good at reading are also good at math.

The first effect might sound kind of trivial, but it is r = 0.86. And the second effect might sound immense, but it is only r = 0.64.

The real correlation between standardized reading and math tests is in between, r = 0.72.

It’s easy to cheat!

Obviously someone wanting to exaggerate or downplay the generality of intelligence could choose which of these two ways they wanted to “put it into context”. I don’t have a solution to this except for constant vigilance and lots of examples.

So here are a lot of examples. I thought I was the first to do this, but partway through I found some prior art. None completely satisfied me, but I’ve stolen a little from all of them

Effect Size: (interesting list)

Correlation (another interesting list, incl:)

Depression vs. anxiety: 0.64

SAT verbal score vs. SAT math score: 0.72

The same student’s score taking the SAT twice: 0.87