john-h-k
7 days ago
Has anyone here actually read the paper? Heavily clickbait headline, completely exaggerating the study results, which themselves are not very rigorous (to be generous)
Tiny sample size - 36 people completed
Numeracy has R^2 = .27 Language has R^2 = .31
They then run stepwise regression to determine variance contributions, seemingly ignoring their earlier results, and this leads to almost no contribution from numeracy. Why? Because they have ~10% shared variance and stepwise regression is greedy - will just take whatever you give it first.
I can't mention this part enough. If you got a second, very similar language test, and added it to the model, you would _also_ find it has almost no unique variance added.
Every thing they measure is incredibly noisy and they do not once attempt to deal with this. Human based reviewers, time-to-completion, etc.
p-value for "language learning is more significant than numeracy" on the values they give (Steiger test) gives 0.772. Utterly insignificant.
Also, to beat the point home, just think about the argument here: * Numeracy contributes 27% of variance
* Language skills contribute 31% of variance
* After regression step, numeracy contributes only 2% of unique variance. Because you added a correlated variable!
hansvm
7 days ago
To be somewhat fair, their numeracy scatter plot visually looks more like a dart board than any real process. The fact that you got a positive-sloping line out at all from the regression has to do more with the positions of the outliers than anything else. You're at least able to visually examine the language aptitude plot and see an up-and-to-the right connection between the two variables.
john-h-k
7 days ago
I agree, but also via some napkin math, the chance of getting those different results is something like 40%.
(Ie, if you sample the same signal twice for the numbers they did in the study, there’s a 40% chance it’ll be >0.04 away from the original sample, as numeracy was from language)
SanjayMehta
7 days ago
It reads like an ad for something.