hackernews client

Scientists used the same data, but their politics predicted the results

29 pointsposted 2 days ago

23 Comments

jjk166

2 days ago

This is why the goal of experiments needs to be falsifying hypotheses. If you can make a model that fits the raw data without absurd assumptions which contradicts your hypothesis, then the data doesn't support your hypothesis.

If you're checking whether a coin is fair and you toss out significantly more tails than heads because you didn't feel they were proper tosses, of course you're going to reject the null hypothesis that the coin is fair. Even if your criteria for what was a proper toss is objective and reasonable, you're no longer testing whether the coin is fair, you're only testing whether your criteria for counting a toss is fair assuming the coin isn't.

When you construct your experiments to reject a hypothesis - this particular model can not be right if we see this - then you can make real progress towards truth.

chickenimprint

2 days ago

Then people will simply falsify the logical inverse of their current hypotheses. Preregistration is a more promising approach.

jjk166

2 days ago

Falsifying the logical inverse is fine. The key though is that showing the inverse of what you believe can not be true is much harder than showing that what you believe might be true.

Preregistration only helps with a select set of researcher bias, specifically where they retroactively change parts of their model to get the intended result. However it does nothing to protect against bias baked into the model ahead of time. Presumably the same reasons that researchers are strongly politically opinionated are the reasons they emphasize certain things over others in their models long before they see any data.

chickenimprint

2 days ago

Falsifying the logical inverse of X is identical to verifying X. There's nothing about negation that does anything here. You're making the same mistake people make when claiming "You can't prove a negative".

> The key though is that showing the inverse of X can not be true is much harder than showing that what X might be true.

This is nonsense modal logic. You're saying ¬◻¬X, which if necessity and possibility are duals, is equivalent to ◇X, and otherwise an irrelevant statement. The inverse of X is ¬X. ¬¬X is logically equivalent to X.

jjk166

a day ago

> Falsifying the logical inverse of X is identical to verifying X.

That's exactly the point. The issue with positivist hypotheses is that you can find evidence that supports, but does not actually verify, the claim. This seems convincing until you try to flip it around. So for example if a prediction of my model is that the sun will rise tomorrow, and the sun does rise tomorrow, that seems like it supports my model. But if my model can be wrong and the sun would still rise tomorrow, then looking for the sun rise was never going to answer the question.

If you are doing a test that will actually verify X then bias doesn't matter.

pepinator

2 days ago

how do you falsify a hypothesis in social sciences?

jjk166

2 days ago

Same way as the hard sciences - you make a prediction for something that you would never observe if the hypothesis is true, and then you go look for it. If you find it, then the hypothesis must be rejected. I might suspect that hunter gatherer tribes don't go to war, and I might observe many such tribes which don't, but that doesn't prove my hypothesis right. On the other hand, if I can find just one tribe which does go to war, then the hypothesis has been falsified.

pepinator

2 days ago

The problem is that there are no two equal situations in social sciences, so you won't ever have the same set of initial conditions. I don't know why they call them sciences, but the scientific method is intrinsically incompatible with social phenomena.

jjk166

a day ago

You won't ever have the exact same set of initial conditions in any experiment. You drop two balls from the tower of pisa and do it again you have different air conditions, the planet has moved, the tower may have leaned slightly more, etc. You either control for these variables or assume they don't matter. The value of science is that even if you drop two balls on the moon, you're still going to get the same result after controlling for differences - the results are broadly applicable. It's exactly the same in the social sciences. If you're looking at something real, it will show up consistently in repeated experiments with proper controls despite variation in irrelevant initial conditions. The phenomena that social sciences try to understand may have many more variables, but there's nothing inherently special about humanity that makes us impossible to describe.

xve

2 days ago

The foundation of science was built on a certain "political" view (the enlightenment). It's interesting that maybe science isn't exactly science without those values.

Voultapher

2 days ago

Science has a massive blind spot, one it can't fathom exists. For many today, especially on HN science is closer to a religion than what they themselves view it as. This is not a particularly popular believe since it contradicts a lot of nicely build up self-perceptions. Science can't figure out what is worthwhile pursuing and what isn't _without_ biased input at the very beginning of that chain. Science can't reason about the limits to it's power since it assumes that everything can be analyzed and broken into smaller problems. The fact that science is a tool, one profoundly incompatible with certain types of very real properties of our world does not fit into the religion of science. "Science can solve all problems and if it hasn't we just haven't tried hard enough". Science is a hammer that insists it is the right tool for every problem and if it doesn't work well you're just holding it wrong.

I was born into and shaped by a science and enlightenment religion world-view and lack proper words to describe the issues with it, but I feel them.

xve

20 hours ago

I sympathize with your feeling. All these questions are not going away, science can't do it, religion doesn't have those answers either. The religion of science offers the worst of both worlds. We are struggling to see something beyond these two options but I think there must be some other approach.

ratelimitsteve

2 days ago

when i took stats in high school our final had one set of data and two questions. one was to use that set of data to prove a hypothesis, and the other to use that same set of data to disprove it. the trick was manipulating confidence, and the real trick was realizing that this was almost certainly happening any time we allow someone else to make the critical decisions for how to crunch any particular dataset. there seems to be an unspoken translation layer: you ask a question in english, then use statistics as the process of translating that question from english to math, then apply the translated question to the dataset and get an answer. if translation were mechanical that would be fine, but often the english has to go through interpretation. How confident do we have to be in a "yes" answer? 99%? 95%? 50/50? What constitutes an outlier? Sometimes a very slight shift in the line between good and bad data will include or exclude a single data point that can shift the calculations. The problem seems to be standardization of this translation layer as much as anything else, which is to say that everyone is ostensibly trying to answer the same question but under the hood they're asking very different questions of the data.

wtcactus

2 days ago

No, they aren’t scientists, they are social “scientists”.

Physicists, geologists and other kinds of real scientists’ personal politics don’t determine the outcome of their research.

It’s high time we denounce and publicly defund social “sciences” for that they really are: astrology at best and political propaganda at worst.

pepinator

2 days ago

saying that social science is astrology at best is quite extremist. serious people work in those fields

smt88

2 days ago

I used to feel the way you do, but as I've gotten older, learned more about the fields, and met more researchers, I've come to disagree. Calling them science is a farce. We should put money into them and continue most of these areas of research, but we shouldn't pretend they're reaching firm, quantifiable, scientific conclusions.

The replication crisis really should have hammered this home to everyone by now.

pepinator

2 days ago

I agree with what you say, but that doesn't make social sciences the same as astrology. their conclusions, even if qualitative, can be valid and useful.

gus_massa

2 days ago

Assuming "zero" bias for physicist is too optimistic, let's say it's quite low.

wtcactus

2 days ago

I’m not saying they don’t have bias. But their bias is not political. We all have bias. From our supervisors, from our department, from the teachers we had. They have their own pet theories and they do imprint a part of them in their students.

But they aren’t dictated by political ideologies.

whattheheckheck

2 days ago

Science only worked because of the religion/politics ecosystem

bigbadfeline

2 days ago

> Science only worked because of the religion/politics ecosystem

In many cases what worked did so despite the religion/politics ecosystem.

Moreover, some science theories worked while others didn't. Some helped the ecosystem, others damaged it. At their current state of development, the utility of social sciences is between useless and severely damaging, depending on case.

And let's not try to hide faulty science by wrapping it in the generic term "science" which is riding on the good name of sciences with much better track records. What's faulty has to be fixed, not covered up.

csnover

2 days ago

> real scientists’

This is a classic “no true scotsman” fallacy.

Some people working in genetics allow their personal politics to draw conclusions about racial superiority. Some people working in medical science allow their personal politics to draw conclusions about the safety and efficacy of vaccines and other medicines, or about the health effects of certain diets or body weights. Some people working in geology allow their personal politics to draw conclusions about the age or shape of the Earth, or about the safety of extractive industries. Some people working in biology allow their personal politics to draw conclusions about the origin of life. Some people working in computer science allow their personal politics to draw conclusions about the likelihood of AGI and the emergence of machine sentience.

Does this mean genetics, medicine, geology, biology, and computer science “fake” sciences? Of course not. The fact that some people in a field are engaged in scientific malpractice doesn’t invalidate the premise that some subject is worth studying in a scientific way. There are many in the social sciences who publish research that runs counter to their own personal politics because they are, in fact, doing science.

Setting that aside, what even is the relevance of a distinction between “personal politics”, and, say, someone who is willing to accept money in exchange for publishing favourable research in any direction? Or someone who engages in fraud in order to feed their ego? Or people who spend 30 years down the amyloid plaque rabbit hole due to fraud and what appear from the outside to be very unhealthy group dynamics (which might not be so unhealthy if they took some cues from the social sciences)?[0]

But let’s set all that aside for a moment.

If we were to do as you propose and “denounce and defund” whatever you define as the “social sciences”, what method should be used instead to guide our lawmaking and personal decision-making about important questions that fall under that umbrella? Majority rule? Might makes right? Whatever fable we learn as children must be true and remain unquestioned?

What you are proposing is to take a system that at least attempts to be objective some of the time, and say it should be destroyed in favour of… what, exactly?

I cannot object to the premise that there is a lot of junk science in the social sciences. I wish it were better. It is deeply ironic that you are here using research from a social science field as proof for your claim that social sciences should be denounced and defunded.

Getting back to your original claim. Perhaps a big reason why something like physics may seem as though “personal politics don’t determine the outcome” is because most of it is sufficiently abstract that, today, there is rarely some direct conflict with any deeply ingrained cultural belief. Social sciences, on the other hand, usually point the spotlight directly on things people hold as sacrosanct. This is a double-edged sword, since it means researchers are also more likely to put a thumb on the scale—which is exactly what this research suggests. But there is no fundamental error in the idea that the scientific method can and should be used to look at humans and human systems, so a call to “denounce and defund” is reactionary nonsense.

[0] https://www.statnews.com/2025/02/11/amyloid-hypothesis-alzhe...

xve

2 days ago

So many of the questions we all really want answers to are in the social sciences area. While some of us want to see interesting work done in physics, humanity as a whole craves some kind of answers to all those junk studies that at least attempted to apply rigorous methods. Immigration is used as an example in this article. If studying a topic like that is off the table,what are we even left with?