olooney
3 days ago
Good discussion, but this conclusion:
> I don’t know. Maybe a little.
is not the right way to interpret the null result (insufficient evidence to reject null hypothesis that it has no effect) because the prior for supplements that people are trying to sell you is so low. In the absence of clear, strong evidence, you should assume the the whole thing is just an example of motivated reasoning. People want nootropics to be real, and other people really want to sell you readily available powders by claiming they have nootropic properties. In that environment, they were always going to trying to concoct a similar narrative about some supplement, and it just happened to be creatine. Those efforts were always going to result in a handful of "positive" studies that turn out to be non-reproducible, maybe because of p-hacking, maybe because of publication bias, maybe because of outright fraud. This is what the literature always looks like for stuff that just doesn't work. If it did work - if the effect size was large enough that you could personally detect it in your own life - then the papers would be trying to put error bars around the effect size, not trying (and failing) to barely distinguish it from a placebo.
“If your experiment needs statistics, you ought to have done a better experiment.” - Ernest Rutherford
somenameforme
3 days ago
I think this is quite different because (1) creatine has been proven to be completely harmless for people of all ages absent some rare thing like a kidney disorder and (2) it has a bunch of clearly measurable positive effects.
So this is more like trying to determine whether ginseng is good at keeping away colds or whatever. Maybe it does (actually quickly searching this, it definitely does but let's pretend it's as folksy as it sounds for a minute) but even if it doesn't then it's harmless and has a bunch of other effects, so when one takes it - the whole possible cold prophylaxis can just be seen as an added perk.
chmod775
3 days ago
Issue being that if you took everything that's "probably harmless" at once, cramming your body full of supplements, you might just end up harming yourself with substances that are toxic above certain amounts (acute iron toxicity etc.). Also the more random stuff you take, the more likely it is that something you took actually does turn out to have been harmful later.
So no, you should not take something merely because it's "probably harmless".
JumpCrisscross
3 days ago
> So no, you should not take something merely because it's "probably harmless"
You and I should not. But I'm also not going to bother correcting someneone taking homeopathy for a cold. It doesn't do anything. But unless it contains heavy metals, it also isn't hurting.
When I found ivermectin in my father's fridge, on the other hand, I absolutely asked him what he was taking it for. When I confirmed he's also taking real medicine, I dropped it provided he told his doctor in case there are drug interactions. (There aren't. Ivermectin is pretty safe. That's why the nutters congregate around it.)
glhaynes
3 days ago
But as you hint at in your second paragraph, people do take them instead of real medicine. So in that sense, it can be harmful.
wuliwong
3 days ago
You are strawmanning what he said. What he said is much better summarized as if a substance has other beneficial effects and there is one that is potentially just placebo, then it is ok. That is completely different than your paraphrase.
zardo
3 days ago
Is there a particular rare kidney disorder that creatine is contraindicated for? Because kidney disease is not rare, something like 14% of US adults have kidney disease.
adolph
3 days ago
My lay person understanding is that supplemental creatine screws with a common lab test to screen for kidney problems. Let your physician know if you are taking any supplements so they can interpret your lab test results appropriately or order different tests.
somenameforme
2 days ago
I don't know the specifics beyond the fact that your kidneys play a key role in both synthesizing natural creatine and filtering out creatine from your body, and as thus is generally not recommended as a supplement if you have any sort of kidney disorder.
maleldil
2 days ago
Creatine raises creatinine in kidney function tests. For a healthy person, this isn't a problem, but if you already have high creatinine (from a kidney problem), it makes it harder to interpret test results.
thingortwo
3 days ago
This is basically "wanting" it to be true then it actually being there are so many variables and so many other things that even the minuscule benefit you gain from this heavily marketed science backed supplement could get completely offset by your other lifestyle problem like poor inconsistent sleep. I at one point was deep in this "optimizing" phase listening to all these pseduo-intellectual channels like "What I've Learned" and articles like the one linked and they always overblow the finding to support some conclusion. I tried creatine for full year and didn't felt any different no extra 20% energy nothing still fatigued after work same with fish oils all this stuff is at best either placebo or in case of creatine only tiny benefits that might only be felt by someone really at peak of their training.
The actual things that I now swear by and actually help is consistent sleep of whatever hours that doesn't make you feel sleepy in the day/night (depending on shift) since actual consistency matters more then timing and also in case of workouts really balancing the fatigue to intensity ratio. I understood this late and thought one should always push hard but there needs to be a balance. It's kind of like how cheetah can run really fast but only for 20-30 seconds and then it needs a 30 minute reset period or something. So if you go to failure or always at very high intensity it affects later sets and increases recovery period and in a way driving down overall weekly sets/volume which is main hypertrophy factor.
jliptzin
3 days ago
It’s not completely harmless, for me it causes heart palpitations that are so bad my watch tells me I’m in atrial fibrillation. Yes I know this sounds crazy. I’ve ruled everything else out. No, it is not creatine mixed with caffeine or whatever. Just pure creatine. No, I don’t have any other heart issues. I can give myself an irregular heartbeat just by having some creatine. It doesn’t happen otherwise. I have seen a cardiologist, he said it’s rare but it is theoretically possible.
golem14
3 days ago
Have you ruled out your supply is contaminated with something that causes your side effects?
That might be relatively likely.
piguin
3 days ago
Scientifically interesting, but it's kind of the same problem whether supplements are dangerous on their own or from contamination. Given how little value an unnecessary medication is it makes no sense to test it to eliminate a risk that is also eliminated by not taking it.
jliptzin
21 hours ago
No, I haven’t, I could theoretically try a new brand and see what happens
kingkongjaffa
3 days ago
Given creatine is present in several foods, do you have an intolerance to those as well as creatine in supplement form? Curious if it's a simple allergy or the dosage.
the-smug-one
3 days ago
Creatine breaks when cooked, so for the most part he's not gonna get too much from eating a steak or whatever. Taking 5g/day of pure creatine to satiate your stores might be too much for this particular individual. Anyway, the parent produces its own creatine as well!
chabska
3 days ago
> Creatine breaks when cooked
Into what?
NichoPaolucci
3 days ago
Smaller pieces, probably
coldtea
2 days ago
> (2) it has a bunch of clearly measurable positive effects.
Clearly measurably neutrally and scientifically -- or clearly measurable as measured by supplement company funding studies or cherry-picked by health influencers?
ffsm8
3 days ago
and the thing people seem to not understand: how exactly do you _expect_ to be able to notice this change? Its something that happens after you take it for a prolonged period of time (>7days) and only has a minor/medium effect.
It doesn't make you a genius. I think a better view on it is that it may mitigate some of the adhs symptoms (at least for me).
I can keep on track for a significantly longer time since ive started supplementing with it at ~15g/day - and while i dont feel a difference, i can observe it when actually paying attention to what i do / how often i drift off etc.
user
3 days ago
rubyfruit
3 days ago
People have no idea how successful big pharma’s info campaign has been since what I’m increasingly calling the real Web 3.0 (post-influencer social media and arrival of big corporate narratives).
According to Reddit, Epsom salt has no effect (doesn’t absorb) - you’re wasting your time by using it. They’ll cite all sorts of real, funded studies that agree and yet always the comment section is filled with anecdotal evidence of the contrary - to which a typical response is “hot water in general is soothing” (ignoring efficacy of magnesium sulfate in ice baths).
I digress - creatine is a similar situation. All this confusion at best. It’s also not unlike the health effects of cannabis just prior to going effectively legal - it either causes or cures cancer, or something? We need more research! That’s the real problem!
Any time you see this sort of narrative it’s corporate politics and influence at play.
By virtue of how hard it is to get any information on creatine, it probably threatens some market or vertical at least (whether or not it “works”). Like A. muscaria mushroom or sassafras, you’ll have to do your own research to really know.
coke12
3 days ago
The Epsom salt meme predates your “Web 3.0” thesis by many decades. I have no idea if it’s true but it’s not indicative of any new trend.
rubyfruit
2 days ago
It doesn't predate advertising. Web 3.0 just means advertising found a new home
asdfman123
3 days ago
Lots of state actors criticizing the US too. Bots everywhere.
red-iron-pine
2 days ago
30-50% of social media at this point
"look to the left of you, look to the right of you, one of you isn't real"
asdfman123
2 days ago
It's me. I'm funded by a network of stodgy older devs to be a shill for clean code.
rubyfruit
2 days ago
"clean code" is one of my favorite myths.
Especially when the guy so into code cleanliness can barely dress himself, can't design well, and so-on.
A lot more parallels between visual design / art and people who structure code nicely in files and use the "latest syntax" (omg so clean)
chabska
3 days ago
> By virtue of how hard it is to get any information on creatine
What exactly is hard about that? You can get all sorts of information about creatine, it is one of the most studied supplement ever.
nemomarx
3 days ago
if there's a strong measurable effect to any of these supplements, why can't it be studied? there are companies on both sides, so someone who sells creatine could fund one if they like.
rubyfruit
3 days ago
Magnesium? Its widely known that magnesium is used by the body lol
What they “doubt” is that it can be absorbed through the skin.
But anyone who takes magnesium tablets can tell you the effect is the same.
And it is studied btw - problem is there are countless on both sides.
JumpCrisscross
3 days ago
> Magnesium? Its widely known that magnesium is used by the body lol
Who disputed this?
> What they “doubt” is that it can be absorbed through the skin
The science is helpful. Magnesium sulfate is absorbed less effectively than magnesium chloride. Of course, you're not eating the amount of Epsom salts one dumps in a bath.
> And it is studied btw - problem is there are countless on both sides
This is why meta analysis exists.
rubyfruit
2 days ago
> Magnesium sulfate is absorbed less effectively
Drinking the same Epsom salt in water that you soak in has the same effect as soaking in hot or cold water with it, and it's also the same effect as taking magnesium supplements sold over the counter.
You can try it yourself - totally safe. You already eat magnesium all the time.
Whatever you're doing with your comment, I'm not convinced.
superhuzza
3 days ago
There is an incredible amount of evidence for creatine being beneficial for making you stronger/muscle growth. Which is probably the primary reason it's sold. So it's not as if it's not studied or there are no known effects of supplements.
pibaker
3 days ago
Reddit is shit tier source for all kinds of information about commercial products. Even subreddits for niche hobbies are full of astroturfed advertising for random online businesses. There is no way the supplement industry isn't doing its own campaign over Reddit and other platforms especially when you consider how large of a market it is.
But sure, keep believing they are on your side.
techteach00
3 days ago
That website examine is good for supplement info. They had to paywall most info because LLMS scraped the c$#p out of them m
cinntaile
3 days ago
They paywalled it before LLMs.
switchbak
3 days ago
What I'm getting from all this is that we have an armchair expert explaining details about something they're not an expert in, and we have you reviewing the way they present this information, saying their very weak hedge wasn't strongly negative enough. Based on priors about the industry.
Basically, HN doing HN things - having a good deep think about topics outside their realm of expertise, then debating the minutiae. I'd just rather trust someone who has spent their career researching this stuff, thanks (unless that's what you do - in which case please tell us more).
dundarious
3 days ago
"HN doing HN things" over someone saying "good discussion, however, one point about the conclusion" and having that point be very rational and still open to debate.
Is discussing such secondary but still highly relevant aspects verboten? Must every comment here be by a lead researcher in the specific field? Would a biomedical science researcher necessarily have any more expertise on these important secondary aspects?
"Trust me, I'm a scientitian". No thanks.
switchbak
2 days ago
They're not saying "one point" about the conclusion, they're saying that their statement isn't nearly confident enough, and that they would prefer that they say with strong confidence that there is unequivocally NO positive effect on cognition.
It is a hell of a thing for a random person on an internet forum to be so unilateral, without providing some kind of strong evidence (aside from statements about "the industry", which is kind of beside the point). I would also suggest if you want to be so unequivocal, that the burden of proof is especially high for you.
This "point" isn't actually all that rational - it's knee jerk skepticism. It's a fine default for yourself, but it's not something you get to force on others because they admit there's room for the potential of benefit.
Again: You're perfectly fine to hold that opinion yourself, but when you start trying to force that on others - you now have an especially high bar to reach, and yes - being a lead researcher in the field is exactly what I'm looking for when someone says "X has been proven to not have a benefit for Y".
"Trust me, I'm a scientitian" - Not sure the point of that juvenile wordplay, but Ok: we do have these things called scientists, some are good, some are terrible. You still need to back up your statements - that's the key part, this isn't a blind appeal to authority. And it's a hell of a lot better than "Trust me, I'm a nerd with very strong beliefs about a topic I know very little about".
colechristensen
3 days ago
The top of HN threads is generally nitpicking minutiae which are more likely technically correct than not but also highly irrelevant. It's done by non-experts and drowns out any more valuable discussion.
It'll be people arguing about exact language not being someone's preferred level, or overloading the importance and exact definition of a technical term that experts don't really care about, or blowing up some edge case or another.
dundarious
3 days ago
It wasn't a point about exact language in and of itself. It was a point about the conclusion, and how it doesn't account for highly relevant aspects of the publication record and industry. This was not "a better word would be" or similar. This was "the meaning was wrong, the thinking was wrong, and here's why".
colechristensen
3 days ago
Exactly my point. You are making an essentially meaningless distinction in attempts to "correct" what I said to be more precise in a way that does not matter at all with regards to what I was trying to say.
dundarious
3 days ago
My understanding from this exchange is that for you, large distinctions of meaning and reason are reducible and accurately described as meaningless language nitpicks, but regardless, you dress up the last post as if it's the opposite, so I will not respond further.
antonvs
3 days ago
It may be minutiae, but olooney’s point is valid: the evidence doesn’t seem to support saying “maybe a little”. Remove that statement from the conclusion and it’s on the right track, aside from the arbitrary IQ point calculation which is nonsensical.
Retr0id
3 days ago
If you're being academically rigorous about it, sure, but I think it's fine as an informal conclusion.
jdiff
3 days ago
Even as an informal conclusion I think it's important to know who you're up against and what their motivations are. Waving that away is how we get people like known, obvious charlatans like RFK Jr in any sort of position of power. I know this is nitpicking word choice but the informal conclusion is "Not likely, and not enough to matter for anything."
Retr0id
3 days ago
Personally I take creatine for entirely non-cognitive reasons, so it's reassuring to know it's probably not making me any dumber.
hombre_fatal
3 days ago
Though you also have to consider potentially adulterated powder (accidental or not) in the unregulated supplement industry.
That's why I dropped creatine since I didn't notice any benefits when I tried it for a year.
somenameforme
3 days ago
With most industries companies live and die by their reputation. Find a bulk provider with a positive reputation and the creatine you buy is almost certainly going to be safer than e.g. eating at Taco Bell. And with creatine in particular it's so dirt cheap that a lot of motivation for dodgy stuff goes away unlike e.g. with protein powders where amino acid spiking can save a company a bunch of money. And incidentally creatine is one of the substances dodgy companies use for amino acid spiking, just because it's so cheap!
That said it's obviously the right decision to drop it if you're not seeing any effects. The interesting thing about creatine is that people's bodies are very different in their natural levels of creatine, and if you're already at 'saturation' then more creatine isn't going to do anything. Same reason the typical benefits of creatine don't show much difference beyond 5g/day. That's enough to send the overwhelming majority of people towards 'saturation.'
bhelkey
3 days ago
There is a brand of creatine manufactured in the US (con-cret) and a brand manufactured in Germany (Creapure). Either of those companies should address the concern around adulterated powder.
> That's why I dropped creatine since I didn't notice any benefits when I tried it for a year.
Some percentage of people (~20%) are creatine non-responders [1]. It's pretty reasonable to try creatine and then drop it if you find no benefit. I personally find the gain very marginal.
econ
3 days ago
For me it has an insanely large effect. Roughly 30% more reps and endurance.
zorobo
3 days ago
When I stopped taking it, I went from 12 reps to 10, and slower recovery.
My level at tennis dropped too, but bicycle endurance remained the same.
I stopped because of high blood creatinine, and wanted to see what baseline was
infinite_spin
3 days ago
It's not unregulated.
https://www.ecfr.gov/current/title-21/chapter-I/subchapter-B...
NopIdoN
3 days ago
it might even be counteracting the negative effects of your frequent contact with airborne nitrogen
infinite_spin
3 days ago
what negative effects? N2 is inert under physiological conditions.
NopIdoN
18 hours ago
nothing it's fine if you buy enough creatine. It's much easier with my proprietary blend though
infinite_spin
3 hours ago
You still haven't specified what negative side effects you're referring to. Nitrogen tends to be inert, so I'm questioning your claim that N2 causes some negative side effect, not your claim that creatine hinders this yet-unnamed side effect
thisisit
3 days ago
You do realise that you cannot prove a negative? So the right conclusion even informally cannot be “ probably not making me any dumber”.
infinite_spin
3 hours ago
Yes you can. E.g. Euler's proof that there is no largest prime number.
ttul
3 days ago
What value is there to an informal conclusion when the topic is as important as intelligence? The downsides to creatine if taken without good scientific backing should not be disregarded, yet that’s exactly what many ordinary people are now doing because of “informal conclusions” like this.
overgard
3 days ago
Are there really downsides to creatine though? (Genuine question). All the advice I've seen makes it seem like it's basically just good for you even if it doesn't accomplish some specific thing. The only downside I've heard is maybe water retention?
ttul
3 days ago
That's not the point. Often in medicine, the downside of using one thing is that you're not using another thing that would be safer and more effective. When people chase results from unproven therapies, often it means they're avoiding doing something else that is known to produce a result - even if the result of the alternative is not as spectacular as the unproven therapy promises to be.
But speaking of creatine itself, while not inherently toxic, consuming large doses can cause serious indirect harms, such as triggering falsely elevated kidney markers on blood tests that lead doctors to misdiagnose organ failure, unnecessarily cancel crucial medical treatments, or order invasive biopsies. Furthermore, overloading the body with creatine while taking kidney-stressing medications (like heavy doses of ibuprofen) or having an undetected renal condition can cause actual acute kidney strain, while the massive shift of water into muscle cells can trigger severe dehydration, heat exhaustion, and electrolyte imbalances during intense heat or exercise if fluid intake falls behind.
But, I don't want to belabour the point about creatine. Rather, I want to focus on the negatives stemming from NOT doing other things that ARE proven to be safe and effective. People tend to think creatine will solve a lot of their problems (for instance, they may believe it will fix their feeling of being "low energy). But by avoiding things that might actually help their problem get better, they're persisting a poor state of health.
freehorse
2 days ago
Taking creatine does not bar you from doing other things. Being "low energy" is too general a thing, and to approach you probably need to start eliminating factors anyway until you find the issue, by getting better sleep, doing blood tests etc. Taking creatine can be part of this.
Creatine supplementation not having significant effect in X domain in population level does not mean that it will not have effect on YOU if for example you are vegan and don't get enough creatine from diet. If you have low energy, probably you have something that may not show in a population level. Similarly you may be low in B12 and then B12 can help you, but it may not show effects in population level if most of the population is not low in B12 (and your study is not powered enough to catch nuances).
It is not like when you have to choose between two medicines and excluding one can be harmful even if the other is not harmful to take per se. You can supplement creatine and try whatever else at the same time.
CGP_Labs
3 days ago
[dead]
freehorse
3 days ago
What are the "downsides"? It is one of the most studied substances and except in special cases like kidney issues it is totally safe.
artrockalter
3 days ago
The null is surely different in this case because we know that supplemented creatine goes to the brain and the brain uses it, which is definitely not a claim all supplements can make.
The fact that there's a biologically plausible mechanism plus mixed statistical evidence makes "maybe" a perfectly reasonable conclusion.
Zababa
3 days ago
I think the way you interpret the null result is downstream of considering creatine as "a supplement". You can make the null result say anything by changing your prior about creatine or supplements. That's the issue with priors.
You also seem to reject the possibility that some things help just a bit. The author has another article in the same vein about "things that can help maybe a bit but the evidence we have doesn't really help detecting small effects" https://dynomight.net/vitamin-d/
Ker102
3 days ago
You are essentially trying to say that most studies regarding supplements are not valid
taeric
3 days ago
Heavily agreed. The supplement industry has such a low level of reliability that it has really soured me on so many practices people have. And I flat out do not understand how some of the supplement stores survive.
PaulHoule
3 days ago
Hard sell. I remember the time I went to GNC and they gave me a free sample of the notorious Jack3D which had some kind of amphetamine-like stimulant in it and was associated with young soldiers in the US Army falling down dead.
antonvs
3 days ago
They survive because people want to believe. Look at the history: the phrase “snake oil” exists because people actually sold products by that name as a remedy. There were also tonics based on radium.
“There's a sucker born every minute.”
fragmede
3 days ago
The story of how the phrase snake oil came to be is a fascinating one. Clark Stanley, an American, ripped off a Chinese product which was actually a decent anti-inflammatory, and didn't put any snake oil in his snake oil, which is why it didn't work!
Johnny555
3 days ago
>And I flat out do not understand how some of the supplement stores survive.
I think mostly through placebo effect, people buy a supplement to treat condition X and they think it's getting better, so they keep buying.
I mean, just look at the wide variety of homeopathic "cures" that are often so diluted that there might not be any molecules of the active ingredient left. Yet people keep buying it.
abirch
3 days ago
mandatory XKCD https://xkcd.com/882/ "Jellybeans cause acne"
mrexroad
3 days ago
I was visiting a friend’s lab at NASA Ames the other day. I smiled seeing this on the wall there. I sometimes daydream about what it’d be like if our primary education system devoted an hour a week to reading / discussing comics (well, XKCD specifically), or pages form What If…? Maybe it’d kill the magic of it, but I still feel plenty of seeds would get planted along the way for more critical thinking later in life.
0xdeadbeefbabe
3 days ago
I think you are on to it. The magic would die. It's a recipe for dystopia same as the Agile Manifesto was such a recipe.
pvaldes
3 days ago
Just the green ones. If they found a link, they found a link
I'm sure that the comic wants to be sarcastic but is missing the mark here. This is how science works. Trying and failing every color except green, does not mean that the green experiment has an invalid result.
The author is not understanding that each experiment is independent.
The correct cartoon should show researchers trying green jellybeans 100 times, with 99 negative results and one positive result, and then the press keeping only the positive (Or in this comic, claiming that all jellybeans produce acne).
Mocking the scientific method, and showing scientists as lazy people that play video-games is funny, but is just a cartoon, not the reality. People should not take XKCD as god's speech without any critical thinking. This in your workplace says to me "I don't really understand statistics".
There is an effect that is statistically rare just by chance, AND is consistently absent within a huge control group of "non-green jellybeans" (the author seems to think that this weakens the result, but in fact is reinforcing it). And it needs further research. Not less, not more.
dpark
3 days ago
> If they found a link, they found a link
You could have skipped the wall of text. They didn’t find a link. They found a random correlation.
The scientists here essentially engaged in P-hacking: https://en.wikipedia.org/wiki/Data_dredging
Munroe didn’t misunderstand the independence of the experiments at all. He understands that misleading spurious results can arise from repeated experiments.
> And it needs further research. Not less, not more.
No it doesn’t. At least not anymore than the other colors. A positive result is no more deserving of further investigation than a negative result. What you are describing is bad science.
pvaldes
3 days ago
This is exactly my point. They are not repeating experiments. Each one is a different one, and independent of the other, because the chemicals in each color are different. The cartoon don't show any data that proves that this is a random correlation, because they are not testing what you think that they are testing. Jellybeans here are irrelevant (They are present in both groups, the control and the experiment). It may be, or not be, a random correlation, we can't know at this point. And this leads us directly to the next line:
> A positive result is no more deserving of further investigation than a negative result.
This is not how science works.
Your null hypothesis is rejected, you publish and continue trying to explain why. Otherwise you stop and try something different.
Films have sold us the idea that science can be done in the countdown of a bomb, experiments take 30 seconds; and if it does not work in the first try, it means that you are not a real genius. The weird guys build a time machine in a weekend with empty cans of soda, and the hackers broke into high security buildings with a paperclip and then remove their glasses and nobody recognizes then anymore. Yeah. Master of disguise. This Big Bird costume pays for itself every time we break into the Pentagon. G'morning Mr Hegseth
But this is a fantasy, not the real life. Scientists stumble and limp all the way. Even the geniuses. Specially the geniuses. And they faint at the price of the required spare part that must be ordered and imported from one, and only one, company in Finland, and will be here in five months. Meanwhile, researchers throw things to the wall to see what sticks, and laser focus on the unexpected strange result that is different in a subtle way, because their grants depend on it.
dpark
3 days ago
> This is exactly my point. They are not repeating experiments. Each one is a different one, and independent of the other, because the chemicals in each color are different. They are not testing what you think that they are testing.
They ran the same experiment 20 times with a meaningless variable changed until they got a positive result with a confidence of 95%. (i.e. 1/20 chance of a random positive result.)
The point of the comic is that this is bad science. Or at least it is ver bad science reporting. This kind of nonsense is why there is a multibillion dollar market for supplements that do not hold up to further testing.
> This is not how science works in the real life.
True. There is a lot of bad science in real life.
pvaldes
a day ago
"Scientists bad because I don't understand statistics" is boring. Just because you decided that is a meaningless variable does not mean that it is.
P-hacking doesn't mean doing a lot of experiments. This is called working hard or going the extra mile. Working hard is not cheating. That the real scientists would do is frown upon the green jellybean result, think "hum, this is strange" and test further for effects of the green dye in other products. Science knowledge is in-cre-men-tal.
P-hacking is: changing the statistical model that you use to analyze the data, until you find one that gives you the desired (publishable) p-value.
P-hacking is manipulating the methodology in your favor and sometimes grooming the data also. All of that while ignoring other models available known to be more robust, precise and better for your particular case. This is why p-hacking is cheating.
-------------------------
Second. There are a market for fake supplements because, to express it crudely: people is idiot and the government does not care about them.
1) Some people didn't cared to learn even basic biology at the school, and will believe that the moon is made of cheese if it comes from somebody in their tribal team. Scientists take a lot of pain to assure that their knowledge is available to the public.
There are thousands of scientific articles about creatine. Never has been so easy in the history to find them. And with AI, you don't even need to read the whole horror anymore, to have a general idea of cons and pros before to jump in the water. Do your f*k homework before to gulp a mysterious pill. If not. Don't blame scientists later.
2) The government make laws. Is not scientists fault, if it allows to sell legally dubious supplements, or even more if politicians profit of it, (ehum, mandated testosterone tests for every soldier, ka-ching!).
Their job is to protect citizens from scammers. Most countries ban anything that is remotely medical and not proven effective, or at least advice strongly against taking it.
dpark
a day ago
> "Scientists bad because I don't understand statistics" is boring. Just because you decided that is a meaningless variable does not mean that it is.
It’s literally a comic about how spurious results arise.
I understand what you are trying to get at with your insistence on pedantically misreading the comic, but you’re missing the forest for the trees.
> Your null hypothesis is rejected, you publish and continue trying to explain why.
In this case the “why” is “we ran so many experiments that a false positive was inevitable”.
> Otherwise you stop and try something different.
“Only publish the positive results” is exactly the problem. This is terrible science because it magnifies false positives and hides the negatives (false or not). Do enough experiments and eventually you get a positive result for basically anything.
jrm4
3 days ago
Okay, but it kind of sounds like you want to nanny-state the writer's language.
Their conclusion seems like an absolutely fair interpretation of what they saw, and it also strikes me as dangerous -- perhaps more dangerous -- to suggest "writer genuinely believes, or doesn't believe, a thing, but they should say it differently from their own interpretation because..people are dumb?" Something like that.
jdiff
3 days ago
One person is not a nation.
jrm4
3 days ago
It's a metaphor.
jdiff
3 days ago
What kind of metaphor? You can be a metaphorical nanny state of something smaller like a household or comment section, I suppose, but even then not with a single statement. The problem with a nanny state is the scope. And people expressing contrary opinions isn't nannying.