As a counter-argument, HCI was investigated pretty thoroughly in the 80s and 90s, and operating systems of the time actually had the results of that well implemented in them. I feel that modern OS developers seem determined to throw away all these lessons.

Don't get me wrong, I think the modern HCI on mobile phones is remarkably good. But I haven't seen any improvement (except maybe the mouse scroll wheel and having a higher resolution screen) on real computer interfaces since the 90s.

And then you have some real useful psychological theories on attention and user-guiding that are used for evil to create antipatterns. I don't think we're making progress.

This is especially important in industrial settings. If a machine operator makes a mistake it's not just expensive, it can cost lifes. There where instances where operators actively fed fuel into fires because they misunderstood the situation displayed on the HMI.

Some time ago I found a really nice presentation about the ISA 101 standard covering this topic. The basic idea is: The HMI looks boring everthing is okay, if something goes into a dangerous direction colors and other elements are used to draw your attention.

> there are hundreds of scientific papers about collective intelligence, how to make teams online more effective, how to socialize newcomers to online sites,

I'm curious what research there is about how to create better-socialized groups of people in general; obviously some cultures are more successful in certain areas than others, despite starting with basically the same human genetics--is there any evidence that a culture can learn/adapt in intentional pro-social ways? How does a society learn to be less corrupt over time? How do people decide to stop littering/speeding/parking illegally? How does a society develop a respect for their environment, for their neighbors, for future generations, etc.?

It's really telling how it's much easier to progress when things that you are working on are directly measurable rather than self-reported or estimated through proxies.

Also progress in any science is contingent on progress in technology. There's only so much you can figure out before you'll need new, more precise way of measuring things to go on any further.

> perceptual psychology research in AR and VR too

That sounds interesting, would you mind sharing where would you point me if I wanted to follow up with latest research?

Something like arxiv but for psychology? Unless it' only in the magazines ("Psychology today")? I'd be happy to hear the magazines names too, if you'd be so keen to share.

Thank you very much!

In the broader category of cognition, I think we understand a bit better how people rationalize their decisions. How many things we do almost entirely on pure reflex and then manufacture a story that explains it without sounding crazy or just saying “I don’t know.”

My suspicion is that in these areas, of "nuggests of knowledge" psychology studies kinda work well and can be applied piecewise to it.

But I feel that anyone that think psychology will be fully predictable, or even up to the standards of medicine today will be for a disappointment

(but oh well, they can still run their experiments on Grad Students or Amazon MK workers and get another grant)

"I'm willing to bet that the results of a majority of psychology studies are not reproducible"

Indeed

> Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).

https://en.wikipedia.org/wiki/Replication_crisis

The author talked about it in length in an other blog post. https://www.experimental-history.com/p/im-so-sorry-for-psych...

This is basically just scientific fraud, no?

I suspect that after 400 years of the scientific method, that we may be reaching the limits of single variable experiments in a number of fields. Statistical methods can find those patterns, and as we advance in those areas I expect us to advance in messy sciences like psychology. We’ll be able to more reliably look at people or other chaotic systems and see how three inputs work together to create a single effect.

partially disagree with this, every proto-science historically had a bunch of wrong but highly sophisticated theories. medicine, alchemy (as mentioned in the article), physics, biology (Aristotle), astronomy. for some reason it seems you need the wrong theories to organize the empirical data.

I actually think Freud’s elaborate mental structures have some of this feeling to them.

Astronomy: you might go with Galileo Galilei, and you wouldn't be too wrong. [0]

[0]: https://en.wikipedia.org/wiki/Galileo_Galilei

  > there needs to be a long period of existing as a proto-science where people aren't doing experiments and are just observing and describing.

  > before darwin

There was an understanding of natural selection even back to antiquity. How could there not be? Did people not tame the animals and plants? These are experiments, and they saw the results.

There were great contributions to astronomy long before Kepler. There were many experiments that influenced the whole field. There was a lot of important chemistry that happened long before Lavoisier (conservation of mass) and Dalton (atomic model).

The proto-sciences are nothing to scoff at. They aren't useless and they weren't ill-founded. They were just... noisy (and science is naturally a noisy process, so I mean *NOISY*). There's nothing inheriently wrong with that. The only thing wrong is not recognizing the noise and placing unfounded confidence in results. That famous conversation between Dyson and Fermi discussing von Neumann's elephant wasn't saying that Dyson didn't do hard work or that the work he did had no utility, it was that you can't place confidence in a model derived from empirical results without a strong underlying theory. You'd never get to that if you only observed because you'd only end up making the same error Dyson did.

Science, in its nature, is not about answers, it is about confidence in a model that approximates answers. These two things look identical but truth is unobtainable, there is always an epsilon bound. So it is about that epsilon! Your confidence! So experiments that don't yield high confidence results aren't useless, but they are rather just the beginning. They give direction to explore. Because hey, if I'm looking for elephants I'd rather start looking where someone says they saw a big crazy monster than randomly pick a point on the globe. But I'm also not going to claim elephants exist just because I heard someone talking about something vaguely matching the description. And this is naturally how it works. We're exploring into the unknown. You gotta follow hunches and rumors, because it is better than nothing. But you won't get anywhere from observation alone. Not to mention that it is incredibly easy to be deceived by your observations. You will find this story ring true countless times in the history of science. But better models always prevail because we challenge the status quo and we take risks. But the nature of it is that it is risky (noisy). There's nothing wrong with that. You just have to admit it.

By forcing formal study of the mind into the constrained methods used for studying the physical world, it allows the government and profit/power seekers to be the only actors free to use the methods that work best.

I wonder if this is purely a coincidence.

Speaking as a tenured professor of clinical psychology, this part kind of irked me a bit. It's not exactly false but it's a little misleading (like some other parts of the essay).

Lots to say about it but this is a finding that has been reported intermittently for decades. However, it's being spun a little misleadingly.

Note that the author says that untrained professors were selected for their ability to be warm and empathetic. It's not everyone (we all know not everyone is warm and empathetic), and even trainees learn very very early (like immediately in their first term) the basics of therapy. Not everyone is warm and empathetic, and people going into clinical psychology are sort of self-selected in their empathy to start with.

This research is kind of being taken out of context too. Wampold, one of the authors cited (who I have the greatest respect for) is very big on "nonspecific factors", meaning things like empathy, good social skills, and so forth. His studies in general tend to be focused not on "does training matter?" but "do specific therapy protocols matter, or is it about the clinician's social/relationship skills?"

If you want some kind of medical standards, you can't just say "oh it's ok, everyone can just be warm and empathetic". You have to train on it, grade it, hold it to some standard. Otherwise you get manipulative, self-serving therapists who do harm in the long run (the length of a study versus real settings is another issue).

Another issue is that many of these issues are not unique to psychology. In lots of medical scenarios it's been shown that the amount of training needed to competently do a wide variety of procedures is lower than current standards in the US require. Experienced clinicians in many fields have acquired biases that interfere with practice, young trainees are much more worried about performance and are more open-minded and so forth (on average a little; not trying to stereotype).

A huge, enormous volume of studies over many years have shown that therapy works compared to all sorts of placebos and controls; that some therapists are reliably better than others; but that what makes therapy "work" overall is not what protocol-driven therapies (CBT etc) assert. It's not so much that training isn't necessary, it's that the field has has been obsessed with scientific details that, although well-intended, don't matter, and healthcare in general is full of phenomena that we'd rather not admit.

As much as that’s an eye-catching headline, even the author admits it was a bad study that hasn’t been reproduced.

> professional training as a therapist does not produce a statistically significant improvement in ability to treat mental health conditions.

It produces a statistically significant improvement, just not with people who are already gifted at it. You can get not gifted people and teach them to be not worse than gifted. It is not much, but it is not nothing either.

Where did it say the education was ineffective? There are reasons to believe it is not the only path to being effective at helping others, but that does not invalidate that if you spend a few years learning tools and techniques and pattern matching to behaviors, you have a valid toolkit in front of you for being a therapist.

Now, it is a valid argument whether or not it should be required (and there is no requirement to label yourself as a "coach"), and the price tag on it is of course always a consideration. But being dismissive of higher education is just as silly as being overly dependent on it.

Part of the problem is the therapists (and medical practitioners in general) are often forbidden from doing the thing they were trained to do for a variety of reasons: risk and liability, patient turnaround, standardization. These things can get in the way of doing the right thing in the times where that is known. That’s before considering the ambiguous cases.

I think a lot of people just never find the right therapist and then assume all therapists are terrible.

It’s interesting because even the most staunch opponents of mental health talk therapy have people in their life they talk to, they just don’t consider them therapists.

There was another research that indicated that the therapeutical framework that the therapist uses has no influence on the probability of postive outcome. What mattered more is whether a therapist was able to form a meaningful connection with a patient.

Given that “humans are complex” and “it’s individualized”, would advancements be greater and faster by just allowing clinicians and scientists to just talk things out instead of coming up with “studies” which pretend to be “science” with a low reproducibility rate (and non-publishing on null results)?

> Meaning that a person who kills somebody else definitely has mental issues that come from their childhood. So what about parents in those cases, aren't they having their part on the sick mentality of their child? Why not pressing charges to them?

Even assuming the premise of killing someone implies mental illness, and assuming the mental illness stems from trauma, there's a pretty large leap in reasoning here. Why must the trauma come during childhood? Why not in adulthood? Even in childhood, why does it have to come from the parents?

Then you have the idea of continuing up the causal chain. Why are we pressing charges to the parents? If the parent's traumatized their kids, there's good chance they did it due to their own mental illness/trauma, which means the parents themselves were abused in childhood. So we should go after their parents.... except that just means we should go after there parents.... ad infinitum.

> a person who kills somebody else definitely has mental issues that come from their childhood

People that kill other people often function well in their society. It doesn't make sense to me to classify that as mental issues. People are inherently territorial, aggressive animals - at least to the extent being so doesn't make them much of an outlier.

> Meaning that a person who kills somebody else definitely has mental issues that come from their childhood.

Is that even true?

That's very superficial imo. For me it is remembering that I am unique and being in tune with the inner voice which is really mine.

Maybe some people grow into that mindset naturally but it wasn't the case for me (even if there were no particular reason for the societal pressure to be super strong).

i think that that issue leads to particularly intense 'folk psychology' because everybody has experience with a mind which will lead to everybody having their own litle set of folk psychology beliefs that the scientists need to overturn.

contrast that to something like... geology. i'm sure there was folk geology back in the day about where different types of rocks come from, but for 99% of people they weren't particularly invested in it. so when the scientists came out with empirical research, most people would be fine with it.

but when psychologist confront folk psychology, people often take it as a personal affront. any comment section about psychology research (HN included) is filled with armchair experts contradicting researchers with their own pet theories

I had the same question. It's Cincinnati chili. Wikipedia has an article [1] and Serious Eats goes into way more detail [2]. It actually sounds pretty good. I'll give it a go but with much less cheese.

[1]: https://en.wikipedia.org/wiki/Cincinnati_chili

[2]: https://www.seriouseats.com/cincinnati-chili-recipe-8402230

I think this is very fair. Every science that was starting out was indistinguishable from nonsense when starting out. But to say we shouldn't have pursued alchemy, because we didn't understand chemistry? One likely led to the other.

As someone who did statistics and psychology, I'm very surprised by this take, for a few reasons:

1. Many of the early pioneers in statistics were psychologists.

2. The econ x psych connection is strong (eg econometrics and psychometrics share a lot in common and know of each other)

3. Many of the people I see with math chops trying to do psychology are bad at the philosophy side (eg what is a construct; how do constructs like intelligence get established)

Not all psych is as jurasic as you describe. For example cofnitive psychology has better theorios with more predictive power than personal psychology that is often picked at. Sure journals are flooded with underpowered studies and studies with very little links to theory, and there is still massive gaps in scientific knowledge but core consturcts are solid.

Lots of great points. I would start with semiotics, including during the problem definition phase, otherwise you could easily end up lost in language without the slightest clue of the predicament you're in.

Epistemology is also useful, because it might allow one to wonder if the problem space is non-deterministic (or not discoverable as).

Psychology is not inherently treated as soft, it's jusst that its human element attracts intuitive people much more than rational ones. If nore rationally minded people took up the study and research of psychology fields, more hard stuff would come to the front, although soft stuff is hardly behind in intelligence.

The parallels to ML you drew are on point. ML has this tendency to oversimplify complex phenomena with a easy to produce datasets, because that's what ML folks do, they find a smart and easy way to create a dataset and then they focus on the models. But this falls apart pretty quickly when you go into societal problems, such as hate speech or misinformation. Maybe there it would be nice to have some rigor and theory behind the dataset instead of just winging it. I am working on societal biases in NLP and I feel confident that majority of the datasets used have practically no validity.

> I do think that's where the field is at, because there is no fundamental laws

I think that there are some fundamental laws, which are based on perceptions and their interplay. Speaking very briefly, there are five classes of perceptions: emotions, wishes, thoughts, beliefs, and body sensations. The division of perceptions into these classes is not a result of purely intellectual exercise or idle theorizing. If one starts carefully and diligently observe contents of their mind, these contents will delaminate into such classes naturally. Try that yourself and you will see it as a fact.

Further introspection and assessment of arising perceptions would reveal some interesting patterns: there are two mutually incompatible kinds of emotions, and two mutually incompatible kinds of wishes, and so on.

One could make observations about the interplay of these perceptions and their dynamic. For example, if someone in some specific situation experiences an emotion X and a wish A (with some specific qualities), they can either realize that wish or choose not to do so. Each choice will lead to some changes in the contents of the mind: emotion X is replaced by emotion Y, and/or wish A is changed into wish B, and so on. Gather enough observations of that kind, and you could eventually formulate some hypotheses about possible general laws of perceptions (e.g. make a prediction that emotion X will change to emotion Y in specific set of circumstances).

These hypotheses could be verified by training several people to observe the same five classes of perceptions in the same manner. Arrange various test events for them, record their choices and outcomes of these choices (described in the same language of five classes of perceptions).

If most of them report more or less the same subjective outcomes (without being told about hypothesis and predictions, of course), that's the first step of verification for a possible general law of perceptions.

The second step of verification would be to apply brain imaging to those trained people, allowing us to map emotions X,Y,Z to some distinct patterns of the brain activity. After that do the same experiments with people who are untrained: arrange same test events for them while recording their brain activity. If changes in their brain patterns match for emotion X changed to emotion Y, that would be an objective confirmation of hypothesis formulated earlier.

I feel like that's saying weather forecasting is voodoo bullshit because it's all quantum mechanics deep down

It's possible that fields can fluctuate in how scientific they are over time.

Physics has in the past been a case study of how to successfully deploy the scientific method, but modern physics is often criticized from within for spending all its time on mathematical "theories" that can't actually be tested, something conventionally considered unscientific. Epidemiology was once built on basic scientific observation, it's now also disappeared down the drain-hole of endless mathematics without real world hypothesis testing.

Medicine in contrast spent a lot of time being unscientific in the past, and is now much more rigorous. The problems here are usually fraud done in aid of avoiding the scientific method without being detected, rather than the actual lack of a scientific method at all.

And some fields have got more scientific over time, or at least more quantitative. Economics and education are like this.

It isn't very scientific if a discipline persists in bad research practices, and that's what psychology is still doing. Indeed, it's very hard to get experimental evidence for a theory by measuring people, but if you know you can't, then don't do it. Wait until the means to do so become available, and try to find those ways instead. Unfortunately, that's not the state of psych. research.

There is of course also a more practical side. For psych and med interventions, you don't need to know how it works. If method A works better than method B, it's a good idea to try A before trying B. However, that research is also difficult, and many papers have been found wrong afterwards. While that's the way to go, it shows that even when attempting to establish superficial effects, the disciplines are failing.

I think you are ignoring a massive driving factor in both medicine and psychology; they are businesses.

Both fields, scientific as they often try to be, are subject to the sway of funding and profits more-so than other sciences like physics (as an example). Vague and often unsupported claims sell medical and mental health products, making it a profitable venture whether the product actually works or not.

I say this as a former student of psych, degree and all. The mental health industry has exploded in the last ten years or so, yet not much new thinking has been brought to the table at all. It's a lot of old standards being repackaged as a revolutionary solution (looking at you CBT) but being sold as mobile apps, hooking people with hope for relief in the form of a convenient, easy-to-use package, the same way medicine does when they add a bit of caffeine to acetaminophen and call it a solution for migraines that you don't have to visit a doctor to obtain. Once a field is driven by profit growth, it becomes this twisted, ersatz version of it's former self and this works because people are so desperate, they'll buy into just about anything you offer that might make them feel slightly better.

I would absolutely say that medicine is not science - or at least that it is very bad science with a low probability of being reproducible - and anyone with even tangential exposure to most medical research would do the same. Small sample sizes, confounding variables in treatment, and non-random assignment are the norm, simply because practicing medicine is not the same as performing scientific research.

Medicine is the practice of healing sick people, and (even if they wanted to, which most do not) it is very, very hard for doctors to convince an IRB that it's a good idea to give sick people a placebo. Also, despite some attempts to perform placebo knee surgeries [0], there are many medical interventions where it is simply impossible to do things like a basic RCT.

[0] https://www.nejm.org/doi/full/10.1056/NEJMoa013259

Would people also suggest that Medicine is not a science

I think that depends on whether those people believe that evidence-based medicine is scientific.

I doubt it's to the same degree, maybe if you include cases like this https://x.com/cremieuxrecueil/status/1844596459162812869

That shouldn’t surprise anyone on the medicine end. People across the industry, from reviewers to those writing textbooks, are receiving money from those they’re reviewing. They’re incentivized to lie to make their sponsors look good. It’s the opposite of science.

Now, we come to the root of the failure. God’s Word tells us to test the source and the content of a message. Certain sources are all about ego, money, or pleasure. They’re willing to manipulate others for selfish gain. Simultaneously, God’s Word teaches habits and traits that honest people should possess. So, we look at their behavior to spot the warning signs that their content should be dismissed or simply reviewed more.

Applying the ancient wisdom to organized science, I found that this problem was pervasive. The institutions, both funding and research, had biases that caused more of specific types of work to be created. Then, there were biases at the individual and group level. These aren’t always bad but are rarely considered.

Further, there were three incentives driving lots of bad science: financial incentive called “publish or perish;” citation index or scores; sensational media. Two of those incentivized cranking out lots of low-quality papers that pressed the right buttons to get more money or citations. The third incentivizes specific claims that receive fame. All three usually penalize, by finance or fame, both steady, grunt work and replication of existing results.

So, rather than the scientific method, what’s happening is more like a game of TV show with winners and losers. It always looks like it uses the scientific method with some amount of real science in it. There’s also some portion of useless work, necessary work not happening, fraud, and censorship. These aren’t random: they’re baked right into the incentives and biases of the system.

Realizing this has made me wonder what I can trust or what I even know out of prior, scientific reporting. Fortunately, we’re blessed that most facts are so unimportant that being wrong doesn’t hurt us. The fields we depend on day to day are close enough to the truth that the products work acceptably well or don’t hurt us. Past that, I wonder what it will take to get to a point where organized science is actually doing science. Consistently.

I hope we achieve this. I worship Christ, not science. It is one of many forms of knowledge, many gifts, that make our lives better if used correctly. I thoroughly enjoy real science. I just think it’s in huge decline with society getting more and more dependent on the fake forms of it.

>Would people also suggest that Medicine is not a science?

Yes, I would. If one looked closely at it it's got the trappings of religion. To give you a guesstimate I'd say 3/4 of medicine is pseudoscience. either completely useless or actively harmful.

I found it insightful and pleasant to see that there are people inside of a an important field being intellectually curious and honest about the direction of their field of study.

Its a piss poor article that was written to vent, but it didn't try very hard to find good psychological research. Episodic memory psychological literature is very strong, IMO, yet never gets brought up in these kinds of articles. Its always the fluffy puffy research that fuels tabloid headlines, not the research that shows, for example, differential patterns of memory strategies over child development, or the contributions of context to recognition memory, the differences between recollection and familiarity processes supporting recognition memory..you know, all the stuff that is not flashy for tabloids, but is real psychological science. Dr. Charan Ranganath was a member of my dissertation committee who recently wrote a wonderful book about memory and gave some really fantastic interviews. For example, on Fresh Air: https://www.npr.org/transcripts/1233900923 Now yes, some of this is informed by neuroimaging and neuroscience spanning human and animal models, but also lots and lots of behavioral memory research. And the findings that are discussed are pretty reliable, shown over and over again in different ways. So, no, this article is not great. It did not do diligent research. Its a rant that focuses on specific types of research that is a small majority of the REAL field.

I think neuroscience will not replace psychology for the same reasons physics doesn't replace chemistry. In theory it might encompass it, but in practice it's a difficult way to get there, and therefore not the effective path.

> neuroscience will replace psychology

That seems like say that hardware engineers should be the ones debugging software.

Philosophy is often very well grounded, often annoyingly so. We gave up on it because it turns out you can create iPhones if you ignore the philosophical problem of induction, go do science instead, and assume the laws of physics won’t try and conspire against you.

Rather, I wonder if Psychology would be better thought of as something even less grounded than science. Something where we’re just are happy with an accumulation of stuff that’s happened to work well enough, without pretending that we’re hunting fundamental principles. Something like a profession: Engineering, Doctoring, that sort of stuff.

Neuroscience isn't immune from non-reproducibility, for example this https://news.ycombinator.com/item?id=41672599 from just a couple of weeks ago

What a stereotypical hackernews comment, wow.

> I, a completely unqualified internet commenter,

Just leave it there.

In a similar way Im wondering what gender studies would be called?