One thing that I find useful is to look at the about page, as well as terms of service or privacy policy, of any website.

If there's no entity named, or you can't verify that a named entity actually exists, it's a red flag.

In this case, there's no individual or company named anywhere that I can see. Not on the About page, and not in any of the legal agreements. And the phone number looks like it's a VoIP number from Onvoy, a major wholesaler, which is a yellow flag.

I think this will become ever more common as AI improves. We used to joke, "On the Internet, nobody knows you're a dog," but now that is clearly true for AI agents. They can automate every part of the process, from domain and hosting registration, to web content creation, to responses to emails and even phone calls. Entire personas are trivial to create and maintain. This is what many companies are openly trying to do as cost cutting measures, but taken to its extreme you end up with entire companies generating income that are entirely agent driven. So long as the tokens cost less than the revenues they can generate, people will deploy these to passively make money for them, just like the early 2000s when there were entire webpages just present to show ads to unsuspecting visitors.

It's interesting to notice which fields try to garner positive publicity by emphasizing that they DON'T use AI. There is increasing negativity about AI generally but it is quite revealing for an organization to try to make their refusal to use it a big part of their identity. Maybe I can start offering artisanal software development services to go after that sweet hipster dollar - code done 100% by hand, cruelty-free.

Even the website looks AI generated. Lots of cards with hover effects, icons in square boxes, the button shapes and behaviours, and just the general look and feel has Claude written all over it.

Surely the FTC would be all over this for truth in advertising, even if the FDA wasn't. The "services for medical researchers" reads like "tell us the result you want and the level of journal prestige and we'll price an outcome for you"

So the government is going to track down everyone involved with running Research Gold and prosecute them for fraud, right?

Not long ago companies claiming “100% AI-powered” services were secretly relying on humans because the tech wasn’t good enough.

Now, the opposite is true - AI has gotten so good, people are trying to pass AI work off as “100% human-powered.”

To me this remarkable inversion in a short period of time demonstrates how powerful these models can be; even if AI is overhyped and the financing/economics seem like a stack of cards, this example is driving home the point to me of just how far these models have come.

So they are not telling the truth.

love the irony

I got downvoted the other day for saying that anonymous website ownership and perverse financial incentives are bad for the health of the internet.

One of the humans mentioned in the article, Sebastian Rowan, a Ph.D. candidate at the University of New Hampshire, is quoted as saying “But a fundamental problem with using AI, even specialized tools, for anything is their tendency to hallucinate, which as far as I know is believed to be an unsolvable problem.”

I don’t believe that’s an unsolvable problem.

I’ve done some private experiments where I had Claude, over many sessions, design and implement original research projects in fields in the humanities and social sciences, with a research paper as the final product of each project.

I included detailed instructions on research ethics in CLAUDE.md and other project documentation, insisting, among other things, that it double- and triple-check all citations and references to make sure not only that the referenced papers existed but also that they actually said what the paper Claude wrote claimed they said. I gave Claude an OpenRouter API key and a daily budget, and it outsourced much of that adversarial checking to models made by other companies. In my checks of its final papers, I found no instances of hallucinated or otherwise invalid references. At various steps in the research process, Claude also, on its own initiative, had other models critique its research designs, data analyses, written drafts, etc., and it made changes based on that feedback.

The final papers were, to my eye, well done, if quite narrow in their scope and claims. The ethical alignment I imposed on it, as well as its lack of ego, seemed to keep it from making even weak claims if they could not be rigorously justified.

I’m a retired academic, and I no longer have any need or desire to produce research papers myself, with or without AI. Rather, I am interested in the impact of AI tools on the research process, particularly on how research and researchers will be evaluated in the years ahead.

I did not, and will not, release any of those papers written for me by Claude, though I have released what it produced for another research project I had it do, on a topic in the philosophy of language. In case anyone is interested:

https://tkgally.github.io/ai-semantics/