Hansenq
3 hours ago
61% AI generated, according to Pangram https://www.pangram.com/history/e5a00ace-94cc-436e-b87b-a094...
c'mon, you're Carlyle, a trusted institution for financial advice! How can I trust what you're saying if the AI-generated text is so blatantly obvious?
nkurz
3 hours ago
Is there any reason to trust Pangram's tool over Carlyle's reputation? I don't know whether this was AI generated or not, but an online tool claiming it's AI generated doesn't sway my opinion much. Are there any studies showing that Pangram's tool is well calibrated for this type of article? If not, what makes you trust it?
(I'm not sure why it was marked that way, but I vouched to bring your comment back from auto-dead. I glanced at your comment history, but don't see any clear reason for this.)
Hansenq
3 hours ago
I only went to Pangram because I read through the essay and could not help but notice the Claude-like sentence structure and AI-isms in it, which distracted me and completely made me distrust the thesis of the piece.
My understanding is that Pangram is the best out of all of the AI detectors and if there is a better one I'm happy to switch to it. And it's easier to point to them than to give explicit sentences and examples about why it reads so AI-generated (and since you want it, it's these sentences in particular: "The message was unambiguous: energy is finite, security is earned, and comfort has a cost.", "That template is being applied again today — and markets have accepted it.", "One country never made the West’s mistake.", etc.)
Reading through it again, there are so many emdashes. And don't get me wrong, I was a liberal user of emdashes before AI! But just like, if your job is to communicate your thoughts to a wide audience at least respect their intelligence enough and not rely on crutches just to get an article out against a deadline.
stereolambda
an hour ago
To me these look like professional writing, and I've seen worse LLM'isms. I am with some people in that I don't usually want to read pieces where GenAI was involved. But this is one of the cases where I wouldn't want the remedy to be a life of constant purity tests and suspicion. There has been enough of that in contemporary culture. I want to trust people and let them be by default. I mean, I don't want writers to specifically memorize current LLM'isms in order to avoid them: or worse, to prompt an LLM to strip them. I believe from experience than GenAI texts have secondary bad characteristics, mainly being bland in style and thought, and often going nowhere, like mandatory student papers. This should be enough to bury or criticize them on merits more often than not.
I do see myself treating slightly broken grammar, typos as a slightly positive signal in writing in the recent years. I also see that this is messed up - I'd like to respect someone in kind if they put care in writing - and easy to fake if anybody wants to, with an LLM. If you do strongly suspect GenAI writing, I mean it's fair if a sincere opinion, but I'm tired of having those as top comments with a whole response tree. Ironically contributing to that now.
lacker
3 hours ago
Yes there are studies, for example last year Pangram's false positives were measured to be under 0.5%.
https://www.pangram.com/blog/third-party-pangram-evals
Personally, at first I thought these sorts of tools were dumb and wouldn't really work, but I think it works because it just isn't designed to be "adversarial". If you want your AI to trick Pangram, you can make an AI to trick Pangram. It just catches people who are cutting and pasting from the AIs without putting any more effort into hiding it.
sdenton4
2 hours ago
Any binary classifier can have a FPR under 0.5% if you don't have any restriction on FNR...
D-Machine
2 hours ago
While I am quite skeptical of the claims linked above, that link does indeed cover the FNR at the FPR of 0.005, and finds it broadly to be on the same order of magnitude, i.e. also below 0.005.
lelanthran
2 hours ago
> Is there any reason to trust Pangram's tool over Carlyle's reputation?
You have your own chatbot, right? Ask it. I've never had one disagree with GPTZero yet.
massysett
2 hours ago
I do not understand why this matters. Judge the content on its merit. It makes no difference if “AI wrote it”.
ASalazarMX
43 minutes ago
This has the same energy as people who don't care if controversial images in social media are AI generated, as long as they're engaging.
It makes a huge difference if the writing was manual or automated. LLMs generate verbose, generic writing, and ideas that could be concisely expressed in a sentence inflate to entire paragraphs. It's disrespectful to readers when the author saves a couple of hours by wasting thousands of their readers.
orbital-decay
an hour ago
Article from a trusted organization is supposed to be grounded in real-life events and filtered through their specific area of expertise. They bet their reputation on it, at least in theory. Current agents have none of that, and you can't trust their output to the same extent in practice. Too much recognizable slop in the article suggests high degree of autonomy of the agent that has been used, and raises doubts about trusting the entire article. It might be the valid opinion of the author rephrased by the model, but you can't tell as it obscures actual intent and the amount of real life data.
watwut
an hour ago
If AI wrote it, it is not worth effort engaging with.
mapontosevenths
2 hours ago
Nonsense. Of course the tool matters.
That's why I only use code written in Vim. Emacs corrupts the othewise identical bits, just like AI. Gets 'em all greasy and then they smell funny.
LearnYouALisp
2 hours ago
Get me some holy water
trhway
2 hours ago
I wasted some time trying to get a sense of the content, to "judge it on its merit". It is a total slop. And if i knew beforehand that it is AI, i'd have spent much less time as the first short look would have confirmed that it is slop.
IAmGraydon
an hour ago
Is there a particular part you are claiming is incorrect? If so, point it out and name your sources. If not, what’s your point?