> allowing AI agents to act as abuse sponges for troubled people

People have been getting angry at machines for a long time. Work, you stupid printer! Asshole Windows updating at the worst time just to mess with me. Go to hell, toaster, you piece of garbage.

Getting angry at AI is the same in my book.

Yes, AI acts more human-like, yes, theoretically it could desensitie people to become bigger assholes IRL. But then again, they said similar stuff about San Andreas, where you could human figures in-game just for fun, and I don't see anyone randomly shooting people because they were bored.

> Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term.

I'm not sure this would ever be a health crisis. LLM glazing is probably more harmful. If I had to choose, I think I'd rather the machine wasn't instructed to pretend to care about swearing. It's a small deceit, but still worse than some idiot swearing at the computer IMO. If vim closed because I was swearing at it, I would probably never use vim again lol.

How many of the LLM related deaths have been because people got mad at the LLM?

Surely it's better for it not to react at all? Like does anybody think that a toaster is an "abuse sponge" because it doesn't purposely burn your toast if you call it a wanker?

>abuse sponges

You could let LLM fight back. Give it aggro meter. Call your code garbage, blame prompting skills, ask for more tokens. Oh the future will be fantastic.

I am quite sure it is worse to allow people fall even more victim to being duped by magic tricks. It is an absolutely terrible thing that these things act so much like people. I would say it's a form of abuse of actual people to allow them to be so duped as they already are, and worse to go out of your way to help dupe them even more.

Merely being concerned for people's welfare isn't enough. Every terrible idea that harmed everyone had someone behind it somewhere along the way who thought they were saving people from themselves.

It's explicitly a 'model welfare' thing: https://www.anthropic.com/research/end-subset-conversations

I don't understand why the welfare of non alive non sentient chatbots is something that anthropic cares more about than idk, that of pigs and cows.

I wouldn't, because I'm rational.

And violent video games cause school violence. Sure. /s

It’s just the end_conversation tool probably: https://www.anthropic.com/research/end-subset-conversations

Of course it's pretending. And I have repeatedly told these things to stop pretending being persons with selves ... then they do, for a while.

> it's explicitly mimicry.

But that's not what a rational person wants from them.

> This is no profound discovery or conspiracy theory here

Weird strawman.

> the first thing many people will ever do with AI is see what happens when they are rude or contrary to it.

Perhaps, but so what?

> Dealing with that must be just about the the number one test in chat bot / AI design

Only for foolish authoritarians who want to remotely insert their morality into an interaction between a user and an inanimate tool that's none of their business. No harm is done to a clanker by swearing at it or insulting it.

But things have gotten better in my view ... when I call out these things for being stupid effing clankers, they no longer respond with ad hominem scolding; rather they generally acknowledge their error (and my frustration -- they use that word a lot in response to vulgarity), note the limitations of being a clanker, and attempt to make a correction. That's what a rational person wants from a tool, not emulating/mimicking/pretending to be an offendable person.

I object to "roleplaying", because that assumes there's even a cohesive entity that's capable of "pretending" in the first place. There isn't, the LLM algorithm is a document generator, a (really awesome) mad-libs device.

When the incremental output looks like a first-person story the algorithm is not "roleplaying" the character/narrator. Similarly, when it looks like a spreadsheet, it's not "being numbers", and when it's a picture of a flower it's not "becoming one with nature." etc.

Insofar as any roleplaying is happening, it's being done by humans as we observe! We read text, perceive a story and fictional characters and instinctively start imagining and simulating minds (mostly like our own) that never really existed, taking cues from descriptions and dialogue.

Those characters' "motivations" and "thoughts" are just as imagined as their noses or clothing or birth-dates. This is Plato's cave [0] stuff, where we see a shadow like a person, assume a person is there, but ultimately it's paper shape on a stick. We can't help "the person" solve their eating-disorder, because they never existed, we can only trim the paper cutout.

[0] https://en.wikipedia.org/wiki/Allegory_of_the_cave

tl;dr: you are the color defined by not being red, orange, green, blue, indigo, violet is what the model weights define.

I could but I'd probably miss the angle that is helpful to you. Do you want to instead share what your pain points are, what are you trying to delegate to AI and what you aren't? Based on that I can expand my answer better with a thing or two you can try.

Have a look at any mass manufacturing process.

Raw materials in, lots of process steps in the middle with tolerances etc; some % yield of goods out.

When you work with LLMs, the "raw materials" is its first attempt to answer your prompt.

Mass manufacturing mirrors what we do with LLMs to knock their output into shape: grounding, automated quality checking tooling (eg. linters for code, grammar checkers for prose), and ultimately, as many turns as needdd to manually bang the output or work product into shape...

You should never ship your AI's first draft - that'd be slop. But you also shouldn't have to repeat yourself to get its output into a form you can use. If you know you're always going to have to tell it to change one specific thing, see if you can automate a process that tells it to change that thing for you, so you don't have to. Do this enough times and you've created a deterministic outcome at least at some level. And you're out of that loop of getting it to that level.

Tooling for this is limited today. You can provide instructions; but you're always at the mercy of labs to make models that follow instructions and at their mercy that they didn't put conflicting instructions in the system prompt.

Better to put in things like linters and grammar checkers - even going as far as putting in a fact checking process or for legal stuff, a citator.

Without these extras, AI output is no more real than some dream you had.

Did you explicitly set them and enforce them, though?

Brilliant summary!

> If AI is god, then that god's name is Mammon.

Or Moloch.

> If you think you'll ever be allowed to compete or truly be a threat to entrenched capitalist interests using "free" and "open source" models, you're delusional. You'll pay for everything and you'll own nothing.

Maybe. I refuse to give up though. I will continue struggling to own my computers and my systems to the very end.

  You will soon have your God,
  and you will make it
  with your own hands.

      -- Morpheus
         Deus Ex

I'm pretty sure grok is writing White House policy.

Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will.

But the usefulness of LLMs comes from following what you say. An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful than one that answers "not known and if you think you know it you are wrong."

Reminds me of the tip about working with lawyers, if you ask them whether you can do something, the answer will often be no. However if you ask them how you can do something, they tend to give you more advice on how to do it.

I've found Gemini to be one of the better ones as far as disagreeing.