Gentoo AI Policy

191 pointsposted 5 months ago
by simonpure

134 Comments

AdieuToLogic

5 months ago

Perhaps the most telling portion of their decision is:

  Quality concerns. Popular LLMs are really great at 
  generating plausibly looking, but meaningless content. They 
  are capable of providing good assistance if you are careful 
  enough, but we can't really rely on that. At this point, 
  they pose both the risk of lowering the quality of Gentoo 
  projects, and of requiring an unfair human effort from 
  developers and users to review contributions and detect the 
  mistakes resulting from the use of AI.
The first non-title sentence is the most notable to consider, with the rest providing reasoning difficult to refute.

jjmarr

5 months ago

I've been using AI to contribute to LLVM, which has a liberal policy.

The code is of terrible quality and I am at 100+ comments on my latest PR.

That being said, my latest PR is my second-ever to LLVM and is an entire linter check. I am learning far more about compilers at a much faster pace than if I took the "normal route" of tiny bugfixes.

I also try to do review passes on my own code before asking for code review to show I care about quality.

LLMs increase review burden a ton but I would say it can be a fair tradeoff, because I'm learning quicker and can contribute at a level I otherwise couldn't. I feel like I will become a net-positive to the project much earlier than I otherwise would have.

edit: the PR in question. Unfortunately I've been on vacation and haven't touched it recently.

https://github.com/llvm/llvm-project/pull/146970

It's a community's decision whether to accept this tradeoff & I won't submit AI generated code if your project refuses it. I also believe that we can mitigate this tradeoff with strong social norms that a developer is responsible for understanding and explaining their AI-generated code.

bestham

5 months ago

IMO that is not your call to make, it is the reviews call to make. It is the reviewers resources you are spending to learn more quickly. You are consuming a “free” resource for personal gain because you feel that it is justified in your particular case. It would likely not scale and grind many projects to a halt at least temporarily if this was done at scale.

user

5 months ago

[deleted]

AdieuToLogic

5 months ago

> I've been using AI to contribute to LLVM, which has a liberal policy.

This is a different decision made by the LLVM project than the one made by Gentoo, which is neither right nor wrong IMHO.

> The code is of terrible quality and I am at 100+ comments on my latest PR.

This may be part of the justification of the published Gentoo policy. I am not a maintainer of same so cannot say for certain. I can say it is implied within their policy:

  At this point,  they pose both the risk of lowering the 
  quality of Gentoo  projects, and of requiring an unfair 
  human effort from  developers and users to review 
  contributions ...
> LLMs increase review burden a ton ...

Hence the Gentoo policy.

> ... but I would say it can be a fair tradeoff, because I'm learning quicker and can contribute at a level I otherwise couldn't.

I get it. I really do.

I would also ask - of the requested changes reviewers have made, what percentage are due to LLM generated changes? If more than zero, does this corroborate the Gentoo policy position of:

  Popular LLMs are really great at generating plausibly 
  looking, but meaningless content.
If "erroneous" or "invalid" where the adjective used instead of "meaningless"?

jjmarr

5 months ago

   I would also ask - of the requested changes reviewers have made, what percentage are due to LLM generated changes? If more than zero, does this corroborate the Gentoo policy position of "Popular LLMs are really great at generating plausibly looking, but meaningless content." 
I can only speak for my own PR, but most requested changes were related to formatting and other stylistic issues that I didn't fully grasp as a new LLVM contributor. e.g. Not wrapping at 80 characters, forgetting to declare stuff as const, or formatting the documentation incorrectly.

Previous codebases I've worked on during internships linted the first two in CI. And the documentation being formatted incorrectly is because I hand-wrote it without AI.

Out of the AI-related issues that I didn't catch, the biggest flaws were redundant comments and the use of string manipulation/parsing instead of AST manipulation. Useless comments are very common and I've gotten better at pruning them. The AI's insistence on hand-rolling stuff with strings was surprising and apparently LLVM-specific.

However, there was plenty of erroneous and invalid behaviour in the original AI-generated code, such as flagging `uint32_t` because the underlying type was an `unsigned int` (which wouldn't make sense as we want to replace `unsigned int` with `uint32_t`).

I prevented most of this from reaching the PR by writing good unit tests and having a clear vision of what the final result should look like. I believe this should be a basic requirement for trying to contribute AI-generated code to an open-source project but other people might not share the same belief.

user

5 months ago

[deleted]

benreesman

5 months ago

I'm a bit later in my career and I've been involved with modern machine learning for a long time which probably affects my views on this, but I can definitely relate to aspects of it.

I think there are a couple of good signals in what you've said but also some stuff (at least by implication/phrashing) that I would be mindful of.

The reason why I think your head is fundamentally in a good place is that you seem to be shooting for an outcome where already high effort stays high, and with the assistance of the tools your ambition can increase. That's very much my aspiration with it, and I think that's been the play for motivated hackers forever: become as capable as possible as quickly as possible by using every effort and resource. Certainly in my lifetime I've seen things like widely distributed source code in the 90s, Google a little later, StackOverflow indexed by Google, the mega-grep when I did the FAANG thing, and now the language models. They're all related (and I think less impressive/concerning to people who remember pre-SEO Google, that was up there with any LLM on "magic box with reasonable code").

But we all have to self-police on this because with any source of code we don't understand, the abstraction almost always leaks, and it's a slippery slope: you get a little tired or busy or lazy, it slips a bit, next thing you know the diff or project or system is jeopardized, and you're throwing long shots that compound.

I'm sure the reviewers can make their own call about whether you're in an ok place in terms of whether you're making a sincere effort or if you've slipped into the low-integrity zone (LLVM people are serious people), just be mindful that if you want the most out of it and to be welcome on projects and teams generally, you have to keep the gap between ability and scope in a band: pushing hard enough to need the tools and reviewers generous with their time is good, it's how you improve, but go too far and everyone loses because you stop learning and they could have prompted the bot themselves.

m463

5 months ago

This comment seems to use "I" a lot.

jlebar

5 months ago

As a former LLVM developer and reviewer, I want to say:

1. Good for you.

2. Ignore the haters in the comments.

> my latest PR is my second-ever to LLVM and is an entire linter check.

That is so awesome.

> The code is of terrible quality and I am at 100+ comments on my latest PR.

The LLVM reviewers are big kids. They know how to ignore a PR if they don't want to review it. Don't feel bad about wasting people's time. They'll let you know.

You might be surprised how many PRs even pre-LLMs had 100+ comments. There's a lot to learn. You clearly want to learn, so you'll get there and will soon be offering a net-positive contribution to this community (or the next one you join), if you aren't already.

Best of luck on your journey.

close04

5 months ago

> They know how to ignore a PR if they don't want to review it

How well does that scale as the number of such contributions increases and the triage process itself becomes a sizable effort?

LLMs can inadvertently create a sort of DDoS even with the best intentions, and mitigating it costs something.

jjmarr

5 months ago

Thanks. I graduated 3 months ago and this has been a huge help.

thesz

5 months ago

  > You might be surprised how many PRs even pre-LLMs had 100+ comments
What about percentages?

jubalfh

5 months ago

here's to the hope you get banned from contributing to llvm.

JonChesterfield

5 months ago

This is exciting. Thank for for raising the point. I've posted https://discourse.llvm.org/t/our-ai-policy-vs-code-of-conduc... to see what other people think of this. Thank you for your commit, and especially for not mentioning that it's AI generated code that you don't understand in the review, as it makes my point rather more forcefully than otherwise.

totallymike

5 months ago

[flagged]

tomhow

5 months ago

You can't comment like this on Hacker News, no matter what you're replying to. You've been on HN a long time and we've never had to warn you before, but please take a moment to read the guidelines and make an effort to observe them, especially these ones:

Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.

When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."

Please don't fulminate. Please don't sneer, including at the rest of the community.

Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.

Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

https://news.ycombinator.com/newsguidelines.html

0000000000100

5 months ago

Go look at the PR man, it's pretty clear that he hasn't just dumped out LLM garbage and has put serious effort and understanding into the problem he's trying to solve.

It seems a little mean to tell him to stop coding forever when his intentions and efforts seem pretty positive for the health of the project.

jjmarr

5 months ago

I didn't make a decision on the tradeoff, the LLVM community did. I also disclosed it in the PR. I also try to mitigate the code review burden by doing as much review as possible on my end & flagging what I don't understand.

If your project has a policy against AI usage I won't submit AI-generated code because I respect your decision.

noosphr

5 months ago

That's no different to on boarding any new contributor. I cringe at the code I put out when I was 18.

On top of all that every open source project has a gray hair problem.

Telling people excited about a new tech to never contribute makes sure that all projects turn into templeOS when the lead maintainer moves on.

MangoToupe

5 months ago

I think the project and reviewers are both perfectly capable of making their own decisions about the best use of their own time. No need to act like a dick to someone willing to own up to their own behavior.

sethammons

5 months ago

Your final sentence moved me. Moved to flagging the post, that is.

fuoqi

5 months ago

Well, some people just operate under the "some of you may die, but it's a sacrifice I am willing to make" principle...

JohnBooty

5 months ago

There are a number of other issues such the ethical and environmental ones. However, this one in isolation...

    Popular LLMs are really great at 
    generating plausibly looking, but meaningless 
    content. They are capable of providing good 
    assistance if you are careful enough
I'm struggling to understand this particular angle.

Humans are capable of generating extremely poor code. Improperly supervised LLMs are capable of generating extremely poor code.

How is this is an LLM-specific problem?

I believe part of (or perhaps the entire) the argument here is that LLMs certainly enable more unqualified contributors to generate larger quantities of low-quality code than they would have been able to otherwise. Which... is true.

But still I'm not sure that LLMs are the problem here? Nobody should be submitting unexpected, large, hard-to-review quantities of code in the first place, LLM-aided or otherwise. It seems to me that LLMs are, at worst, exposing an existing flaw in the governance process of certain projects?

wodenokoto

5 months ago

It means, if you can’t write it, they don’t trust you to be able to evaluate it either.

As for humans who can’t write code, their code doesn’t tend to look like they can.

user

5 months ago

[deleted]

SAI_Peregrinus

5 months ago

> Nobody should be submitting unexpected, large, hard-to-review quantities of code in the first place,

Without LLMs, people are less likely to submit such PRs. With LLMs they're more likely to do so. This is based on recent increases in such PRs pretty much all projects have seen. Current LLMs are extremely sycophantic & encourage people to think they're brilliant revolutionary thinkers coming up with the best <ideas, code, etc> ever. Combined with the marketing of LLMs as experts it's pretty easy to see why some people fall for the hype & believe they're doing valuable work when they're really just dumping slop on the reviewers.

29athrowaway

5 months ago

LLMs trained on open source make the common mistakes that humans make.

wobfan

5 months ago

> make.

No, made. Which is a very important difference.

perching_aix

5 months ago

[flagged]

AdieuToLogic

5 months ago

[flagged]

johnfn

5 months ago

But it's also difficult to prove it correct by argument or evidence. "Refute" is typically used in a context that suggests that the thing we're refuting has a strong likelihood of being true. This is only difficult to prove incorrect because it's a summary of the author's opinion.

paulcole

5 months ago

How is it telling at all?

It’s just what every other tech bro on here wants to believe, that using LLM code is somehow less pure than using free-range-organic human written code.

Kwpolska

5 months ago

Tech bros want the exact opposite, so that they can sell their AI crap and replace human developers with AI bots.

paulcole

5 months ago

That has not been what I have seen in the comments here on HN lol

puilp0502

5 months ago

Every time I encounter these kinds of policy, I can't help but wonder how these policies would be enforced: The people who are considerate enough to abide by these policies, are the ones who would have "cared" about the code qualities and stuff like that, so the policy is a moot point for these kinds of people. OTOH, the people who recklessly spam "contributions" generated from LLMs, by their very nature, would not respect these policies in very high likelihood. For me it's like telling bullies to don't bully.

By the way, I'm in no way against these kinds of policy: I've seen what happened to curl, and I think it's fully in their rights to outright ban any usage of LLMs. I'm just concerned about the enforceability of these policies.

userbinator

5 months ago

I think it's a discouragement more than an enforcement --- a "we will know if you submit AI-generated code, so don't bother trying." Maybe those who do know how to use LLMs really well can submit code that they fully understand and can explain the reasoning of, in which case the point is moot.

joecool1029

5 months ago

> I can't help but wonder how these policies would be enforced

One of the parties that decided on Gentoo's policy effectively said the same thing. If I get what you're really asking... the reality is, there's no way for them to know if a LLM tool was used internally, it's honor system. But I mean enforcement is just ban the contributor if they become a problem. They've banned or otherwise restricted other ones for being disruptive or spamming low quality contributions in the past.

It's worded the way it is because most of the parties understand this isn't going away and might get revisited eventually. At least one of them hardline opposes LLM contributions in any form and probably won't change their mind.

puilp0502

5 months ago

I see. So if I'm understanding correctly, then this policy serves as a kind of "legal ground" from which the maintainers can take action against perpetrators, right?

To add a bit more context, when I was writing the original comment, I was mainly thinking of first-time contributors that don't have any track records, and how the policy would work against them.

joecool1029

5 months ago

> I see. So if I'm understanding correctly, then this policy serves as a kind of "legal ground" from which the maintainers can take action against perpetrators, right?

They aren't government and it's not that bureaucratic. As with any group, if you break the guidelines/rules they just won't want to work with you.

> To add a bit more context, when I was writing the original comment, I was mainly thinking of first-time contributors that don't have any track records, and how the policy would work against them.

No matter what, somebody has to review the contribution. First time contributors get feedback, most of them correct their mistakes and some go on to be regular contributors (like me). Others never respond, and still more others make the same mistakes over and over again.

On the topic, Gentoo has projects like GURU where users can contribute new packages that maybe aren't ready for main tree or where a full developer wouldn't be interested, it's a good place to learn if interested in working towards becoming a developer: https://wiki.gentoo.org/wiki/Project:GURU

totallymike

5 months ago

If nothing else, it gives maintainers a sign to point to when closing PRs with prejudice, and that's not nothing. Bad faith contributors will still likely complain when their PRs are closed, and having an obviously applicable policy to cite makes it harder for them to keep complaining without getting banned outright.

h4ny

5 months ago

You just stop accepting contributions from them?

There is nothing inherently different about these policies that make them more or less difficult to enforce than other kinds of polices.

yifanl

5 months ago

You enforce them by pointing out the policy and closing the issue/patch request whenever you're concerned about the quality of the submission.

If it turns out to be incorrectly called out, well that sucks, but I submit that patches have been refused before LLMs came to be.

bgwalter

5 months ago

You cannot prevent cheating with other policies like the Developer Certificate of Origin either. Yet no one brought up the potential cheating at the time these policies were discussed.

Several projects have rejected "AI" policies using your argument even though those projects themselves have contributor agreements or similar.

This inconsistency makes it likely that the cheating argument, when only used for "AI" contributions, is a pretext and these projects are forced to use or promote "AI" for a number of reasons.

fuoqi

5 months ago

It's often quite easy to distinguish LLM-generated low-effort slop and it's far easier to point to the established policy than to explain why the PR is a complete garbage. On Github it's even easier to detect by inspecting the author's contribution history (and if it's private it's an automatic red flag).

Of course, if someone has used LLM during development as a helper tool and done the necessary work of properly reviewing and fixing the generated code, then it can be borderline impossible to detect, but such PRs are much less problematic.

WD-42

5 months ago

If someone uses an LLM to help them write good code that is indistinguishable from human written code, you are right, it's not enforceable. And that's what most people that are using LLMs should be doing. Unfortunately sometimes it is possible to tell the difference between human and LLM generated code (slop). Policies like this just make it clear and easy to outright reject them.

CJefferson

5 months ago

We do tell bullies not to bully, and then hopefully when they are caught, they are punished. It’s not a perfect system, but better than just ignoring bullying happens.

sensanaty

5 months ago

https://xkcd.com/810/

To me the point is that I want to see effort from a person asking me to review their PR. If it's obvious LLM generated bullshit, I outright ignore it. If they put in the time and effort to mold the LLM output so that it's high quality and they actually understand what they're putting in the PR (meaning they probably replace 99% of the output), then good, that's the point

mctt

5 months ago

What happened to curl? The comment is referring to how the curl project is being overwhelmed by low-quality bug/vulnerability reports generated (or partially generated) by AI (“AI slop”), so much so that curl maintainers are now banning reporters who submit such reports and demanding disclosure, because these sloppy reports cost a lot of time and drain the team.

[generated by ChatGPT] Source: https://news.ycombinator.com/item?id=45217858

hjdjeiejd

5 months ago

This is on-brand.

There was a time that I used Gentoo, and may again one day, but for the past N years, I’ve not had time to compile everything from source, and compiling from source is a false sense of security, since you still don’t know what’s been compromised (it could be the compiler, etc.), and few have the time or expertise to adequately review all of the code.

It can be a waste of energy and time to compile everything from source for standard hardware.

But, when I’m retired, maybe I’ll use it again just for the heck of it. And I’m glad that Gentoo exists.

atrettel

5 months ago

At least when I used Gentoo, the point of compiling from source was more about customization than security. I remember having to set so many different options. It was quite granular. Now I just compile certain things from scratch and modify them as needed rather than having an entire system like Gentoo do that, but I do see the appeal to some people.

bombcar

5 months ago

This is exactly why I use it where I use it - on my servers. I don’t need to compile X or X support for programs that could have it, because they’re headless.

mikepurvis

5 months ago

Nix is another route as far as a compile-from-source package manager with lots of options on many packages.

Cyph0n

5 months ago

I feel like most Gentoo folks probably moved over to Nix/NixOS.

The security argument for recompiling from source is addressed by the input addressed (sic) package cache. The customization aspect is mostly covered by Nix package overrides and overlays. You can also setup your own package cache.

yjftsjthsd-h

5 months ago

> I’ve not had time to compile everything from source,

Then use the official binary packages?

> and compiling from source is a false sense of security, since you still don’t know what’s been compromised (it could be the compiler, etc.), and few have the time or expertise to adequately review all of the code.

That would still leave you in a strictly better position, surely? Any other distro would pull the same code and build with compilers, so that attack surface exists regardless.

sgarland

5 months ago

Granted, I wasn’t into Arch at the time, but in the mid-aughts, Gentoo’s forums were a massively useful resource for Linux knowledge in general. That’s why I used it, anyway. The joy of getting an obscure sound card (Chaintech AV-710) to work in Linux, and sharing that knowledge with others, was enough.

jimmaswell

5 months ago

I use it on some systems so strong that most emerges hardly take much longer than a binary package install. It's pretty nice there.

dizlexic

5 months ago

This might get me in trouble, but with all the negativity I’m seeing here I’ve got to ask.

Why do you care? Their sandbox their rules, and if you care because you want to contribute you’re still free to do so. Unless you’re an LLM I guess, but the rest of us should have no problem.

The negativity just seems overblown. More power to them, and if this was a bad call they’ll revisit it.

attentive

5 months ago

> and if this was a bad call they’ll revisit it.

how would they know? - this is (one of) the ways for people to let them know

const_cast

5 months ago

Let's stop bullshitting, nobody here is going to contribute to Gentoo and is now put off because of this policy change.

What we're looking at is mostly JavaScript monkeys who feel personally offended because they're unable to differentiate criticism of their tools from criticism of their own personal character.

The outrage is purely theoretical.

dizlexic

5 months ago

As a JavaScript monkey I believe you have a point, and this was the core of my original question.

How many contributors to gentoo are upset by this? Probably none.

How many potential contributors to gentoo are upset by this? Maybe dozens?

I'll be amazed if this has any notable negative outcomes for Gentoo and their contributions.

joecool1029

5 months ago

It isn't though. This is just noise. It's a good conversation thread for HN, but it has absolutely zero influence on Gentoo policy.

The only way it'll be revisited is if active Gentoo developers and/or contributors really start to push with a justification to get it changed and they agree to revisit discussing it again. I can tell you every maintainer has heard the line: 'I would have contributed if you did X thing'.

h4ny

5 months ago

Not speaking for everyone but to me the problem is the normalization of bad behavior.

Some people in this thread are already interpreting that policies that allow contributions of AI-generated code means it's OK to not understand the code they write and can offload that work to the reviewers.

If you have ever had to review code that an author doesn't understand or written code that you don't understand for others to review, you should know how bad it is even without an LLM.

> Why do you care? Their sandbox their rules...

* What if it's a piece of software or dependency that I use and support? That affects me.

* What if I have to work with these people in these community? That affects me.

* What if I happen to have to mentor new software engineers who were conditioned to think that bad practices are OK? That affects me.

Things are usually less sandboxed than you think.

incomingpain

5 months ago

>Why do you care? Their sandbox their rules, and if you care because you want to contribute you’re still free to do so. Unless you’re an LLM I guess, but the rest of us should have no problem.

Exactly this. It's their decision to make; their consequences as well.

Then again I would have bet $1000 that gentoo disappeared 15 years ago. Probably around 2009? I legitimately havent even heard about them since at least that long.

So rejecting contributions from who might even still be around seems like a bad decision.

adastra22

5 months ago

I like the idea of Gentoo, and I've considered switching back to it. I won't now, as I don't see a future for it if this is the attitude they take towards new technologies.

globular-toast

5 months ago

This seems like the kind of thing you'd want from a distro. Would you be happy if your doctor just started giving you new drugs because they're "new technology"? Or would you prefer it to go through rigorous rounds of testing and evaluation to figure out the potential problems?

adastra22

5 months ago

I certainly hope my medical team is using AI tools, as they have been repeatedly demonstrated to be more accurate than doctors.

Only downside is my last psychiatrist dropped me as a patient when he left his practice to start an AI company providing regulatory compliance for, essentially, Dr. ChatGPT.

PeterStuer

5 months ago

To be fair, any of their 3 objections are absolutely not exclusive to AI, but can be levied against any human contribution as well.

Any contributer, wether openly using AI or covertly, should be reputed by earned merit of the contribution history.

Yes, I know there are still holdouts that realy do not use AI, but that number is shrinking rapidly. A no AI policy when strictly enforced (how?) would probably just lead to project EoL.

bgwalter

5 months ago

It is interesting that so many people are upset about this policy. If "AI" actually worked, you could plagiarize the whole of Gentoo in one hour.

But no, you always have to infiltrate and harass existing projects, because without the actual human developers your poor "AI" "contributions" are nothing.

hsbauauvhabzb

5 months ago

> Their operations are causing concerns about the huge use of energy and water.

I’d be curious how much energy gentoo consumes versus a binary distro.

mrheosuper

5 months ago

I wonder how would they enforce it. If they let a pure LLM Commit into their codebase, then there are a lot of another commit that depends on it, would they remove it and try to submit another commit to replace it, or just let it be ?

DrNosferatu

5 months ago

I would say the question is, do you iterate on the output or not?

My perspective is that this criticism is only valid for “single-shot in spirit” / “prompt and forget” LLM powered contributions.

sumeno

5 months ago

That ignores two out of the three reasons they gave for this decision

user

5 months ago

[deleted]

nextaccountic

5 months ago

Does this apply to upstream packages distributed by Gentoo too? Or just things that the Gentoo project is itself upstream, like emerge?

ares623

5 months ago

Maybe we’ll see a (new) distro with AI assisted maintainers. That would be an interesting experiment.

Unfortunately one caveat would be it will be difficult to separate the maintainers from the financial incentives, so it won’t be a fair comparison. (e.g. the labs funding full time maintainers with salaries and donations that other distros can only dream of)

perching_aix

5 months ago

Dated 2024-04-14 and features nothing special.

tptacek

5 months ago

Interestingly --- while I doubt it would make a difference to the decision Gentoo in particular would make --- the cost/benefit of LLMs for coding changed sharply just a month or two after this, when the first iteration of foundation models tuned for effective agents came out. People forget that effective coding agents are just a couple minutes old; the first research preview release of Claude Code was this past February.

malfist

5 months ago

> the cost/benefit of LLMs for coding changed sharply just a month or two after this

People say this every month.

tptacek

5 months ago

Do they? I'm referring to something specific. While I happen to think LLM coding agents are pretty great, my point didn't depend on you thinking that, only on a recognition of the fact that the capabilities of these systems sharply changed very shortly after they published this --- in a very specific, noticeable way.

sothatsit

5 months ago

Marketing people say this every month, but that doesn't mean there haven't also been actual step-changes in AI-assisted coding in the last year.

The policy is dated to 2024-04-14. After they approved this, there were all of these releases that were all pretty dramatic advancements for coding: 3.5 Sonnet (for taste + agentic coding), o1-preview (for reasoning), Claude Code (for developer experience), o3 (for debugging), Claude 4 Opus (for reliability), and now GPT-5 Pro (for code review).

We have advanced from AI that can unreliably help you look up documentation for tools like matplotlib, to AI tools that can write and review large complex programs in the last year alone. Sure, these tools still have a lot of deficiencies. But that doesn't negate the fact that the change in AI for coding in the last year has been dramatic.

notherhack

5 months ago

Important point. A lot has changed in coding AIs since then.

simianwords

5 months ago

> Ethical concerns. The business side of AI boom is creating serious ethical concerns. Among them: Commercial AI projects are frequently indulging in blatant copyright violations to train their models. Their operations are causing concerns about the huge use of energy and water. The advertising and use of AI models has caused a significant harm to employees and reduction of service quality. LLMs have been empowering all kinds of spam and scam efforts.

Highly disingenuous. First, AI being trained on copyrighted data is considered fair use because it transforms the underlying data rather than distribute it as is. Though I have to agree that this is the relatively strongest ethical claim to stop using AI but stands weak if looked at on the whole.

The fact that they mentioned "energy and water use" should tell you that they are really looking for reasons to disparage AI. AI doesn't use any more water or energy than any other tool. An hour of Netflix uses same energy as more than 100 GPT questions. A single 10 hour flight (per person*) emits as much as around 100k GPT prompts. It is strange that one would repeat the same nonsense about AI without primary motive being ideological.

"The advertising and use of AI models has caused a significant harm to employees and reduction of service quality." this is just a shoddy opinion at this point.

To be clear - I understand why they might ban AI for code submissions. It reduces the barrier significantly and increases the noise. But the reasoning is motivated from a wrong place.

themafia

5 months ago

> AI being trained on copyrighted data is considered fair use because it transforms the underlying data rather than distribute it as is.

It's not a binary. Sometimes it fully reproduces works in violation of copyright and other times it modifies it just enough to avoid claims against it's output. Using AI and just _assuming_ it would never lead you to a copyright violation is foolish.

> uses same energy as more than 100 GPT questions.

Are you including training costs or just query costs?

> But the reasoning is motivated from a wrong place.

That does not matter. What matters is if the outcome is improved in the way they predict. This is actually measurable.

simianwords

5 months ago

>That does not matter. What matters is if the outcome is improved in the way they predict. This is actually measurable.

Ok lets discuss facts.

>It's not a binary. Sometimes it fully reproduces works in violation of copyright and other times it modifies it just enough to avoid claims against it's output. Using AI and just _assuming_ it would never lead you to a copyright violation is foolish.

In the Anthropic case the Judge ruled that AI training is transformative. It is not binary as you said but I'm criticising what appears as binary from the original policy. When the court ruling itself has shown that it is not violation of copyright, it is reasonable to criticise it now although I acknowledge the post was written before the ruling.

>Are you including training costs or just query costs?

The training costs are very very small because they are amortised over all the queries. I think training accounts around .001% to .1% of each query depending on how many training runs are done over a year.

ses1984

5 months ago

The idea that models are transformative is debatable. Works with copyright are the thing that imbues the model with value. If that statement isn’t true, then they can just exclude those works and nothing is lost, right?

Also, half the problem isn’t distribution, it’s how those works were acquired. Even if you suppose models 44are transformative, you can’t just download stuff from piratebay. Buy copies, scan them, rip them, etc.

It’s super not cool that billion dollar vc companies can just do that.

simianwords

5 months ago

> In Monday's order, Senior U.S. District Judge William Alsup supported Anthropic's argument, stating the company's use of books by the plaintiffs to train their AI model was acceptable.

"The training use was a fair use," he wrote. "The use of the books at issue to train Claude and its precursors was exceedingly transformative."

I agree it is debatable but it is not so cut and clear that it is _not_ transformative when a judge has ruled that it is.

perching_aix

5 months ago

> The idea that models are transformative is debatable. Works with copyright are the thing that imbues the model with value. If that statement isn’t true, then they can just exclude those works and nothing is lost, right?

I don't follow.

For one, all works have a copyright status I believe (under US jurisdiction; this of course differs per jurisdiction, although there are international IP laws), some are just extremely permissive. Models rely on a wide range of works, some with permissive, some with restrictive licensing. I'd imagine Wikipedia and StackOverflow are pretty important resources for these models for example, and both are licensed under CC BY-SA 4.0, a permissive license.

Second, despite your claim being thus false, dropping restrictively copyrighted works would make a dent of course I'm pretty sure, although how much, I'm not sure. I don't see why this would be a surprise: restrictively licensed works do contribute value, but not all of the value. So their removal would take away some of the value, but not all of it. It's not binary.

And finally, I'm not sure these aspects solely or even primarily determine whether these models are legally transformative. But then I'm also not a lawyer, and the law is a moving target, so what do I know. I'd imagine it's less legal transformativeness and more colloquial transformativeness you're concerned about anyhow, but then these are not necessarily the best aspects to interrogate either.

shmerl

5 months ago

I don't get this idea. Transformative works don't automatically equal fair use - copyright covers all kind of transformative works.

notpachet

5 months ago

> AI doesn't use any more water or energy than any other tool. An hour of Netflix uses same energy as more than 100 GPT questions. A single 10 hour flight (per person*) emits as much as around 100k GPT prompts. It is strange that one would repeat the same nonsense about AI without primary motive being ideological.

We should stop doing those things too. I'm still surprised that so many people are flying.

simianwords

5 months ago

I agree but the magnitudes are important. I don’t want to give up a few prompts per day because of climate. That is stupid.

user

5 months ago

[deleted]

CursedSilicon

5 months ago

That's quite a strawman definition of "copyright infringement" especially given the ongoing Anthropic lawsuit

It's not a question of if feeding all the worlds books into a blender and eating the resulting slurry paste is copyright infringement. It's that they stole the books in the first place by getting them from piracy websites

If they'd purchased every book ever written, scanned them in and fed that into the model? That would be perfectly legal

infamia

5 months ago

> Highly disingenuous. First, AI being trained on copyrighted data is considered fair use because it transforms the underlying data rather than distribute it as is.

Your legal argument aside, they downloaded torrents and trained their AI on them. You can't get much more blatant than that.

simianwords

5 months ago

Yes but that was one company and it is not core to their infra or product. So I don’t know how one can characterize AI fundamentally to be unethical because one company pirated some books

logicprog

5 months ago

There are reasonable ethical concerns one may have with AI (around data center impacts on communities, and the labor used to SFT and RLHF them), but these aren't:

> Commercial AI projects are frequently indulging in blatant copyright violations to train their models.

I thought we (FOSS) were anti copyright?

> Their operations are causing concerns about the huge use of energy and water.

This is massively overblown. If they'd specifically said that their concerns were around the concentrated impact of energy and water usage on specific communities, fine, but then you'd have to have ethical concerns about a lot of other tech including video streaming; but the overall energy and water usage of AI contributed to by the actual individual use of AI to, for instance, generate a PR, is completely negligible on the scale of tech products.

> The advertising and use of AI models has caused a significant harm to employees and reduction of service quality.

Is this talking about automation? You know what else automated employees and can often reduce service quality? Software.

> LLMs have been empowering all kinds of spam and scam efforts.

So did email.

Veedrac

5 months ago

I get why water use is the sort of nonsense that spreads around mainstream social media, but it baffles me how a whole council of nerds would pass a vote on a policy that includes that line.

const_cast

5 months ago

To be completely fair, AI really does use more water than other typical compute tasks, because AI takes A LOT of compute.

No, it's not like email, or a web server. I can run an email server or apache on my rinky dink computer and get hundreds of requests per second.

I can't run chatgpt, that requires a super computer. And of the stuff I can run, like deepseek, I'm getting very few tokens/s. Not requests! Tokens!

Yes, inference has an energy cost that is significantly more than other compute tasks.

Veedrac

5 months ago

The energy use claims are questionable, but I at least get where they're coming from. The water use is the confusing part. Who looks at a server rack and goes ‘darn, look at how water intensive this is’? People use water as a coolant in large part because it's really hard to boil, plus it's typically cheap because it regularly gets delivered to your front door for free.

As to actual numbers, they're not that hard to crunch, but we have a few good sources that have done so for us.

Simple first-principles estimate: https://epoch.ai/gradient-updates/how-much-energy-does-chatg...

Google report: https://arxiv.org/abs/2508.15734

Altman claim inside a blog post: https://blog.samaltman.com/the-gentle-singularity

simianwords

5 months ago

Because it is ideologically motivated.

bleepblap

5 months ago

>> Commercial AI projects are frequently indulging in blatant copyright violations to train their models. > I thought we (FOSS) were anti copyright?

Absolutely not! Every major FOSS license has copyright as its enforcement method -- "if you don't do X (share code with customers, etc depending on license) you lose the right to copy the code"

pabs3

5 months ago

> I thought we (FOSS) were anti copyright?

For Free Software, copyright creates the ability to use licenses (like the GPL) to ensure source code availability.

AdieuToLogic

5 months ago

>> Commercial AI projects are frequently indulging in blatant copyright violations to train their models.

> I thought we (FOSS) were anti copyright?

No free and open source software (FOSS) distribution model is "anti-copyright." Quite to the contrary, FOSS licenses are well defined[0] and either address copyright directly or rely on copyright being retained by the original author.

0 - https://opensource.org/licenses

bombcar

5 months ago

Some of the ideas behind the GPL could be anti-copyright, insofar as the concept they’d love to see is software being uncopyrightable.

CursedSilicon

5 months ago

>I thought we (FOSS) were anti copyright?

FOSS still has to exist within the rules of the system the planet operates under. You can't just say "I downloaded that movie, but I'm a Linux user so I don't believe in copyright" and get away with it

>the overall energy and water usage of AI contributed to by the actual individual use of AI to, for instance, generate a PR, is completely negligible on the scale of tech products.

[citation needed]

>Is this talking about automation? You know what else automated employees and can often reduce service quality? Software.

Disingenuous strawman. Tech CEO's and the like have been exuberant at the idea that "AI" will replace human labor. The entire end-goal of companies like OpenAI is to create a "super-intelligence" that will then generate a return. By definition the AI would be performing labor (services) for capital, outcompeting humans to do so. Unless OpenAI wants it to just hack every bank account on Earth and transfer it all to them instead? Or something equally farcical

>So did email.

"We should improve society somewhat"

"Ah, but you participate in society! Curious!"

simianwords

5 months ago

> the overall energy and water usage of AI contributed to by the actual individual use of AI to, for instance, generate a PR, is completely negligible on the scale of tech products.

10 GPT prompts take the same energy as a wifi router operating for 30 minutes.

If Gentoo were so concerned for the environment, they would have more mileage forbidding PRs from people who took a 10 hour flight. These flights, per person, emit as much carbon as a million prompts.

logicprog

5 months ago

> [citation needed]

Sure, here ya go:

https://andymasley.substack.com/p/individual-ai-use-is-not-b...

https://blog.giovanh.com/blog/2024/08/18/is-ai-eating-all-th...

https://blog.giovanh.com/blog/2024/09/09/is-ai-eating-all-th...

The first comprehensive environmental audit and analysis performed in conjunction with French environmental agencies and audit environmental audit consultants, which includes every stage of the supply chain, including usually hidden upstream costs: https://mistral.ai/news/our-contribution-to-a-global-environ...

https://andymasley.substack.com/p/for-the-climate-little-thi...

> Disingenuous strawman. Tech CEO's and the like have been exuberant at the idea that "AI" will replace human labor. The entire end-goal of companies like OpenAI is to create a "super-intelligence" that will then generate a return. By definition the AI would be performing labor (services) for capital, outcompeting humans to do so

Isn't that literally the selling point of software, performing something that would otherwise have to be done by humans, namely both keeping calculating research, locating things, transferring information, and so on, using capital instead of labor, transforming labor into capital, and providing more profits as a result?

> Unless OpenAI wants it to just hack every bank account on Earth and transfer it all to them instead? Or something equally farcical

It's extremely funny that you pull this out of your house and say that this is the only way that I could be justified in saying what I'm saying, while accusing me of making a disingenuous straw man. Consult the rod in your own eye before you concern yourself with the speck in mind.

> >So did email.

> "We should improve society somewhat"

> "Ah, but you participate in society! Curious!"

Disengenuous strawman. That comic is used to respond to people that claim that you can't be against something if you also participate in it out of necessity. That's not what I'm doing. I would be fine with it if they blank it condemned all things that enable spam on a massive level, including email, social media, automated phone calls, mail, and so on, while still using those technologies because they have to to live in present society and get the word out. There are people who do that with rigorous intellectual consistency and have been since those things existed. My argument is by condemning one but not the other irrespective of whether they use them or not, they are being ethically inconsistent and it shows a double standard and a biased towards technologies that they're used to over technologies that they aren't. It shows a fundamental reactionary conservatism over an actually well thought through ethical position.

ericdotlee

5 months ago

Humans are important - but I have to wonder how any of this will be enforced?

mmaunder

5 months ago

Posted April 2024. I wonder how they feel about this now. Or will next year. Claude Code wouldn’t exist for another year when this was posted. Nevermind Codex. It’s already awkward. Within 12 months it will be cringeworthy.

drnick1

5 months ago

I don't understand this anti-AI stance. Either the code works and is useful, and it should be accepted, or it doesn't work and it should be rejected. Does it really matter who wrote it?

q3k

5 months ago

The code is only a projection of someone's mental model, which is what actually allows the project to succeed, especially in the long term.

That's why codebases die when they lose maintainers and forks often don't make it past the first few months.

LLM-generated code might work, but it's not backed by anyone's mental model. And the industry has had a long running term for code which is there but no-one understands it nor the reason behind it: legacy code.

miningape

5 months ago

Because the quality of the code matters more than the raw quantity.

Having code that compiles and runs is the bare minimum - we should (and do) hold ourselves to a higher standard of professionalism.

globular-toast

5 months ago

Putting together and maintaining a GNU/Linux distribution and maintaining it for 23 years requires a bit more than "works for me and is useful".

cuu508

5 months ago

The linked API policy lists specific concerns in 3 categories: copyright, quality, ethical. Which one do you not understand?

drnick1

5 months ago

I don't care about "ethics" in the abstract if the code works and is of good "quality" (however you choose to define that). AIs don't have copyright over anything they generate, so that's a non issue. In fact, if the code is any good, it should be impossible to tell if it was written by AI at all.

sensanaty

5 months ago

LLMs give idiots the power to effectively DDoS repos with useless slop PRs that they have to expend the time and effort to triage and ignore. Like the curl maintainers have said, the review burden of looking at mountains of AI-generated crap is horrifically time consuming.

danpalmer

5 months ago

This is a prime example of poor AI policy. It doesn't define what AI is – is using Google translate in order to engage on their mailing lists allowed? Is using Intellisense-like tools that we've had for decades allowed? The rationale is also poor, citing concerns that can be applied far more widely than just LLMs. The ethical concerns are pretty hand-wavy, I'm pretty sure email is used to empower spam and yet I suspect Gentoo have no problem using email.

The end result is not necessarily a bad one, and I think reasonable for a project like Gentoo to go for, but the policy could be stated in a much better way.

For example: thou shalt only contribute code that is unencumbered by copyright issues, contributions must be of a high quality and repeated attempts to submit poor quality contributions may result in new contributions not being reviewed/accepted. As for the ethical concerns, they could just take a position by buying infrastructure from companies that align with their ethics, or not accepting corporate donations (time or money) from companies that they disagree with.

Spivak

5 months ago

Or because this is a policy by and for human adults who all understand what we're talking about you just don't accept contributions from anyone obviously rule-lawyering in bad faith.

This isn't a court system, anyone intentionally trying to test the boundaries probably isn't someone you want to bother with in the first place.

danpalmer

5 months ago

This policy being so specific in what it bans means that you can't enforce it easily against people who are close but technically within the letter of the policy, and you create a grey area and friction for those who are meeting the spirit of the policy in good faith, but technically in violation.

I have friends and colleagues who I trust as good engineers who take different positions on this (letter vs spirit) and I think there are good faith contributions negatively impacted by both sides of this.

dmead

5 months ago

> It doesn't define what AI is

this is a bad faith comment.

malfist

5 months ago

The whole argument smacks of bad faith "yet you participate in society" arguments.

danpalmer

5 months ago

Honestly, I tried to make this in good faith. The examples I gave were perhaps extreme, but my point is that AI is a moving target. Today it means specifically generative AI done by large models – usually not classification, recommendations, and usually not "small" models, all of which have been normalised. LLMs are becoming normalised, and policy needs to be able to keep up to the shifting technological landscape.

Defining policy on the outcomes, rather than the inputs, makes it more resilient and ultimately more effective. Defining policy on the inputs is easy to dismantle.