> What is Productivity? [1]

> Productivity is a measure of economic performance that compares the amount of goods and services produced (output) with the amount of inputs used to produce those goods and services.

The inputs here are money and the labor of the engineer. With this new form of automation, it takes far less of both. So much less in fact, that a previously uneconomical project is now positively cheap.

If your point is that the result isn't worth what it would have cost before, that's true, but it's the increase in productivity that changed the value proposition and allowed the project to be accomplished economically.

1. https://www.bls.gov/k12/productivity-101/content/what-is-pro...

We literally do say that having more infrastructure is a result of increased productivity.

You say "not worth pursuing", which could also be phrased as "too costly to pursue". With much more capacity available, it becomes possible to do more things. This is true for lots of things.

Is that entirely true?

Before those 200 people would be occupied digging the tunnel.

Now you get the tunnel plus 200 people doing other work in society…

I'm more interested to know if LLMs lead to more intelligent decisionmaking than whether they boost "productivity" by any measure.

Because everyone in the world doing more of the wrong things faster isn't necessarily a good thing.

Agreed, but remember this was a reply under an article about modernizing existing code.

Of course, it's just one example.

>Productivity went down because an engineer spent 2 weeks

Yes, that is something you have to be careful of. In this case, as I mentioned I believe, it was done in spare time, not full time, and it was mostly the AI that spent 2 weeks, I'd guess he probably spent a day or two, and the testers probably spent another day (however, I think this ended up coinciding with our quarterly "full site" test, so it would have happened anyway).

That's a lot of dough, extrememacaroni.

https://www.youtube.com/watch?v=mR3jnW2kcUs

Thanks. Maybe that's what people were talking about in the conversations I've heard that mentioned that 150 figure.

It seems odd that this apparently leaves anyone working solo on a commercial project without a good option. There must be a lot of freelancers or people working at agencies who get software set up for each client or project that would be excluded by all of these options. Although API pricing appears to be so expensive at present relative to the subscription plans that it's probably still worth buying a "team" subscription with two seats and only using one of them in that kind of situation.

I'm not a lawyer - and even if I were I don't know what jurisdiction you're in or what kind of lawyer would be qualified to give you a proper answer to that question.

My personal observation was that when we looked into setting up those monthly subscriptions for some users at a small business in the UK we stopped at the point where they appeared to be offered only under Anthropic's Consumer Terms of Service because those terms clearly state:

"Non-commercial use only. You agree not to use our Services for any commercial or business purposes and we (and our Providers) have no liability to you for any loss of profit, loss of business, business interruption, or loss of business opportunity."

I can see this wording right now at https://www.anthropic.com/legal/consumer-terms but maybe if you're not in the UK you will see different content at that location.

In contrast the API key route appeared to be specifically offered under Anthropic's Commercial Terms of Service and those obviously don't contain the same restriction.

I would be very happy to find that there's some misunderstanding here and Anthropic do actually allow the use of their (very much cheaper) monthly subscriptions instead of API keys for small businesses like ours. But they aren't exactly known for their transparency in either their legal agreements or their billing policies and pricing plans and so until we know better we've chosen to err on the side of caution.

You are the 2028 Developer: https://youtu.be/uYG1TAzexdo

Horrifying was the amount of memory discipline they had to use to make sure every possible message was always copied into buffers that could never overflow, and that didn’t get freed and allocated in unsafe ways. Someone poured enormous effort into enforcing those checks, which a safer language would have enforced automatically. It’s possible they now have compiler layers or tooling that does it for them, but if so those weren’t present for the last batch of CVEs. And if they do have tooling I doubt they use it on all their code because many decisions seem very specific to the buffer sizes they need for BGE parsing.

I was only a passive observer on most of this, but the feeling was “oh look, an unchecked copy, maybe this is exploitable, four layers out towards the attack surface, oh nope they have explicit checks here that lock everything down, rinse repeat N times.”

ETA: Maybe I’ll get GLM to write up its thoughts and post them here later.

Do you even need abliterated GLM to do this? I haven't had it refuse to reverse anything yet.

Of course there's a law for it!

Thanks!

What’s that? Is there an MCP server for it?

Need to keep the bubble alive one more day

All software has bugs, but right now you have a legacy product in production that works, then you have a rewrite that nobody tested. Do you accepted the risk to yolo it?

> to the speed, reliability, and assurance requirements your market segment expects.

I suspect that the average Cisco customer expects a bit more stability than $RANDOM_SAAS.

The alternative is spending money on AI and spending money on someone reviewing the code. But the last one is an hard sell

The self-driving bus factor.

Not if you have a good harness. Docs and tests first

The tests aren't reliable if the entire codebase was rewritten, including the tests, and then nobody read the tests.

> [if] the tests are reliable

Reading the tests is meant to check that.

wait till you see what H1B hiring practices do to american applicants!

The more things change, the more they stay the same.

[flagged]

Bonkheads was released 10 years after Super Mario Bros 3.

It is almost an exact copy of the Mario battle game, Im surprised Nintendo didnt nuke it.

And yet somehow, Windows 11 remains a steaming pile of incoherent garbage.

>So was Oracle and Microsoft Windows.

Cisco did provide updates via CCO and anonymous ftp server. They quite openly told that just download new image and test in your lab, and then if feature set did require licensing, please do it before deploying production. Trust was put on customers and they did very little to check and verify if customers did comply. Which is quite contrary to what for example Oracle practise is been.

>If history has taught us anything. It’s that the measure of a successful product is usually more to do with its sales people.

As someone who worked Cisco gear over thirty years I still remember why us networking did want cisco instead of many other brand devices and it hadn't anything to do with marketing and sales people did.

The Cisco IOS text based configuration was in early days of internet which made it so popular.

Many other manufacturers like HP, 3Com, Cabletron, Madge (IBM), Bridge, etc. did not get why we wanted and complained “give us textual configuration format like IOS has”.

With a plain text based configuration it was easy to adapt configuration from one model to have ready made templates for each model, upgrades and migration to new models were easier, you could use version controls (rcs,cvs, svn,... long before git) and then when RANCiD came it was easy to automate config backups, revert and rollback changes, pass audit with human readable configs and what was in production even long time after etc. In short text based configs made it possible to build large network scale configurations management tools.

IOS has compact, but quite easy to read config snippets made it much easier emailing others, posting examples newsgroups, opening TAC at Cisco CCO, books with examples, learning and teaching etc.

It took quite long before for example HP understood and switched text formats right after turn of century. Juniper seemed to understood right from beginning. But some like Fore, Nortel, Alcatel (ATM-switch makers) and many others did not understand before going out of business. Most of these others had either menus (3Com, HP) or a lousy CLI with obscure unique for just that device model and quite often not even option to print config or if they had not just with changes from default config but printing everything and providing no means to drop showing default. This made it manually while online very hard without external version control to even know what is current status and what had been changes.

BTW, this is still an issue with some manufacturers haven't understood and makes their devices much worse than they understand. Fortigate firewalls CLI is still like this. You can select a subsection of config to print, but it will no indication what differs from default configs. And the whole firewall config is hundreds of kilobytes. Good luck finding what's been changed unless you had saved default config when you started using that device!

When you have thousands to even tens of thousands of network devices in use as many were working you need to be rely that even when you encounter device failure and you don’t have same model any more available, you are able to adapt and find working fix configuring something else doing what is needed.

Binary and any non textual configurations may fit the purpose SOHO category devices, but when network grows even bit larger and configurations more complex there isn’t anything better than text base configs as far as us humans will need to configure and maintain those devices.

Cisco have had issues with software quality for sure, but it was Cisco IOS that was the reason it became so popular and we wanted to use it.

And we did put more weigh on manageability which that IOS text based configuration provided than for example speed some manufacturers got with their devices (Madge, etc.) and even when some Cisco switches dropped some Network World tests more packets we still wanted that manageability more and relied Cisco will fix those issues with software upgrades which it often did.

Juniper when it came end of 90's did understand this and made even better configuration system for Junos than IOS was and still is. Cisco have improved things with IOS-XR, but it's still not up to what Junos is currently.

e: fixed few typos and added that manageability chapter near end. Places where I did work none were always everything Cisco places. No, but either so called ABC (Anything But Cisco) places either. Cisco was and is still popular for those reasons mentioned above.

This is very true, and it's easy to forget that it often has nothing to do with how good/great the product is, unless it is self-evident and thus self-promoting (which for most non-trivial things is simply not the case).

Or perhaps the lack thereof if you are one of the unsuccessful companies. In the 90s, I worked for a company that competed with Cisco I worked for a company that competed with Cisco. story was in the mid 80s an executive went to Silicon Valley. His report was that Cisco was a bunch of nobodies that would never go anywhere, which they'd go down the street and buy Vitalink instead. You have heard of Cisco and not Vitalink for a reason.

It doesn't, but Cisco was great and they've just evolved into a laughing stock. Their stuff is really bad these days.

And they've overstretched their product line to too many things, trying to be the "be all" company. Fortinet is making the same mistake, but they're earlier in that process.

I mean, it's not really a mistake on their parts, their primary goal is to get rich and they are succeeding.

Citation: I've been working with Cisco technologies since the mid-1990s. At my current job, we are a few weeks away from phasing out the last of our Cisco junk. And since people will ask: to Juniper/Mist and Arista.

True. There were few (no other?) options for routers back then. I have fond memories of locking myself out of an ISP's core router then having to power cycle it.

I think the goose farm is related to this (sorry I'm too lazy to find a non reddit link for the screenshot from social media):

https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fv...

#goals

The date picker example is perhaps the best worst they could have picked.

Anyone who has used the built in date picker knows why everyone rolls their own.

That’s what I thought. Then a colleague introduced it for real in a code review.

It doesn’t matter if it’s satire. Developers seem to be so vulnerable to all these AI promises of ‘automating’ everything.

BTW it’s terrible. It gave two very hollow suggestions that did not solve the real problem. Hour wasted, angry colleagues, no code change.

His job isn’t to rewrite all the code. His job is to make the market satisfied that Cisco is “more AI” than competitors.

You forgot to mention the translation company he helped found was also acquired by this same CEO. :))

No, I believe it's called 'favoritism', and it occurs with white people or Indian people, or any group of people.

I'm probably not describing the scale of the issue adequately. Lots of these bugs only show up when you have a specific bit of hardware on a specific version of code, and it's connected to another specific bit of hardware running another specific version of code.

Lots of this stuff should work in theory, you only find out that it doesn't in practice when you try it on real hardware. And there are thousands of bits of hardware.

It's good, but it can't create information that is undiscoverable.

I like your take.

Not that straightforward. Similar to CPUs and other HW, there are errata and interop quirks between devices.

There are also tight time budgets (basically SLAs) that must be met along with memory considerations (both in SW and on the NPU/HBM), not to mention how the device needs to respond to load and various failure scenarios.

Is this really the case?

I'm talking about heterogenous legacy hardware that isn't owned by Cisco, not complex topologies.

Regression tests?

I think people are assuming the worst but AIs most helpful contribution beyond churning out code is explaining the code base.

Who knows if they will do a good or bad job that's down to the engineers and not the tools imo.

Documentation in "plain english", architecture diagrams, comments, and documentation are notorious for being unreliable, out-of-sync, and not as precise a specification as the code itself.

The issue with using "the code as the spec" is that humans can't read 100K lines of code in a day, can't hold the entire ball of unstructured mud in their brains all at once, and generally require hierarchical "structure".

Modern AIs just don't have these issues! Their context windows are big enough to encompass an entire codebase, so they can "see everything", all at once.

Each line of new code they write is compared against every relevant line of code in the original, which they treat as a spec.

My main issue in my experience is that that AIs tended to reproduce the legacy behaviour "bug for bug", which was not my intention.

Similarly, the AIs were hamstrung by external constraints: I typically prefer to leave the DB schema unchanged, which meant the torturous workarounds were needed for design errors in some stored procedures, etc.

The older the code, the bigger the issues with baked-in assumptions about transactions, stateful sessions, data types, string formats, etc.

This trips up AIs just as much as humans. You can't trivially maintain backwards compatibility and modernise the codebase! Those are conflicting goals, no matter what.

> It has a penchant for maintaining API/etc contracts to a fault in the absence of instructions to the contrary.

AI has no magic powers to figure out what the “API/etc contracts” are in a nasty old codebase. This problem is fundamentally hard and not always well defined.

AI is also (at least for now) not consistently good at writing tests that test the behavior that matters. Recent models seem to be getting slightly less bad, but again, this is a hard problem.

I’ve personally used AI to rewrite gnarly code where the goal is to have exactly identical results to the old code. GPT-5.6 Sol could do this correctly with some extra help guiding it to rest the right things, and I haven’t tried this particular task on a newer model.

But ancient code like in the article is probably an enormous buggy mess full of accidental behaviors and implementing protocols in ways that do not fully match the spec. Some, but not all, of the mismatches will be cases where the spec is wrong. Sure, an AI can probably generate a new implementation with a similar set of bugs, but is this what is actually wanted? Also, depending on the gnarly set of bugs, a clean implementation may be unable to reproduce the bugs, leading to a messy implementation instead.

If this was true then Python would already have been rewritten to be GIL-free, and the entire ecosystem of Python packages would have been rewritten to respect its API.

The entire point of a rewrite is to get rid of the behaviour you don't want, while you keep the behaviour you do want. Consider the pathological case: Rewriting your FORTRAN codebase in Rust would obviously be useless if you had to reimplement every bug in the rewrite.

An LLM doesn't have enough context to know what you so and don't want. If you give it call logs for every function that was called and how, maybe, but a seasoned engineer will have more context around what the company needs.

I'd even argue AI is better for maintaining/moderninzing existing code than writing it, just because there is so much less input from developer about architecture it requires, as it has it in the code itself.

But there is pretty big risk of false positives and negatives; some "bugs" are design decisions that were not documented well/at all, and inevitably some new ones will appear when it reasoned something wrong.

> While I certainly don't work at Cisco, I've come to the conclusion that I need to re-examine my priors when it comes to code rewrites and general maintainability.

One of the reasons to avoid rewriting has long been that the time spent rewriting code that works is time not spent writing code to meet new business needs. In other words, it's usually a huge opportunity cost, rarely balanced out by sufficient gains to be net positive.

I do think AI does challenge this prior, as the time needed to do a rewrite shrinks dramatically. It's not zero, of course, but it now becomes worth asking if it's worth doing, rather than defaulting to avoiding rewrites in nearly all cases.

Does the coding assistant demonstrate any tricks to improve the likelihood of success for such rewrites?

They also seem to crave everything that has the word AI in it.

"Java is a DSL for taking large XML files and converting them to stack traces." -Andrew Back

https://www.reddit.com/r/programming/comments/eaqgk/java_is_...

"XML is like violence: if it doesn't solve your problem, you aren't using enough of it." -Chris Maden

https://www.reddit.com/r/ProgrammerHumor/comments/1mbgih/xml...

"XML is a giant step in no direction at all" -Erik Naggum

https://quotefancy.com/quote/1643080/Erik-Naggum-XML-is-a-gi...

Discarded based upon what? They probably think they don't need much human review.

I'm worried for us users of the internet rather than them.

> or who tried out ChatGPT in 2024, saw it make howlingly obvious errors, and wrote off the entire category.

Or they were reading your comments and articles and couldn't reproduce any of the fantastic results that you were claiming, even back then. Things have indeed changed since the first Opus release about a year ago and LLMs are good at programming now. But still, experienced professionals are not so keen to sign off 1000s of lines of code that they never read, take responsibility for it etc. Also, it is a different job and it will take some time to adjust. Can you blame them?

Moreover, other trades have their own guilds or are otherwise protected through regulation. Can you blame programmers for fighting to keep doing things as they were doing it until recently?

You set the agents against themselves and burn tokens like there's no tomorrow.

The feeling of coding like that is a bit like realizing that an elevator is 10cm of aluminum and a 100 meters of empty space under your feet.

After the first few bouts of existential dread you get uses to it.

I write a spec by speaking, not much thought given to grammar and sentence structure. I over-describe concepts, over-describe functionality. A lot of repetition of details and concepts. The audience is not a human. Whisper converts it to text.

Lightly manually edit for speech rec failures.

LLM takes that and converts it into a much much denser, much more structured technical spec in markdown. This is issue 1 in the repo.

I edit this directly extensively, adding detail, removing misconceptions, adding sections with more dictated text.

LLM takes this and further densifies it, closer to an actual implementor’s spec. I carefully note what got added, removed, edited in this step.

I edit this carefully and extensively again. Some removals get added back, some additions get removed. This is the first commit and closes issue 1.

I then have an LLM write an implementation plan and basic project scaffolding. Makefiles, smoke tests, logging, test framework, DI. I edit the plan, and the plan is the second commit. An LLM uses both plus scaffolding plus my repo policy documents (global) to write a README. We’re still single threaded.

An agent files the first n top level implementation tickets. These refer to and sometimes quote any interface contracts in the spec or implementation plan. These usually map 1-to-1 to a module/directory. These sometimes have stub commits to create the module directory and commit interface definitions. Some larger modules have a README in their module directory if their purpose or structure or inputs or outputs are complicated or non-obvious.

Parallel agents write commit level issues that are the task items for each top level implementation ticket. One agent per module.

Then, spawn parallel agents with detailed global instructions, the full spec+plan, and the full readme, and the full code from the module it is working in, plus the issue text. Its output is a single commit on the feature branch and a PR that closes out one feature ticket.

This spawns an adversarial reviewer which has detailed agent instructions as well as our code style guides and repo policies. The scaffolding includes tests and linting and checks. Any failures spawn rework agents until the feature branch is up to snuff.

I review the feature branch diff and merge it, having read the review and rework comments on the PR. I frequently add comments about the implementation (“don’t do x, do y”) and add WIP: back to loop it back for more rework. I don’t see anything that isn’t green, formatted, passes linting, passes tests, and a (dumb) bot thinks is ready to land. Most of the time, it is. Sometimes, it did it the wrong way and I correct it.

The more modular the code, the easier it is to parallelize (obviously). It is ridiculously easy to build large software quickly now, provided it is sufficiently modular so the implementation doesn’t have to cross modules very much. The detailed spec and implementation plan up front helps a lot with this. It’s a lot like writing multithreaded programs really.

Any bugs discovered are tackled with strict TDD, one commit with a failing test, one bugfix commit that must be green and not touch the test.

Individual modules must conform to good contracts, meet best OOP practices, and fit into context. Too big and they get broken down further. Context size is actually a useful constraint for code structure here.

This workflow, the code standards, the styleguide, the linter settings, the fmt settings (obviously only applicable to non-Go as the Go fmt is not configurable), everything is global and stored in another repo and is vendored into each AI managed project repo. It has a fair bit of “I shouldn’t have to say this, but…” stuff (in the style of “fire codes are written in blood” - each warning sign came from experience).

My results are quite good. It’s as the saying goes, much like managing a team of junior devs. It is easy to end up with a LOT of parallel implementors, reviewers, reworkers, and tons of CI, but it’s still so much vastly cheaper than humans that it doesn’t matter.

Someone will make a LOT of money wringing out the inefficiencies from this workflow, because I think the “agent as replacement for third-world contracted junior dev” model is the one almost everyone is using right now.

So is any other hypothesis about why Claude can RE now... You just want to default to some worldview where AI isn't being gamed for certain outcomes, and is some evolving intelligent thing.

Thanks for the very thought provoking reply tho!!!!!!!!!!!