When I teach programming, some students, when stuck, will start flailing around - deleting random lines of code, changing call order, adding more functions, etc - and just hoping one of those things will “fix it” eventually.

This feels like the LLM-enabled version of this behavior (except that in the former case, students will quickly realize that what they’re doing is pointless and ask a peer or teacher for help; whereas maybe the LLM is a little too good at hijacking that and making its user feel like things are still on track).

The most important thing to teach is how to build an internal model of what is happening, identify which assumptions in your model are most likely to be faulty/improperly captured by the model, what experiments to carry out to test those assumptions…

In essence, what we call an “engineering mindset” and what good education should strive to teach.

I am waiting for a llm entusiast to create something like MicroQuickJS from scratch.

TI had similar idea with TI-99/4 - running interpreted BASIC programs using BASIC written in special interpreted language (GPL) running in its own virtual machine, with actual CPU machine code executing from ram accessible thru single byte window of Video processor. Really brilliant system, turtles all the way down.

I wouldn't trust it without a deeper inspection. I've had Claude do a workaround (ie: use a javascript interpreter and wrap it in Python) and then claim that it completed the task! The CoT was an interesting read on how his mind think about my mind (the user want ... but this should also achieve this ... the user however asked it to be this ... but this can get what the user want ...; that kind of salad)

yt-dlp/youtube-dl used a python javascript interpreter to run youtube's JS until recently

idk how complete it is but it solved youtube's challenges etc for a long time

https://github.com/yt-dlp/yt-dlp/blob/6d92f87ddc40a319590976...

You should release it, it'd be quite useful.

How many tests do other JS runtimes like V8 have? ~400 tests sounds reasonable for a single data structure, but orders of magnitude off for a language runtime.

But why Python? Why not a JVM like Graal? I would think that would yield faster code.

Or why not run MicroQuickJS under Fil-C? It's ideal since it has not dependencies.

I just ask it to generate mermaid diagrams, which are just code that you can render using the mermaid diagram website

That may be true, but it does seem like OP's intent was to learn something about how LLM agents perform on complex engineering tasks, rather than learning about ASCII creation logic. A different but perhaps still worthy experiment.

I guess it depends on well people want to know things like "Perl (and C) library to web" skills. Personally, there are languages I don't want to learn, but for one reason or another, I have to change some details in a project that happen to use that language. Sure, I could sit down and learn enough of the language so I can do the thing, but if I don't like or want to use that language, the knowledge will eventually atrophy anyways, so why bother?

If LLMs stopped improving today I’m sure you would be correct- as it is I think it’s very hard to predict what the future holds and where the advancements take us.

I don’t see a particularly good reason why LLMs wouldn’t be able to do most programming tasks, with the limitation being our ability to specify the problem sufficiently well.

I would think/hope that the code assist LLMs would be optimizing towards supportable/legible code solutions overall. Mostly in that they can at least provide a jumping off point, largely accepting that they more often than not won't be able to produce complete, finished solutions entirely.

Judging from this an approach might have been to port the 28 modules individually and check that everything returns the same data in Perl and TS versions:

"I took a long-overdue peek at the source codebase. Over 30,000 lines of battle-tested Perl across 28 modules. A* pathfinding for edge routing, hierarchical group rendering, port configurations for node connections, bidirectional edges, collapsing multi-edges. I hadn’t expected the sheer interwoven complexity."

Well, ideally we teach the AIs how to divide-and-conquer. I don't care, whether my AI coding assistant is multiple LLMs (or other models) working together.

I ask the LLM to split the task for me. It shines.

>> This doesn’t seem to exist in this case, which may simply be the deal breaker.

Perhaps, but perhaps not. The reason tests are valuable in these scenarios is they are actually a kind of system spec. LLMs can look at them to figure out how a system should (and should not) behave, and use that to guide the implementation.

I don’t see why regular specs (e.g. markdown files) could not serve the same purpose. Of course, most GitHub projects don’t include such files, but maybe that will change as time goes on.

As far as I can tell, no. The relationship isn't the same; in software, the "port" is the translated software itself, not the destination platform.

The etymological roots are quite interesting, though. We aren't quite sure where the word "ship" comes from — Etymonline hazards

> Watkins calls this a "Germanic noun of obscure origin." OED says "the ultimate etymology is uncertain." Traditionally since Pokorny it is derived from PIE root *skei- "to cut, split," perhaps on the notion of a tree cut out or hollowed out, but the semantic connection is unclear. Boutkan gives it "No certain IE etymology."

The word "port" goes back to the PIE root "*per-" meaning "forward", and thus as a verb "to lead". It seems to have emerged in Latin in multiple forms: the word "portus" ("harbor"), verb "portare" (to carry or bring). I was surprised to learn that the English "ferry" does not come from the other Latin verb with the sense of carrying (the irregular "ferre"), but from Germanic and Norse words... that are still linked back to "*per-".

Basically, transportation (same "port"!) has been important to civilization for a long time, and quite a bit of it was done by, well, shipping. And porting software is translating the code; the "lat" there comes from the past participle of the irregular Latin verb mentioned above, about which

> Presumably lātus was taken (by a process linguists call suppletion) from a different, pre-Latin verb. By the same process, in English, went became the past tense of go. Latin lātus is said by Watkins to be from *tlatos, from PIE root *tele- "to bear, carry" (see extol), but de Vaan says "No good etymology available."

> It turns out that having a "trainer" to "coach" you is not a coincidence: these two words evolved together from the rail industry to the gym.

This does not appear to be true.

Train (etymonline):

> "to discipline, teach, bring to a desired state or condition by means of instruction," 1540s, which probably is extended from the earlier sense of "draw out and manipulate in order to bring to a desired form" (Middle English trainen, attested c. 1400 as "delay, tarry" on a journey, etc.); from train (n.) For the notion of "educate" from that of "draw," compare educate.

[That train (n.) doesn't refer to the rail industry, which didn't really exist in the 1540s. It refers to a succession (as one railcar will follow another in later centuries), or to the part of your clothing that might drag on the ground behind you, or to the act of dragging anything generally. Interestingly, etymonline derives this noun from a verb train meaning to drag; given the existence of this verb, I see no reason to derive the verb train in the sense "teach" from the noun derived from the same verb in the sense "drag". The entry on the verb already noted that it isn't unexpected for "drawing" [as water from a well] to evolve into "teaching".]

Coach (wiktionary):

> The meaning "instructor/trainer" is from Oxford University slang (c. 1830) for a "tutor" who "carries" one through an exam

Coach might be a metaphor from the rail industry (or the horse-and-buggy industry), but trainer isn't.

Only because humans before Claude wrote many NES emulators…

Pssst, I can write one in 1 minute.

I use this LLM called git clone.

You just need to know what you are doing. In this case, the problem is not "rewriting the logic" but "mapping Perl syntax to Typescript syntax" and "mapping Perl libs to Typescript libs". In other words, you'd be better off with an old-fashioned script that merely works on syntax mangling along with careful selection of dependencies (and maybe some manual labor around fixing the APIs of the consumers).

This is easy work, made hard by the "allure" of LLMs, which go from emphatic to emetic in the blink of an eye.

If you don't know what you are doing, you should stay away from LLMs if there is anything at all at stake.

Expert vibe engineer sounds as silly as exper stackoverflow copypaster.

> A reader (or dare I say a wiser version of me), armed with a future model and dedicated to the task, will succeed with this port where I failed and that makes me uneasy.

Looking forward to seeing how you get on ;-)

This is true, but also: everything I try works!

I simply cannot come up with tasks the LLMs can't do, when running in agent mode, with a feedback loop available to them. Giving a clear goal, and giving the agent a way to measure it's progress towards that goal is incredibly powerful.

With the problem in the original article, I might have asked it to generate 100 test cases, and run them with the original Perl. Then I'd tell it, "ok, now port that to Typescript, make sure these test cases pass".

> I don't think I know Java well enough to understand the ramifications of choosing Integer instead of integer as a parameter type.

[0]

Java's `int` is a 32-bit "machine" integer (in a virtual architecture, but still stored by value with no additional space overhead). Java's `Integer` is an object with reference semantics, like[1] every value in a Python program — but unlike Python's `int`, it still has the 32-bit range restriction. If you need arbitrary-size integer values in Java, those come from `java.math.BigInteger`.

> It always writes a bunch of utility functions. It refactored simple and direct conditionals into calls to utility functions, which might not make the code very easy to read.

Are the names good, at least? I do this sort of thing and often find it helpful. Of course, that does depend on choosing one utility function for the same task and reusing it, and being sure it actually works.

> I hand wrote manual calls to all the functions, filling in the parameters, which the autocomplete LLM in intellij kept trying to ruin. It would constantly put the wrong parameters places and get in my way, which was stupid.

Yeah, Java lacks a lot of Python's nice tricks for this. (I've had those frustrations with IDEs since long before LLMs.)

> it told me to write log statements everywhere. To be fair, that is how I ended up tracking down the slowdown I was experiencing, but that's because I'm an idiot and didn't intuit that hitting a database on the other side of the country takes a long time and I should probably not do that in local testing.

It sounds like you wanted this for immediate debugging. The word "logging" does not autocomplete "to a remote server db" in my head. Sometimes it's useful to have mental defaults oriented towards what is temporary and quick rather than what is permanent and robust.

[0] Did you consider asking the LLM? It can probably deal with this question pretty well if you ask directly, although I don't know how much it would take to get from there to actually having it fix any problems. But I might as well write a human perspective since I'm here.

[1] Unlike Python, all those "objects with reference semantics" can be NULL in Java (and you need a possibly-third-party annotation to restrict that type to be non-null). There is no "null object" analogous to Python's `None`.

> So now we have 4 broken date parsing functions and 1 working one and that will be a pain that we have to fix in the new year.

Property based testing can be really useful here.

In ordinary English, yes.

In software engineering, "ship" commonly means "distribute" (to a deliberately unspecified audience), while "port" commonly means "software manually translated to another programming language or adapted to another platform".

Yea, but Galen’s a beast so it might actually happen.