long shot, apropos of nothing, just recognized your name:

If you are the cincinnatian poet Caleb Kaiser, we went to college together and I’d love to catch up. Email in profile.

If you aren’t, disregard this. Sorry to derail the thread.

or how wide the river is year round

Classic shilling behavior of the insufferably embarrassing: redefining words to the benefit of those who pay your bills to the confusion of everyone else.

The definition of engineering, according to people outside the pocket of the llm industry:

> The application of scientific and mathematical principles to practical ends such as the design, manufacture, and operation of efficient and economical structures, machines, processes, and systems.

How do these techniques apply scientific and mathematical principals?

I would argue to do either of those requires reproducibility, and yet somehow you are arguing the less reproducible something is that the more like "physical engineering" it becomes.

Yep. Consider woodworking - the wood you use might warp over time, or maybe part of it ends up in the sun or the thing you’ll make gets partly exposed to water.

Can you make a thing that’ll serve its purpose and look good for years under those constraints? A professional carpenter can.

We have it easy in software.

Physical engineers might scoff good-naturedly at an attempt by project managers to refer to work scheduling as "logistics engineering".

But they really shouldn't because obviously scheduling and logistics is difficult, involving a lot of uncertainty and tolerances.

Lol. This has to be a troll. No way someone seriously wrote this and meant it.

The tools mechanical and civil engineers use are predictable. You're confusing the things these engineers design, which have tolerances and things like that, with the tools themselves.

If an engineer built an internal combustion engine that misfired 60% of the time, it simply wouldn't work.

If an engineer measured things with a ruler that only measured correctly 40% of the time, that would be the apt analogy.

The tool isn't what makes engineering a practice, it's the rigor and the ability to measure and then use the measurements to predict outcomes to make things useful.

Can you predict the outcome from an LLM with an "engineered" prompt?

No, and you aren't qualified to even comment on it since your only claim to fame is a fucking web app

I completely agree that much of software engineering is not engineering, and building systems around LLMs is no better in this sense.

When the central component of your system is a black box that you cannot reason about, have no theory around, and have essentially no control over (a model update can completely change your system behavior) engineering is basically impossible from the start.

Practices like using autoscorers to try and constrain behaviors helps, but this doesn't make the enterprise any more engineering because of the black box problem. Traditional engineering disciplines are able to call themselves engineering only because they are built on sophisticated physical theories that give them a precise understanding of the behaviors of materials under specified conditions. No such precision is possible with LLMs, as far as I have seen.

The determinism of traditional computing isn't really relevant here and targets the wrong logical level. We engineer systems, not programs.

lol. who is “we”? I honestly can’t tell if you’re being serious.

I’m going to start a second career in lottery “engineering”, since that’s a stochastic process too.

>The ways LLMs fail (and the techniques you have to use to account for that) have more in common than physical engineering disciplines than software engineering does!

Ah yes, the God given free parameters in the Standard Model, including obviously the random seed of a transformer. What if just put 0 in the inference temperature? The randomness in llms is a technical choice to generate variations in the selection of the next token. Physical engineering? Come on.

I'd say this shit is even worse that "good at googling". Literal incantation for stochastic machines is like just two notches above checking the horoscope.