LLMs, and other transformer models before them like BERT for example, have the benefit of "transfer learning."

If you want to make a ticket classification model, you don't need to train across many customers doing ticket classification, who might not want you to train on their data and whose data is impossible to obtain unless you already are a large company. Instead, you can train on a completely different task which forces the model to learn the syntactic and semantic structure of the English language — say predicting missing words in Wikipedia articles, which you can't do successfully without a good understanding of English — and then fine-tune that model on your specific task with much less data.

Because the initial task is so general, you can have one company do the expensive pretraining, share the base model, and then have everybody else just do the finetuning on their (small) proprietary dataset.

LLMs and models like Jev go one step further, letting you use them as classifiers with no training at all, with nothing but rules in the prompt. And with a few tricks, you can indeed use them to get pretty high-quality labels, which you can then distill into a fine-tuning dataset for a much smaller model.

Agree to all of this but I would make a minor correction: the interpretability is true under certain conditions, e.g., you use something like n-grams and train an SVM with a linear kernel. You could change the representation back then too, e.g., LDA (topic models), Latent Semantic Indexing, and of course, you could change the SVMs kernel as well, e.g., use a sequence kernel [1]. Some of these were much less interpretable. But the common pipeline back then was indeed n-grams + Linear SVM. Today it is easy to modernize the pipeline by just changing the representation step to use an embedder like ModernBERT. I have a "one page" classifier if you want to try it out - [2].

[1] I went down the rabbit hole of using sequence kernels quite a bit back then, see https://arxiv.org/abs/1910.13800, but the timing was terrible. RNNs and variants like LSTMs were beginning to become popular, and everyone (including me) switched.

[2] https://pocket-text-lab.abhishek-ghose-82.workers.dev/

And you can now get that world knowledge out of llms and generate synthetic data to train your svm and fine tune on your own dataset!

I have seen Phd in stats validating models with data the models have been trained on. And I don’t live in a country with dubious universities.

So between llm and pseudo experts, at this stage, I prefer to go with a LLM.

> Building an SVM isn't a tricky job, but you probably need a data scientist involved, and to think carefully about a labeled dataset.

Well not anymore... You can just get AI to build it.

although in our experience, synthetic data != real data. You need to be quite careful about the distribution (or just try to get public real data, or label stuff yourself)

non-LLM models just aren't visible to consumers.

Saying that the field got reduced to only LLMs is like saying that the field of combustion engines got reduced to cars. The trucks, combine harvesters and peaker plants are still here, they're just in the background, invisible to people who are not part of the industry.

Banks still use random forests and SVMs, medicine and image processing still use CNNs (and people who take photos with their phone don't even realize how much image processing is being done), TTS, ASR and predictive typing models are in wide use, and there's probably still plenty of BERT in spam classification and such.

The ability to do "everything" is an emergent property of how they are built and trained.

If you want a model that can do X but don't care about Y And Z, intuition says you only have to train the model on data relating to X. However, early LLM research showed that doing this results in a far less X-capable model than one trained on data for Y and Z as well.

> In software engineering, we learned a long time ago that giant monoliths are not the answer.

That depends on the question. If the question is "How do I utilize 10,000 software engineers?" then yes, a monolith is not the answer. However, this isn't a problem most companies face. Starting out with microservices is rarely a good idea either as you'll have a hard time getting the boundaries right.

I was thinking about this recently as well. Smaller, specialized models that can run on device just make so much more sense to me than these huge generalized models that need huge infrastructure run.

I am not sure a best practice in one area translates nicely to another. LLM's power comes with it being a monolith.

> single model that can do everything

"AGI"

You can can't build an SVM-classifier for a ticketing system without domain data. You can ask an LLM to do it, and it will do it very well.

With the LLM you don't even need a data scientist to train the model, you just need someone who can connect to APIs. And the LLM is fine with adding and removing categories. No retraining.

It makes the initial faces incredibly fast. Then it is slow and expensive to run, so once the system settles, it becomes the target for model optimization, e.g, a dedicated, or semi-dedicated model.

I can spin up a classification solution with an LLM in ca. 5 minutes that is likely to do an okay job. To build a custom classifier, I need lots of training data first.

That is the reason.

Once you then have a system that is used enough, it starts making sense to look to replace that with a leaner classifier, but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.

There are absolutely good reasons to use smarter things for classification. Svm with tdf idf for example is super fast but will make very bad classifications at times in ways entirely unfixable with that system (thinking back to bats being an acronym for a theology degree classification or similar)

This is a bit too radical and misses some nuance. In order to start building classifiers you at least need some ML domain knowledge, like what to google at least. Meanwhile, point anyone at the jev/open-jev-clones API and have them start classifying in <10 minutes. There's some value there. (same used to be valid for LLMs before jev became a thing. You can use llms as zero-shot classifiers with the right API wrapper, w/ no domain data).

1000x more expensive run can be overall cheaper. You avoid upfront training/tuning cost. It all depends on your scale.

Exactly. For folks working at smaller companies, think of it as "if you'd Slack someone to ask them to do something, that's a ticket" just purely because Slack doesn't scale to dozens of people needing to figure out which person even needs to get messaged in a given minute of the day.

At my company we also do the same every request to engineering (alarms, security, bug, customer support escalation) gets a ticket.

In some big companied many core corporate teams which hate that level of transparency and build their own opaque systems. I've seen this for departments like IT, HR, legal, etc.

I'd rather have tickets for everything and full transparency except for confidential info.

Perhaps killing the need for tickets will be the real value unlock of llms. (Kidding! Kind of)

I think for this purpose a small model will do reasonably well

Should have rephrased it as a "decision model" it's all a bit confusing terminology to keep up these days. "Noul" I guess!

Oh i see.

Never heard of Strands before, felt complelty different when reading it more like a sales pitch from a consulting company

You'll be surprised how fast things move when there's European GDP levels of investment going into an industry.

I think this is similar to how small changes in phrasing can completely change the answer you get from an LLM. This all goes to confirm that LLMs are, at their core, still text completion engines, they (frequently / always?) don't have a firm semantic grip on the topic at hand.

Here's an interesting reference on this that gives a bit more background information:

-- 8< --

Because LLMs (and many other ML systems) are chaotic, it is possible to manipulate them into doing something unexpected through a small, apparently innocuous change to their input. These changes can be illegible to human observers, which makes them harder to detect and prevent.

For example, flipping a single pixel in an image can make computer vision systems misclassify images. You can replace words with synonyms to make LLMs give the wrong answer, or introduce misspellings or homoglyphs. You can provide strings that are tokenized differently, causing the LLM to do something malicious. You can publish poisoned web pages and wait for an LLM maker to use them for training. Or sneak invisible Unicode characters into open-source repositories or social media profiles.

-- >8 --

Source: https://aphyr.com/posts/412-the-future-of-everything-is-lies...

You're (understandably) a bit mixed up here. Strands Agents is an agent SDK. Strands Decider is a decision model.

Yes, this is more of less an advert, using their internal adoption as a case study

I yearn for the day of going back to make v. buy, not buy v. buy

small, open source, decision model optimized for fast experimentation, local development, and innovation...

But just think of the street cred!

Just because you aren't running into problems doesn’t mean the platform is perfect; it might simply mean your standards are lower.

Use some logic before attacking: Luxembourg is only the size of Saarland (German state). Fewer orders and fewer deliveries naturally mean fewer problems.