For email you can use a classifier

One way: separately embed sender, recipients, subject, body - then use the embedding vectors as input to a logistic classifier

With that setup, I get 95% accuracy on email classification, training on 50-100 base examples. The model trains on CPU in under 1min, and it does inference in under 20ms (most of it is running the embeddings, so you can make it faster if you train your own embeddings model)

Here’s a gist with some sample code: https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecda...

That code applies the embeddings + classifier setup on the Banking77 dataset. It gets 93-94% accuracy depending on the embeddings you use (SOTA for this is ~95%, with much bigger and slower models)

> hold your horses to paint it as dirt cheap

For a moment I thought this was going to be a metaphor — maybe an ancient Chinese proverb about how paint brushes are made from horsehair and how you can't hold the horse to paint before you've turned the hair into a brush.

How are you benefiting from prompt caching for simple classification?

Are you getting better performance from an LLM than a Bayesian classifier?

How many requests per second do you have for spam that you are reliably hitting the Luna cache?

Is that just because the Jev implementation is less mature? Couldn't it also implement prompt caching?

What are the costs compared to an ML model?

Could you not just copycat jev and run a fast, small local model?

[dead]

cool to see!

Sounds like the story of OpenAI in 6 years

Damn, what a way to go.

My bots are all named Jeeves lol. I have a CLI tool I use that connects up to a LLM I made and I call it Jeeves too ...so funny. I really didn't use Jeeves all that much I tended to use...I think it was called Web crawler pre-google era

this is exactly why Jev doesn't have thinking.

when you want a machine to reason about the prompt and generate a structured output not using an actual LLM makes no sense. I have been doing it since the first chain of thought open models became available.

perhaps there may be a way to get a Jev-type model to think for a very specific number of steps to gain control over its latency, if so that would be the next step. truncating LLM thinking like this does not work well, and its thinking isn't efficient anyway.

Not quite.. that boolean is about whether the voltage exceeds some threshold. It's not about how close the voltage is to the circuit's maximum possible threshold, or how much it exceeds the threshold.

In an analog circuit, maybe.

force the user to decide in the end

And that's...bad?

That seems like the only possible way to eliminate hallucination, short of a model that is never wrong.

Wait til you hear about how digital circuits work at die level

jeeves is currently the name of my local hosted assistant, in its context there are rules that tell it to behave like good old jeeves.

soon I'll make sure that my home assistant pod answers to "Hey jeeves"

Doesn't the fact that it's general purpose warrant a new term? It's partly that it doesn't need to be trained, but it's also able to play games based on game state, I'd imagine it would be hard to train a classifier to do something like this because you'd need to represent a good distribution of all the states. The general purpose llm world understanding underneath it allows for this.

I've used it to do web research where it follows the most appropriate links, decides what to record in state, etc. I struggle to see how you could implement something with a classifier. That said, I have no idea how deep the technology is and it might be replaced with open source pretty quickly since its drafting of the frontier models and the open source models seem almost as good.

I like the term decision model and I think it's warranted.

it's a generalistic classifier w/training needed tho. as far as I'm concerned that is somehwat novel and very practical. you get an ok classifier without working out training data and whatnot. sure, you can do classic ML experiments, but the closest that comes to mind is auto ml. not sure how far we got there, it's been a minute for me on this front.

so I don't agree on the premise that it's "just a classifier". it's not something earth shattering, but practical nonetheless

/e: there's another post from sebastian raschka on this: https://magazine.sebastianraschka.com/p/classifier-history-a...

https://news.ycombinator.com/item?id=49845172

I guess it's not really a benchmark but you could say if it can do it faster it sort of could be taken as one.