This is a great question

You can self-host their code. I don't think there is any official definition of "self hosted" that this violates.

For example - if a "self hosted" service supports off-site backups is it self hosted or just well designed?

That was my observation as well. To be fair their business is to sell a hosted version, they’re under no obligation to release a truly self hosted version.

I’ve never worked in such a space where the deployed environment had unfettered internet access, no access at all actually.

I’ve probably missed a huge wave of programming technology because of this, and I’ve figured out a way to make it work for a consistent paycheck over these past 20 years.

I’m also not a great example, I think I’ve watched 7 whole hours of YouTube videos ever, and those were all for car repair help.

I shy away from tech that needs to be online/connected/whatever.

I consider this to be good open source and I'm a happy user of their OSS offering. Want no hosted dependencies? Then go write it all in Rust.

Boy, that should not be the concern of the end user (developer) but those implementing RAG solutions as a service at Amazon, Microsoft, Openai and so on.

> For the searches we use hybrid dense + sparse bm25, since dense doesn't work well for technical words.

One thing I’m always curious about is if you could simplify this and get good/better results using SPLADE. The v3 models look really good and seem to provide a good balance of semantic and lexical retrieval.

Yep- that's all best practice. I want to know if we could push performance further- routing the query to different embedding models or scoring strategies, or using multiple re-rankers- still feels like the process is missing something.

final tip is to also feed the interpretation of the user search to the user on the other side, so he can check if the llm understanding was correct.

I agree completely with your point, especially the difficulty of developing the user's mental model for what's going on with context and the need to move away from chat UX. It's interesting that there are still few public examples of non-chat UIs that make context management explicit. It's possible that the big names tried this and decided it wasn't worth it -- but from comments here it seems like everyone that has built a production RAG system has come to the opposite conclusion. I'm guessing the real reason is otherwise: likely for the consumer apps controlling context (especially for free users) and inference time is one of the main levers for cost management at scale. Private RAGs, on the other hand, are more concerned with maximizing result quality and minimizing time spent by employee on a particular problem with cost per query much less of a concern --- that's been my experience at least.

I wish there was more info on the article about actual customer usage - particularly whether it improved process efficiency. It's great to focus on the technical aspects of system optimization but unless this translates to tangible business value it's all just hype.

I also built a RAG system about a year back for technical search, everything seems the same!

Shill, not schlep.

I'm correcting you less out of pedantry, and more because I find the correct term to be funny.

S3 Vectors is hands down the SOTA here

SOTA for what? Isn't it just a vector store?

Not necessarily? It's been the basis of one of the major ways people would query their data since 2023 on a product I worked on: https://www.honeycomb.io/blog/introducing-query-assistant

The difference is this feature explicitly isn't designed to do a whole lot, which is still the best way to build most LLM-based products and sandwich it between non-LLM stuff.

rag will be pronounced differently ad again and again. it has its use cases. we moved to agentic search having rag as a tool while other retrieval strategies we added use real time search in the sources. often skipping ingested and chunked soueces. large changes next windows allow for putting almost whole documents into one request.

What do you recommend? Query generation?

Compared with what?

Most of my ChatGPT queries use RAG (based on the query ChatGPT will decide if it needs to search the web) to get up to date information about the world. In reality life it's effective and it's why every large provider supports it.

OP here. We've been working on agentset.ai full-time for 2 months. The product now gets you something working quite well out of the box. Better than most people with no experience in RAG (I'd say so with confidence).

Ingestion + Agentic Search are two areas that we're focused on in the short term.

I'm not sure there is a chunker in this repo. The file you linked certainly doesn't seem to perform any chunking, it just defines a data model for chunks.

The only place I see that actually operates on chunks does so by fetching them from Redis, and AFAICT nothing in the repo actually writes to Redis, so I assume the chunker is elsewhere.

https://github.com/agentset-ai/agentset/blob/main/packages/j...

OP. Reranking is a specialized LLM that takes the user query, and a list of candidate results, then re-sets the order based on which ones are more relevant to the query.

Here's sample code: https://docs.cohere.com/reference/rerank

they just grabbed the better one (3-large) right off the bat. 6x cost to 3-small, but it's still tiny.

OP here. rerankers are finetuned small models, they're cheap and very fast compared to an additional GPT-5 call.

I think you should do both in parallel, rather than sequentially. Main reason is vector scoring could cut off something that an LLM will score as relevant

Have you measured your latency, and how sensitive are you to it?

Which do you suggest?

I think they mean query expansion: https://en.wikipedia.org/wiki/Query_expansion

It's described in the remainder of the point - they use an LLM to generate additional search queries, either rephrasings of the user's query or bringing additional context from the chat history.