Have you checked out https://docs.onyx.app/deployment/getting_started/quickstart#...?

We should definitely update the Local guide to reflect the ^, thanks for pointing that out.

For the container footprint - pretty much all of them are necessary to run locally. We hope that just a large # of containers shouldn't be a problem, as long as they are all lightweight (e.g. total resource utilization is low). We of course could bundle them together into a single "mega container", but that seems against the best practices of Docker.

On a related note, we are looking to have a version that doesn't require the vector database (the heaviest part of the system in terms of resource utilization). Our goal is to have a deployment mode with less than 1gb of RAM required.

> every product manager fundamentally misunderstands that users just want a chat window for AI,

I would actually argue chat windows are terrible ui/ux for most cases and users. It does the opposite of `don't make me think`. Too much potential for user error.

Not saying there shouldn't be any LLM integration/features, just that it should be in the form of a button press or something (familiar ux), not the same chatgpt interface that all the early apps are trying to mimic for no good reason.

Haha, yea we've seen that exact story many times! Dissatisfied with Copilot and building a (not great) internal solution that is missing polish + most of the "advanced" feature set.

I immediately thought of Google's Agentspace when I saw this product. The value for me sits in its ability to do RAG via connectors.

> I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI

If you're a non-tech company, why doesn't your org dictate a single model provider? How do these decisions work internally, and how to the departments consume them? (Are they consuming the tools?)

> make their own complicated agents.

Asking a non-tech employee to make an agent sounds like hell.

> we only put up with it because it has access to email, SharePoint and Teams.

Ah, that's how a third party can make money. Bake in external org-wide knowledge and enable search.

Really appreciate the feedback (and glad to hear the core concept resonated with you).

The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left.

In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat.

I've been using Onyx (and Danswer before it) for over a year, and I'd push back on this. We have Freshdesk, Bookstack, Google Drive, YouTrack, and Slack all connected. It seamlessly answers questions like:

"What's Max's GitHub username?" "I need wire transfer instructions for an incoming wire"

We also index competitors' helpdesks and KB articles to track new features they're rolling out. Our tech support team uses it daily because Freshdesk's AI is terrible and their internal KB search is lackluster. Onyx actually finds things. The value isn't in being "one chat to rule them all" — it's in unified search across disparate systems with citations. That's not getting commoditized anytime soon. Keep up the good work, team.

I disagree. This has both APIs as well as connectors. One of the reasons I use Google Workspace as SaaS is because of the extensive API, that gives me the flexibility I need with a great starting point (and continued development, that I continue to benefit from).

Hmm, will have to disagree here. I think "one chat to rule them all" is the way it will end.

It does requires having UI components for many different types of interactions (e.g. many ways to collect user input mid-session + display different tools responses like graphs and interactives). With this, people should be able to easily build complex tools/flows on top of that UI, and get a nice, single interface (no siloed tools/swapping) for free. And having this UI be open-source make this easier.

Have you tried LibreChat? I spun that up to great success inside an org.

Personally, I use Raycast for all my personal work

Yes, exactly! Would love to hear your feedback compared to OpenWebUI

> allow to switch easily using API via openrouter or naga

Ideally not "open"router? It's not open, and don't they charge a margin?

Yep, we have our own implementations! We've spent a lot of time on them, and in our internal benchmarks they compare pretty favorable to the native versions.

RAG specifically is our speciality - we've done a ton of optimizations there (hybrid search, document age-based weighting, giving the LLM the ability to read more from interesting docs and less of irrelevant docs, etc.) and we outperform the implementation within ChatGPT quite substantially in internal blind testing.

Curious what you find if you compare them head to head though!

A few reasons:

1/ No model lock-in / ability to use the ideal model for each use case.

2/ More connectivity. A fuller connector library (contributed to by the open-source community). More built-in tools (similar to the ^).

3/ Customizability and flexibility. If you really need a feature, you can build it rather than waiting months (years?) for your request to go through.

4/ White-labeling. You can make it feel like a product built for/by your company rather than generic.

What format are the docs being uploaded as? By default, images uploaded into the chat would be directly passed through. PDFs would be parsed and fed to the LLM as text.

Writing is a really common use case, and something we'd like to explore more. Currently people often use Onyx for "write something combining X, Y, and Z documents", but I feel that's just scratching the surface.

Hmm, yea that's a great callout. Something we definitely have in our sights longer term (focus for now is to make sure that the desktop chat experience is truly amazing).

I hope you mean "parity" no?

> One thing I really care about is extensibility

Totally makes sense! We've defined simple `Connector`, `Tool` and (soon) `Agent` interfaces to make it easy to plug in your own implementations/apps. If you wanted, you could just use Langchain under our Agent class to build arbitrary flows.

Additionally, the main chat is created from a series of `Renderer` components, and it should be easy to build your own.

Do you think that's in-line with what you're thinking of, or do you want to build outside of those confines?

> canvas mode and scheduled actions

Yes, writing and recurring are two big areas we want to tackle next year.

> The main thing I really care about is voice mode

Interesting. Why do you think that is?

As someone building another competitor in the field, I'll relay some reasons why some of our customers ruled out OpenWebUI in their decision-making process:

- Instability when self-hosting

- Hard to get in touch with sales when looking for SLA-based contracts

- Cluttered product; Multiple concepts seemingly serving the same purpose (e.g. function calling vs. MCP); Most pre-MCP tools suffer from this

- Trouble integrating it with OIDC

- Bad docs that are mostly LLM generated

Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows.

Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.

Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.

Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).

OpenWebUI isn't Open Source anymore. Open WebUI has an egregious CLA if I want to contribute back to it (Which I wouldn't do anyway because it isn't Open Source...)

Onyx Devs: This looks awesome, I will definitely add it to my list of things to try out... close to the top! Thanks, and please keep it cool!

You don't need any subscription to run the code! By default, none of the enterprise code runs (and it can all be completely removed and the app will work as expected). Fully FOSS version here: https://github.com/onyx-dot-app/onyx-foss.

It's not really confusing at all.

Content under backend/ee requires a license, everything else is MIT Expat. Pretty standard stuff.

> Can you call it open source if you need a subscription license to run / edit the code?

MIT is open source, their other stuff isn't. Pretty clear.

That's exactly the same approach employed by Gitlab and is actively being deployed and used by GNOME and F-Droid.

Could you elaborate why this approach is confusing?

Yes. Open source doesn’t mean free.

Outside of pure security, we've seen many choose Onyx for flexibility + connectedness.

One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise).

Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example).

Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors.

Haha, yea the UIs certainly have similarities (much of the industry converges to standard places to put different components, since users are familiar).

"Agents" is a particular area where we feel like we're better than the alternatives (especially if you want something that effectively calls multiple tools in sequence). Curious to hear your thoughts after trying it out!

Exactly. The best they came up with is a generated subject-like summary. So many options to explore here. Categorization by topic, by date, by customer account, clustering by topic, search with various ranking options, conversation(s) tree view, histogram per date/topic/account, integration with email, with an issue tracker, various states per chat/thread e.g. resolved/ongoing/non-viable, a knowledge bank to quickly save stuff you learned (code snippets, commands, facts), integration with Notion or a wiki etc etc. Just off the top of my head.

I was told there would be rapid prototyping with AI. Haven't seen any of the above.

I face the same problem and agree with your "search experience" remark. But it triggered something and raised this idea: just add the search bar to the top of the page instead of hiding it in the sidebar. That would solve 100% of my need to open the sidebar. Maybe show 5 most recent chats under the search bar when I click it before typing anything.

Yes indeed! Our belief is that tool design, compaction (e.g. tool result summarization), and reminders are the things that separate a product that works magically vs one that falls over on any slightly more complex task.

And this is all made all the more important when supporting the wide range of models, even "weaker" open-source models.

Thanks for the kind words!

The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly.

Yes absolutely! There's no license restriction on white-labeling, so we've seen lots of companies do that.

In our opinion, it's a bit silly to build completely in house when you can take something like Onyx as the starting point and be >95% of the way there + have a tons of bells and whistles built in.

Great question! Depends on the specific alternative, but the broad points are:

- "pure chat" experience. From our community (and personal use), we've observed that most queries don't actually involve enterprise search. They much more likely to just require the LLMs internal knowledge (or web search / code execution). Compared to all the companies you've mentioned, we've spent a lot more time refining this more common flow.

- Larger connector suite. As soon as one key source isn't connected, the trustworthiness of the system is dramatically decreased. You second guess "is the info needed to answer this question in there?" for every question. We have a community who builds out connectors for themselves, and then contribute it back for everyone to use. This allows us to cover the long-tail better than companies like Notion and Slack.

- Customizability. An open-source application is the perfect middle ground between a SaaS offering and building blocks. A SaaS option doesn't allow for any customization (we have many customers who have contributed back ux enhancements, small features like guardrails, or enhanced configurations that their users want). Building blocks demand too much domain expertise (search, frontend/UX, ...) for it to be realistic for companies to build something great.

Some of the key differences:

1/ large connector suite + good RAG. Answers at scale is hard, and from our enterprise search roots, we've spent a lot of time with it. It's something that many teams expect from their chat UI.

2/ deep research + open-source code interpreter.

3/ simpler UX. LibreChat has a lot of customizability exposed front and center to the user, which is great for the power user but can be overwhelming for someone new to using AI systems.

What struck me most was the requirement. I've been self-hosting librechat on a raspberry pi. I was hoping to test Onyx on it, but no way. Its requirement is 4 vCPU and 10GB RAM? I wish I can take out components that resource hungry and have only the basic so that I can serve it on a less demanding server.

LibreChat has recently been acquired by ClickHouse, so who knows what their future holds.

You could! We have a number of users who do something like that (e.g. Ramp's in-app chat bot is powered by Onyx)

Thanks! Especially these days, refinement is (by far) the hardest part :)

We have! Likely not for a while, since we want to focus on the web app for a bit more. But long term that would be the plan.

Things like shortcuts to access the chat + ability to reach into the local file system are very compelling.

What is the use case for a desktop app? Just local storage or using local compute for RAG? or perhaps privacy?

Reposting from https://news.ycombinator.com/item?id=46047430:

---

Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows. Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.

Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.

Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).

Thanks! And 404 should be fixed

That's one way people use Onyx! Specifically, the Projects feature (see the left sidebar) works similarly e.g. you can upload arbitrary numbers of files - going well beyond the context limit of your model - and then ask questions of them.

That's a fair point! We were aware of onnx, but felt it was okay since they are very different products so we felt that there wouldn't be too much confusion (people generally know which onyx/onnx they are looking for).

Thanks! Open source is awesome :)

Great suggestion! I haven't heard of it before, but I'll definitely add it to the list.

Would love to hear your feedback after you get a chance to try Onyx out.

Yes, it absolutely does! The chat UX, add-ons (deep research, code interpreter, RAG, etc.), and SSO are MIT licensed. Most deployments of Onyx are using the pure FOSS version of Onyx. Many individuals / teams have done extensive white labeling (something that OpenWebUI doesn't allow).

We have https://github.com/onyx-dot-app/onyx-foss, for a fully MIT licensed version of the repo if you want to be safe about the license/feel freedom to modify every file.

We're definitely looking to add back some of that flexibility / customizability. I don't think you have to sacrifice a nice, simple UI to provide what power users are looking for.

For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice.

You are not the target user.

The target customers are enterprises where 70% of the target users have never used an AI system before and have to go through a AI training before being allowed access to the system.

MIT core + "ee" (enterprise edition) commercially licensed extension subdirectory isn't that strange of a occurrence nowadays.

I also wouldn't pin it as chat app specific. Quite a few VC funded open core software has adopted that pattern post ~2020(?): cal.com, Dagster, Gitlab

Copilotkit is in the same boat. There are parts of the open source codebase that require an enterprise license to use. Basic things like "on error" handlers that are completely offline features. (They might have moved away from this, I haven't checked in a while)

If you tack on these faux-pen source VC licenses and complicate things, you're signaling dishonesty and dark patterns. It might not be the case, but it's not a good look imo. VCs don't seem to care, though - it's all about securing the future payoff, doesn't matter what principles or norms get trampled in the process, and it's only a small set of FOSS nerds that ever get bothered by it, anyway.

Thanks, lawyers, you make everything better!

New tech draws new people. New people have new ideas. Also for licenses.

> As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness.

It's an MIT license. That IS open source.

If they have a commercial strategy - that's a GoodThing. It means they have a viable strategy for staying in business, and keeping the project maintained.

MIT == OpenSource. Pricing == Sustainable. That's a horse worth backing IMO.

Great to hear that you've heard good things. And yea we have many large (>1k+) teams using just the open-source version (something we love to see).

Actually - if you have bunch of VCs on your back, you can't even align with your very own user base let alone any other wider community.

> As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness

I think there can be other valid perspectives than your own.

"Chat UI" can "feel" a bit thin from an eng/product when you initially think about, and that's something we've had to grapple with over time. As we've dug deeper, my worry about that has gone down over time.

For most people, the chat is the entrypoint to LLMs, and people are growing to expect more and more. So now it might be basic chat, web search, internal RAG, deep research, etc. Very soon, it will be more complex flows kicked off via this interface (e.g. cleaning up a Linear project). The same "chat UI" that is used for basic chat must (imo) support these flows to stay competitive.

On the engineering side, things like Deep Research are quite complex/open-ended, and there can be huge differences in quality between implementations (e.g. ChatGPTs vs Claude). Code interpreter as well (to do it securely) is quite a tricky task.

My understanding of YC is that they place more emphasis on the founders than the initial idea, and teams often pivot.

That being said, I think there is an opportunity for them to discover and serve an important enterprise use case as AI in enterprise hits exponential growth.

w24, those were different times.

why?

Totally get this concern.

How we think about it: the chat product should be completely open-source and free (forever). To that end we've moved features like SSO (that used to be "enterprise") to be MIT licensed. The chat interface is something pretty much every team needs (be it a proprietary or open-source solution). You can think of this like Apache Spark for Databricks or Ray for Anyscale.

Also, as other folks have pointed out in the thread, there are quite a few other open source options out there. So there's a a ton of outside pressure for our open-source only offering to be very competitive. We hope this reduces the "enshitification" risk that you speak of.

The truth is more important than your points.

"The reason is because VC needs to show that their flagship investments have "traction" so they manufacture ecosystem interest by funding and encouraging ecosystem product usage. It's a small price to pay. If someone builds a wrapper that gets 100 business users then token use on the foundation layer gets that passed down. Big scheme."