Hi all, wanted to share a package we've open sourced called rag-select.

It’s a framework for comparing RAG configurations end-to-end (ingestion → chunking → retrieval → response) on a shared task set, rather than tuning components in isolation. The idea is to make the RAG architecture decision data/eval-driven, especially when balancing quality, latency, and cost across several possible design choices.

We’ve been using this internally to experiment with different setups and to separate RAG evaluation from application code, and it felt general enough to share.

Repo: https://github.com/conclude-ai/rag-select

Some more details on our company page as well: https://useconclude.com/engineering/rag-select

Feel free to reach out if you're interested in using the package and have any questions or if you'd like to contribute as a developer. Thanks!