malisper
39 minutes ago
Hey author here. Wasn't expecting to see this up.
To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture.
I'm now working on a new, not yet published version of pgrust that incorporates a lot of techniques. Currently the new version:
- Passes 100% of Postgres regression suite
- Implements a thread per connection model instead of the process per connection model Postgres does
- Is 50% faster than Postgres on transaction workloads
- Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
If you have any questions, I'm happy to answer them.tudorg
20 minutes ago
> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?
In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax
It is currently about 30-40% slower than ClickHouse (single node, ofc). The PR to add it to clickbench was just accepted, so you can see the comparison here: https://benchmark.clickhouse.com/#system=+liH|_etx|gQ|saB&ty...
gnull
14 minutes ago
What was your methodology and structure in making the prompts for the rewrite? Did you let the LLM roam in all of the codebase and tests from the beginning, or revealed things to it gradually in some way?
jl6
26 minutes ago
"Is 50% faster than Postgres on transaction workloads" - That is a very big claim! 50% faster on everything? Is it a strict improvement across the board or are there tradeoffs that make some workloads slower?
malisper
18 minutes ago
The 50% is specifically on percona-tpcc[0]. I got there through a mix of batching (postgres processes a row at a time), prefetching, and several handful of other optimizations.
[0] https://github.com/Percona-Lab/sysbench-tpccbarrkel
32 minutes ago
Is it being used in production anywhere, even if only a toy app?
I know you say it's not production ready and not optimized yet, but in the same breath - in your comment here - you say it's already faster.
malisper
29 minutes ago
It's not used in production. I've been using different benchmarks to compare the performance vs other systems. Namely sysbench-tpcc[0] and clickbench[1]