Bridged Indexes in OrioleDB: architecture, internals and everyday use?

76 pointsposted 9 days ago
by pella
(orioledb.com)

16 Comments

rubenvanwyk

9 days ago

Cannot wait for OrioldeDB to reach General Availability. Postgres needs options for open-source separation of storage and compute.

dkhenry

9 days ago

OrioleDB isn't a separation of storage and compute, its a more efficient storage engine for Postgres to replace the existing HEAP engine. This is like how in MySQL we could swap MyISAM for InnoDB and eventually RocksDB.

I did some benchmarks on it previously to show how much of an improvement it gives over the stock HEAP engine

EDIT: correct link to the public dashboard below, thanks for the heads up @kiwicopple

https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N

iamdanieljohns

9 days ago

Is the need for Oriole negated by using a system that separates storage from compute like Neon, Xata?

nikita

9 days ago

(Neon CEO)

Not really. OrioleDB solve the vacuum problem with the introduction of the undo log. Neon gives you scale out storage which is in a way orthogonal to OrielDB. With some work you can run OrioleDB AND neon storage and get benefits of both.

akorotkov

9 days ago

> OrioleDB solve the vacuum problem with the introduction of the undo log.

Way more than just this!

> With some work you can run OrioleDB AND neon storage and get benefits of both.

This would require significant design work, given that significant OrioleDB benefits are derived from row-level WAL.

tudorg

9 days ago

Answering on behalf of Xata, it is orthogonal. I'm curious to try out Oriole on our platform when I get some time.

the_duke

9 days ago

That kind of exists thanks to NeonDB? https://neon.com

Althoug with them being recently acquired by Databricks it remains to be seen how the open source version will fare.

akorotkov

9 days ago

> That kind of exists thanks to NeonDB?

This is unrelated to NeonDB. OrioleDB has been acquired by Supabase. https://supabase.com/blog/supabase-acquires-oriole

mattashii

8 days ago

Neon is indeed unrelated to OreoleDB, but Neon does also provide the separation of storage and compute in Postgres which GP asked about ("Postgres needs options for open-source separation of storage and compute"). A mention of Neon (which is Apache 2 licensed) therefore isn't totally unwarranted.

rubenvanwyk

7 days ago

I understand Neon is open source and I think it’s an awesome product, but apart from the risks associated with longevity of open source once a company gets acquired - although the storage engine is open source, the control plane isn’t and is non-trivial to implement oneself. Orioldedb is positioned as a Postgres extension, which is must easier to setup (even on an existing operating database) than migrating to a completely different architecture that Neon provides.

apavlo

9 days ago

"Bridged Indexes" is a non-standard term. These are just secondary indexes using logical pointers with a mapping index. IIRC, Oracle, Hana, and HyPer do the same thing.

Source: https://db.cs.cmu.edu/papers/2017/p781-wu.pdf (see Table 1 + Section 6.1)

_joel

9 days ago

> secondary indexes using logical pointers with a mapping index.

rolls off the tongue.

mattashii

8 days ago

It's more than just one mapping index, though.

The CMU paper indicates the logical keys are either TupleID or Primary Key, while the bridged index is actually a TupleID that resolves to a Primary Key, which resolves to the actual tuple - one more level than indicated by 6.1's explanation of logical pointers.