Bridged Indexes in OrioleDB: architecture, internals and everyday use?

76 points by pella 1 month ago | 15 comments
  • rubenvanwyk 1 month ago
    Cannot wait for OrioldeDB to reach General Availability. Postgres needs options for open-source separation of storage and compute.
    • dkhenry 1 month ago
      OrioleDB isn't a separation of storage and compute, its a more efficient storage engine for Postgres to replace the existing HEAP engine. This is like how in MySQL we could swap MyISAM for InnoDB and eventually RocksDB.

      I did some benchmarks on it previously to show how much of an improvement it gives over the stock HEAP engine

      EDIT: correct link to the public dashboard below, thanks for the heads up @kiwicopple

      https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N

      • CathalMullan 1 month ago
        There's an experimental feature which separates storage and compute.

        https://www.orioledb.com/docs/usage/decoupled-storage

        • iamdanieljohns 1 month ago
          Is the need for Oriole negated by using a system that separates storage from compute like Neon, Xata?
          • nikita 1 month ago
            (Neon CEO)

            Not really. OrioleDB solve the vacuum problem with the introduction of the undo log. Neon gives you scale out storage which is in a way orthogonal to OrielDB. With some work you can run OrioleDB AND neon storage and get benefits of both.

            • tudorg 1 month ago
              Answering on behalf of Xata, it is orthogonal. I'm curious to try out Oriole on our platform when I get some time.
            • kiwicopple 1 month ago
              fwiw I couldn't access your airtable link, but I found this one online:

              https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N

              thanks for running the benchmarks, it helps to have external parties verifying the progress

            • the_duke 1 month ago
              That kind of exists thanks to NeonDB? https://neon.com

              Althoug with them being recently acquired by Databricks it remains to be seen how the open source version will fare.

              • akorotkov 1 month ago
                > That kind of exists thanks to NeonDB?

                This is unrelated to NeonDB. OrioleDB has been acquired by Supabase. https://supabase.com/blog/supabase-acquires-oriole

                • mattashii 1 month ago
                  Neon is indeed unrelated to OreoleDB, but Neon does also provide the separation of storage and compute in Postgres which GP asked about ("Postgres needs options for open-source separation of storage and compute"). A mention of Neon (which is Apache 2 licensed) therefore isn't totally unwarranted.
            • apavlo 1 month ago
              "Bridged Indexes" is a non-standard term. These are just secondary indexes using logical pointers with a mapping index. IIRC, Oracle, Hana, and HyPer do the same thing.

              Source: https://db.cs.cmu.edu/papers/2017/p781-wu.pdf (see Table 1 + Section 6.1)

              • _joel 1 month ago
                > secondary indexes using logical pointers with a mapping index.

                rolls off the tongue.

                • mattashii 1 month ago
                  It's more than just one mapping index, though.

                  The CMU paper indicates the logical keys are either TupleID or Primary Key, while the bridged index is actually a TupleID that resolves to a Primary Key, which resolves to the actual tuple - one more level than indicated by 6.1's explanation of logical pointers.

                • curtisszmania 1 month ago
                  [dead]