For context, we previously built custom catalogs optimized for specific use cases. But they were hard to maintain, especially as requirements changed, and Apache Iceberg was too heavy for our specific low-latency work.

Since Ducklake is only a spec, we implemented datafusion-ducklake, and it performs as well as any custom or specialized catalog we built. We use Postgres as the catalog store, and it does not get much simpler than that: a transactional database for transactional data.

Plus, it gives us a clear spec for implementing complex parts like time travel, snapshots, etc.

It's been a godsend.

We welcome and encourage contributors!

When it works well it’s really nice. And it beats handrolling a multi level parquet store.

Much faster, but adds a dependency... that you would have added anyway with database based catalogs (that are not the only kind of catalogs)