PgQue: Zero-Bloat Postgres Queue
65 points by gmcabrita 7 hours ago | 6 comments

odie5533 2 hours ago
Postgres durability without having to run Kafka or RabbitMQ clusters seems pretty enticing. May reach for it when I next need an outbox pattern or small fan out.
reply
saberd 2 hours ago
I don't understand the latency graph. It says it has 0.25ms consumer latency.

Then in the latency tradeof section it says end to end latency is between 1-2 seconds.

Is this under heavy load or always? How does this compare to pgmq end to end latency?

reply
samokhvalov 11 minutes ago
(PgQue author here)

I didn't understand nuances in the beginning myself

We have 3 kinds of latencies when dealing with event messages:

1. producer latency – how long does it take to insert an event message?

2. subscriber latency – how long does it take to get a message? (or a batch of all new messages, like in this case)

3. end-to-end event delivery time – how long does it take for a message to go from producer to consumer?

In case of PgQ/PgQue, the 3rd one is limited by "tick" frequency – by default, it's once per second (I'm thinking how to simplify more frequent configs, pg_cron is limited by 1/s).

While 1 and 2 are both sub-ms for PgQue. Consumers just don't see fresh messages until tick happens. Meanwhile, consuming queries is fast.

Hope this helps. Thanks for the question. Will this to README.

reply
rgbrgb 39 minutes ago
[dead]
reply
bfivyvysj 6 minutes ago
Cool
reply
cout 2 hours ago
I think it's great that projects like this exist where people are building middleware in different ways than others. Still, as someone who routinely uses shared memory queues, the idea of considering a queue built inside a database to be "zero bloat" leaves me scratching my head a bit. I can see why someone would want that, but once person's feature is bloat to someone else.
reply
pierrekin 2 hours ago
In Postgres land bloat refers to dead tuples that are left in place during certain operations and need to be vacuumed later.

It’s challenging to write a queue that doesn’t create bloat, hence why this project is citing it as a feature.

reply