I liked Principles of Product Development Flow a lot more because it was easier to digest, although it's a different application of queuing theory.
- "I wish the book was simpler" (Jesse)
- "I wish software engineering was more advanced" (Andrew)
- "I wish the book was more advanced" (Murat)
- "I didn't understand the arguments the author made for why studying single-server exponential response time systems helps with drawing conclusions for time-sharing, heavy-tailed response time systems" (Jesse)
None of these paint the book in a bad colour, as far as I can tell. They say more about the reader's expectations than the book itself.
Supposing that you have “too many” messages in your queue, commanding your frontend client to retry its transaction that would’ve added one more, instead of accepting and enqueuing one additional job, doesn’t seem to me to change much. Instead of creating a mess for whoever is in charge of those servers, the mess is created directly in view of the end user, who sees whatever you show them when their transaction is being retried.
Their point about the bottleneck being the real problem that must be addressed if loads are going to be sustained at such a high level is indisputable, though.
I think I would define the necessary rule as: the queue’s maximum size just needs to be greater than the spikes you expect, but that’s of course no insight, just a definition.
I have found queues to be incredibly valuable at solving situations where load has occasional spikes, but urgency of the jobs being done is low. For instance, every time a user views a piece of content you want to make sure that you increment a counter of how many times the content has been viewed, and you also want to touch the timestamp of when that user last did a thing. If that happens even two hours late, it’s probably gonna be fine. The thing that the queue pattern excels at in the realm of Web applications, especially, is allowing you to have an HTTP GET which can be served entirely by a Web worker that is only allowed to talk to a read replica, which allows extensive horizontal scale. Analytics and other incidentals can be handled async in background jobs (and indeed, in emergencies, load-shedding those ancillary things has barely any impact).
I recognize that all of this probably sounds “obvious” - but I have seen enough codebases that do synchronous writes during GET transactions that I would stop short of calling this “common knowledge.”
There is one truth I have come to know, said by someone far wiser tha me: A queue is either empty or full. Which is to say a queue can either handle all the data coming in, or it can’t. When it can’t it will fill to capacity. This is a probabilistic thing, and you can only decide how many nines to plan for. And it’s worse than it looks at first, because queuing theory is very non intuitive with non linearities that make it very hard to reason about wothout having your nose rubbed in it.
So that means, that yes, you can keep doubling the size of your queue. And no, you can’t ever make it big enough to deal with a poisson distribution. And while you’re at it you will likely need to add workers. And you’re still back to capacity planning and deciding how much money to throw at the problem.
What you may be getting at, and what the article sort of failed at, is that queues are still super valuable for smoothing small spikes, or even large predictable ones. But a queue alone, without backpressure, or overflow will likely cause systems to fall over. Sometimes in ways that are hard to recover from, especially if you have some kind of microservices inspired architecture, where one thing going offline causes another queue elsewhere to fill. Or worse, bringing a failed service back online stresses another system causing it to fall offline. (not meant to be a dig on microservices by the way)
https://news.ycombinator.com/item?id=39041477 - 18 Jan 2024, 153 comments
https://news.ycombinator.com/item?id=8632043 - 19 Nov 2014, 60 comments
TLDR LIFO (stack, not queue) is often a better choice for many workloads, despite violating our sense of fairness.
This feels like an unfinished gripe—summarize the key point now rather than promising a future deep dive. Backing claims with a concise example would make the argument useful instead of vague.
Queues are pretty similar.
The buffer smooths out bursty flow but you don't want that in the middle of the pipeline, as it actually represents mid-pipeline inefficiency. You should actually be fixing the upstream or downstream problem.
[1] or other automation games like Factorio, Mindustry
Understanding the tendency of mid-pipeline buffers to hide problems is useful, but scorning them entirely is also suboptimal.
I think the back-pressure should always be implemented from the very beginning, as it also helps with defining the requirements of what the service should be able to handle