In the ideal case, a text conversation is strictly one thing after another. You ask, you get an answer, you ask a follow-up that only makes sense in light of that answer. Real conversations are messier. People send a correction a few seconds after the question, or split one thought across several texts. Someone I know routinely sends eight messages of three or four words each before the thought is finished. For a human on the receiving end that's hard work: you start composing a reply to the first message, the second one changes the question, and you have to rework the answer before you send it.
A server that answers texts has to handle both of those realities, ordered conversations that arrive in bursts, while serving many people at the same time. For most of RidgeText's life we resolved the tension the simplest possible way: the worker answered one message at a time, for everyone.
This post is about why that stopped being good enough, what broke when we first loosened it, and the design we ended up with. The short version is that RidgeText now runs up to four conversations at once, while each person's own messages are still answered one at a time, in order, with every reply able to see the one before it.
Where we started
When we split RidgeText across two machines in July, the shape was a classic publish/subscribe pipeline. Twilio delivers your text to a small gateway on Cloud Run. The gateway saves it, publishes a message to Google Pub/Sub, and returns immediately. A worker on a virtual server subscribes to that topic, takes one message, calls Gemini and whatever tools the question needs, and texts the reply back through Twilio.
Each Pub/Sub message carried far more than it needed to: the text, any photos, your location, and a full copy of your conversation history, taken the moment your text arrived. With one message handled at a time, that copy was harmless. Once several of one person's messages could be in flight together, each carried its own frozen view of the conversation, and that caused most of the problems later in this post.
The worker also held exactly one message at a time. That was a deliberate choice. The worker shares a small server, and two of its tools were memory-hungry at the time: a headless browser (Playwright) for reading web pages, and a canvas library for drawing maps. A comment in the code justified the limit of one by the risk of several of those running at once.
Duplicate replies
Pub/Sub delivers every message at least once, so occasionally it delivers one twice. The worker recorded a message's ID only after replying, so a second delivery that arrived mid-turn started the same question again, and you got two texts. A camping question takes up to three minutes, which left plenty of time for that to happen.
The fix was to claim the message before starting. A claim expires after five minutes, so a crashed worker doesn't hold a message forever.
The real problem: one slot for everyone
The duplicate replies were rare. The bigger problem was the single slot, and it showed up on October 15, 2026.
I asked RidgeText for a forecast with an infographic. The weather assistant generated an image, hit a temporary rate limit from Google along the way, and then the main model generated a second image afterwards. That one turn took 125 seconds. For those two minutes, every other person who texted RidgeText waited behind it. A customer's one-word reply, "Yes," sat in the queue until my turn finished and was then answered in eleven seconds. So did our canary, a test message we send through the pipeline to check that the worker is alive. Pub/Sub's measure of the oldest unanswered message climbed from zero to five and a half minutes.
The night before, the same thing had happened more quietly. A customer spent an evening refining a flyer with RidgeText's image editor, 44 messages in all. Each edit is an image generation that takes 20 to 65 seconds on Google's servers, and anyone else who texted during that evening waited behind whichever edit was in progress.
What made this frustrating was that the expensive work wasn't happening on our server at all. Image generation runs on Google's machines, and the worker spends those seconds waiting on the network. The browser and map canvas that justified a single slot were used by a small minority of messages. One slot protected against a rare risk by serializing all of the common traffic.
Four slots, and a side effect
We then removed the browser entirely. Web pages and PDFs are now read by Gemini's URL context tool on Google's side, so no browser runs on our server. With it gone, we measured what was left. The worker idles at 230 MB. Its largest single turn, downloading and reading a large PDF, allocates about 120 MB. The process manager restarts the worker at 700 MB, so four of the worst turns at once would sit right at the ceiling, and four typical turns sit far below it. We raised the limit from one message to four.
That fixed the queue, and immediately created a new problem. Four slots don't know whose message is in them. If you send two texts in quick succession, both can be picked up at once and run side by side, and that's wrong in a way that's easy to miss in testing and obvious in use.
Each turn reads the conversation history before it starts. When two of your turns run at the same moment, neither has seen the other's reply. Worse, the second one sees your first question sitting in the history with no answer yet, so it helpfully answers that too.
It isn't only repetition. Image edits are cumulative: "make it a sunset" has to apply to the moose-in-a-canoe picture the previous turn generated, which doesn't exist yet if both turns start together. And once a request to Gemini is in flight, it can't be amended or cancelled, so there's no way to tell the first turn about the second after the fact. The only safe arrangement is to finish one turn before starting the next.
So the requirement became: as many people at once as the server can handle, but only one turn at a time for any one person.
First attempt: a queue per sender
Our first fix kept Pub/Sub as it was and added ordering inside the worker:
- A queue per sender, in memory. Your messages waited in line behind each other and ran in the order they arrived.
- A lease table in Postgres. It recorded which worker was serving which sender, so two workers couldn't serve the same person at once.
This shipped together with the four slots, and it kept each person's messages in order. Three problems remained:
- Waiting messages took up slots. Your second text held one of the four slots while it waited for your first to finish, so one fast texter could take two or three of them.
- The history was still out of date. Each Pub/Sub message still carried a copy of the history taken when the text arrived. A text sent during the previous turn was answered without that turn's reply, which is the repeated-forecast problem in the diagram above.
- Failures were hard to see. A message that failed five times went to a separate dead-letter queue with its own worker, and the only view of the backlog was Pub/Sub's metrics.
All three came from the same source: the Pub/Sub message carried the request and its history, so ordering, retries and the history's freshness all had to be handled through Pub/Sub.
Claiming turns from the conversation table
The gateway was already saving every incoming text to chat_history, the table that holds your conversation, before it published anything. That row has everything the worker needs, and it lives in a database that is very good at answering questions like "what's the oldest thing nobody has started yet?"
So we stopped sending the conversation through Pub/Sub. The worker now reads the request from that row, and the Pub/Sub message shrank to a notification that something new has arrived.
When your text arrives, the gateway saves it with a status of PENDING, along with the few things the worker can't rebuild later, such as photo links and location. Then it publishes a notification containing only your number and the row's ID.
The worker treats that notification as a prompt to check the table. It also looks every 30 seconds and when it starts up, so a lost notification costs at most half a minute. Each time it looks, it claims turns until nothing is waiting or all four slots are busy.
The claim is a single SQL statement that enforces the ordering rule. It finds the sender whose oldest waiting message has waited longest, skipping any sender who already has a turn in progress, marks that sender's waiting messages as PROCESSING and stamps them with the worker's name. A Postgres advisory lock on the sender makes the choice safe when two workers look at once: the second one moves on to the next sender rather than splitting the first one's messages.
This design has four benefits.
- One turn per person, enforced by a query. A sender with a turn in progress is simply not a candidate. Your second text waits in the table, not in a worker's memory, and doesn't occupy a slot while it waits. It is claimed the moment your first reply is sent.
- History is read at the last possible moment. Because the history is no longer copied into the notification, the worker reads it from the table when the turn starts. By then your previous reply is already saved, so every reply is written with the one before it in view.
- Bursts are answered the way a person would answer them. Your first message is claimed and starts immediately. Anything you send while that turn is running waits, and when your sender is next claimed, every waiting message that arrived within 30 seconds of the oldest one is taken together as a single turn. The model is told the messages arrived in a row, with their times, and that a later one may correct, narrow or add to an earlier one. So eight short texts that make up one thought get one considered reply, written with the answer to your first message already in its history, rather than eight replies racing each other. Messages further apart than the window are answered in sequence.
- The backlog can be queried. How many turns are waiting, how old the oldest is and which have failed are all ordinary queries. The admin Health page shows them, and the on-call alert texts us if the oldest waiting turn passes seven minutes.
When things go wrong
Every turn moves through four states, and each failure has a defined path back.
A turn that throws an error goes back to PENDING with a note of the error, and waits 30 seconds, then 2 minutes, then 10, then 30 before each retry. After the fifth failure it becomes FAILED and stays in the table, where the Health page lists it with its last error and a button to replay it. A worker that dies mid-turn leaves its claim behind, and the next sweep releases any claim older than five minutes.
A worker that is slow rather than dead is the one case where two workers end up running the same turn. Its claim passes five minutes, the sweep releases it, and a second worker claims the turn while the first is still working. The obvious rule, that only the current claim holder may send, would refuse the first worker's finished answer. If the second worker were also slow, it would be refused in turn, and nobody would ever reply. So the rule is that whichever worker finishes first sends:
The cost is that the first finisher's answer may be a little older than the second's, which is better than no answer at all.
Moving turns into chat_history made five components unnecessary, and we deleted them:
If you're building something similar
- Keep queue messages small. Save the request to your database first, then publish only its ID. Workers read the latest data when they start, not a copy from when the message was sent.
- Claim before you work. Mark a job as taken before starting it, and let the first worker to finish send the result.
- Put the ordering rule in the claim query. "Skip any sender with a turn in progress" is one condition in SQL, and it's much harder to get right anywhere else.
For RidgeText, the result is that a slow image request no longer holds up anyone else's question, and a burst of texts from one person gets replies that follow the conversation instead of repeating each other.