The One-Line API Call That Hid 27 Processes
Whoever wrote the docs for this payment gateway did a tidy job. One endpoint. One parameter. One action: you send the card number and call charge. That's what the page said.
What the page didn't say was that behind charge there's antifraud scoring, 3D Secure, issuer verification, fund holds, audit logging, retries. A distributed saga. Twenty-seven processes, if you count them. Nobody counted them, because nobody knew they were there.
Then transactions started hanging. Seven minutes each. The integration team had nothing to work with. No SLA they could forward to their own customers. No timeout values. No retry schedule. The product was fine. The docs were the thing that broke. Clients left anyway—not because the gateway failed, but because nobody could tell them what was happening while they waited. If your help authoring tool only shows you the interface, you're blind to all of this.
This isn't a weird edge case, by the way. It's what happens by default with anything invisible. Background automation. AI classification. Webhooks. Async pipelines. The user gets a green checkmark and moves on. The system is sitting in some state you never documented. And the docs, if someone wrote them, describe the button.
Same story at another company, just slower. Their docs were still describing screens and input fields. The backend had already moved to event-driven architecture, AI sorting, background sync. Tickets piled up about things that should have matched but didn't, and nobody on the support side could say why. Six months later the team threw the whole thing out and started over. The new docs were built around triggers, states, and transitions. Support load dropped 40%.
Understanding Background Processes
Background processes are everything that happens between the click and the result. Some people call them quiet features. Autosave. Search reindexing. Antifraud scoring. AI ranking. Webhook delivery. Data replication. None of them have a UI, which is why technical writers skip them. Even people using fairly advanced help authoring tools skip them.
Treating them as implementation details is the wrong move. On a payment gateway, the antifraud scoring is what the customer is actually paying for. On a cloud editor, autosave is the whole reason the thing is usable. And these systems fail quietly. The button still lights up. The spinner still spins. What breaks happens somewhere else, and the user finds out hours later, when the search results are stale or the file is gone.
Standard user docs are built around a different question entirely—how do I click this, where does the field go. Background processes don't fit that mold. They raise other questions. What's happening while I wait. Why did the result differ from what I expected. What do I do when nothing seems to happen.
Why Invisible Mechanics Need a Different Doc Strategy
I spent a few months on this problem at a company that shall remain nameless. The product was a scheduling tool, nothing exotic. You picked a time, you hit confirm, and somewhere in the background a chain of services did the actual booking. Eight services, as it turned out. Nobody had written any of them down.
The first thing I learned is that the user doesn't care about your architecture. They care that they clicked something and nothing happened. One of our users clicked confirm eleven times over four minutes. Eleven bookings. Same slot. The support ticket that came in was two words long and I still think about it.
So the docs had to change. What we ended up doing was writing down, for each background process, four things. What sets it off. Who or what does the work. What it depends on. And what comes out the other end, including the ugly cases. That last one mattered most. Users don't file tickets about the happy path.
Turns out you don't get much from asking engineers what a service does. You get the architecture diagram. What you want to know is what happens when someone closes the tab halfway through.
One more thing, and then I'll stop going on about the scheduling tool. Name things by what they do, not by what they're called in the repo. Our booking pipeline had a service called worker_3 for two years. It was the fraud check. Nobody outside the backend team knew that. Once we renamed it in the docs to "fraud filter," support tickets about it dropped by half, because now the first-line team knew who to route them to.
A Practical Guide to Documenting What You Can't See
The rest of this is what I wish someone had handed me on day one. It's not a framework. It's just the order I'd do things in if I had to start over.
Finding the pipelines
The first snag is obvious once you say it out loud: nobody documents a pipeline they've never heard of. So the job starts with finding the pipelines. Background work sits in cron tasks, in message queues, in event handlers. RabbitMQ and SQS are common homes for it, and Celery too. Once the obvious places are covered, the repos are next. Go through them by hand. Anything called workers, jobs, background, async, or scheduler is worth opening. Infrastructure has its own hiding spots—Kubernetes Jobs, Lambda functions, CI pipelines. Logs come last. Grep for "background," "queue," "retry," "dead letter" and something usually turns up.
Repos and logs only get you so far. The rest comes from people. Sit down with an engineer and ask what happens once the user clicks. Does closing the tab kill the job, or does it keep running? What does the 3 AM cron actually do, and who gets paged when it falls over?
Then write down four things for every process you've got. What sets it off? Could be a user action, could be a timer, could be something external entirely. Who actually does the work—that's usually a service or a worker, though sometimes it's a model. What does it lean on that lives outside the system? Another service, a dataset, an API somewhere. And what comes back out. The user-facing result, the downstream payload, and the shape of an error when one shows up.
Those four are the backbone of any user manual worth reading when backend logic is involved. Name things by behavior. Not worker_3. "Fraud filter."
Drawing the flow
Prose and async logic don't mix. I tried writing out a five-service booking flow in a paragraph once. Three hundred words, and it just sat there. Nobody read it. They skimmed, nodded, and went straight back to the backend team.
Diagrams did the job instead. API and webhook stuff gets a sequence diagram. Long-running jobs get a state diagram, because "pending" doesn't tell you anything until you see what comes before and after. Decision trees get a flowchart. If the model checks language first, then length, then keywords, sketch that out. Put it in prose and the branches just disappear.
Mermaid or PlantUML keeps them in sync with the docs. But a diagram alone isn't documentation. The picture shows the shape. The words around it carry the edge cases, the timeouts, the stuff that goes wrong. Skip that part and you've got a nice picture that answers nothing.
Who's actually reading
One background feature needs three descriptions. Put them all on one page and it's unreadable. Spread them across ten pages and nobody finds the timeout value.
Start with the end user, because they usually want to know what kicked off, how long it's going to take, and what to do if it just sits there. So keep it plain. "We're verifying your payment. This usually takes up to two minutes. If it's been longer than five, refresh the page." That's it.
The integrator gets a different page. They want endpoints, response codes, timeouts, retry behavior, all of it. "Webhook delivery times out at 30s. On HTTP 5xx, we retry three times at 5s, 25s, and 125s intervals." No hand-holding. Just the numbers.
The architect wants to know why any of this happens. "The classification model was trained on 500k tickets. Accuracy is 94.2%. Mixed-language inputs are a known blind spot." That's the stuff that lets them predict what the system will do next.
Start with the user layer. Add integration details when someone asks. Keep architecture in a separate doc or a deep appendix. End users don't need sequence diagrams. Integrators shouldn't have to guess at retry intervals.
Six questions
Every background process doc answers six questions, and they make a decent outline.
Why does this exist? One sentence of user value. Not "Background sync" but "The system refreshes inventory automatically so you see live stock without reloading the page."
What triggers it? User action, system event, or schedule. "Every 15 minutes" is better than "periodically."
What happens? Step by step for linear flows. Diagrams for branching logic. Ten clear bullets beat two vague paragraphs.
What should I expect? Duration, intermediate states, where to check progress. If it takes five minutes, say five minutes.
What can go wrong? Error codes and what they actually mean. "Error 504: the verification server didn't respond within 30s. Wait two minutes, then check status in your dashboard."
What are the limits? Quotas, timeouts, rate limits, AI confidence boundaries. Honesty beats marketing.
Scenarios, not specs
Abstract descriptions don't help much. The reader is thinking "okay, but what does this look like for me?" Scenarios close that gap. Four paths cover most of it.
The happy path is the ideal flow from trigger to completion. Then there's delay: where to check status, when to escalate, whether cancellation is even possible. Hard failure covers what the user sees, where the link leads, who can fix it. And partial success is the messy one—AI returns low confidence, or an import finishes with dropped rows. That last path needs an explanation of how the system surfaces warnings and whether manual cleanup is possible.
Example scenario—AI ticket classification:
Input: Ticket contains "urgent" and "contract."
Expected: High priority, routed to Sales.
Edge case: If the same ticket contains "spam," the model may deprioritize it. The training data correlated "spam" with low urgency, creating a false negative on mixed-intent language.
Three checks before you ship
The draft is done when it passes three filters.
Engineering review means handing it to the developers who wrote the code and asking what's missing. They'll catch hidden retries, implicit dependencies, stale limits. They'll also flag what should stay internal.
Support review means handing it to support and watching. Can they find answers to the top five user questions in under 30 seconds? If not, the information architecture needs work.
User test means finding someone who's never touched the product, giving them a task that depends on the background process and nothing but the docs, and watching where they get stuck. Don't help.
What Teams Actually End Up With
Nobody starts with a clean strategy. You inherit whatever the last writer left behind, and you patch it as you go. Still, there are three shapes you tend to see, and it helps to know which one you're standing in.
Interface-centric docs cover buttons, fields, and notifications. They work until something goes async, and then they don't. There's nothing on the page to describe what happens after the click, because the page was built around the click itself.
Process-centric docs follow the flow from trigger to result. This is where most background documentation lands, and it's a reasonable default. The problem is that support and end users share the same page, and one of them gets lost. Usually the end user.
Event-centric triggers, events, and system reactions are what docs are usually built on. Most people don't think in schemas and delivery guarantees but such event-driven architecture docs think they do. And it is usual situation when links are broken and pages are rot because event schemas are changed.
Stripe's webhook documentation is the example everyone points to. Read it as a user and you learn whether your payment went through. Read it as an integrator and you get the endpoint, the signature scheme, the retry schedule. Read it as an architect and you get delivery semantics. Same pages, three readers, nobody confused about which room they're in. That's not an accident. It's structure.
Most teams end up with a mix. Event-centric for integrators, process-centric for support, interface-centric for end users. What makes the mix work is cross-linking. If you land on the wrong layer, there should be a link to the right one, and it should be obvious which is which.
Costs That Don't Show Up in the Budget
Search is where this falls apart first. Nobody types "async validation pipeline" into a help search. They type "email didn't arrive" or "order stuck." If the docs don't use those words, the search index never surfaces the right page. A page called "Order Status" should point to one called "Background Payment Verification," and the search layer should treat them as related. Headings matter too. Bury a critical process in a paragraph instead of an H3 and it's effectively invisible.
Then there's the analytics trap. Time-on-page and scroll depth measure the wrong thing for reference documentation. Someone who finds a timeout value in twenty seconds and closes the tab succeeded. They didn't bounce. The number that actually tells you something is support ticket volume. If webhook delivery questions drop 30% after the retry policy goes up, the doc is working, regardless of what the dashboards say.
Maintenance is the quiet killer. Background logic moves faster than UI. Retries get added, timeouts shift, models get swapped. A stale doc is worse than no doc, because it builds false confidence. The fix is to keep documentation close to code. Process parameters in config files that feed both the runtime and the docs portal. Structured comments that generate documentation automatically. Where that isn't possible, put a name on each background process section so somebody notices when it drifts.
And the last one is cost, which nobody wants to talk about. Documenting invisible machinery is expensive. Engineer interviews eat time, diagrams rot when architecture shifts, AI accuracy statements expire with every retrain. Be honest about ROI. An internal tool that changes weekly may only need a README and a diagram. A customer-facing pipeline that touches revenue justifies the investment, because the alternative is support overhead and churn.