A quiet source is an outage

Every failure that announces itself is the easy kind. A connection refused, a malformed payload, a schema that changed overnight - all of these produce something to alert on, and any competent operations practice catches them within a minute. The failure that survives is the one that looks like success: a source that answers promptly, returns a well-formed and completely empty response, and does so consistently for eleven hours. Nothing errors. The pipeline runs. The dashboards are green. The data simply stopped.

This is the specific reason data observability is a different discipline from infrastructure monitoring. Infrastructure monitoring answers whether the job ran. It cannot answer whether what the job produced still describes the world, and those two questions diverge exactly when a source changes something on its side without telling anyone. A rate limit tightens, an endpoint starts requiring a parameter it used to default, a region gets geofenced. The response code stays 200 for every one of them.

Detecting an absence is harder than detecting an error, and the reason is statistical rather than technical. An error is a discrete event you can alert on the first time. Silence has to be judged against what silence normally looks like for that source, at that hour, in that region - and normal is wildly uneven. A filings source is legitimately quiet at three in the morning local time, quieter still on a public holiday nobody encoded, and busy for ninety seconds after a scheduled release. Alert on any gap and the on-call rota is training itself to ignore you within a week, which is how a data quality practice dies without anyone deciding to end it. Alert too loosely and you find out from a client, which is the outcome the whole exercise exists to prevent.

What works is unglamorous: a per-source expectation of arrival rate that carries its own seasonality, an alert on a departure from that expectation rather than on a fixed threshold, and a bias toward suspicion when a source goes quiet during a period when its region is demonstrably busy - the same correlated evidence our models use when judging authenticity. The last one matters most, because correlated evidence is what separates a real gap from a slow Tuesday. If nine sources in a region are producing and the tenth has said nothing for an hour, the tenth is not calm.

The structural answer is redundancy, not vigilance. Coverage in a region is built so that no single node and no single source is the only path to an event, which converts a silent failure from a hole in the dataset into a degradation in how many independent observations corroborate a record. That is a different quality of failure: the record still arrives, the provenance on it shows which path it took and what corroborated it, and a research desk can see for itself that this particular event rests on thinner evidence than usual. Failure becomes visible in the data rather than in an incident report published afterwards.

This is also why we treat freshness as something to publish rather than something to assume. Every record carries the event time of the node that heard it, so a client can compute how current their own subscription is at any moment, per subject area, without asking us. A vendor's status page is a claim made by the party with the least incentive to update it quickly. A timestamp on the record you already hold is a measurement, and if a region is degrading, you see it in your own numbers before anybody writes a sentence about it.

There is a cultural half to this that no monitoring configuration delivers. A quiet source has to be treated as a defect with an owner, not as a lull to be revisited if it persists, because the cost of being wrong is asymmetric: a false alarm costs an engineer twenty minutes, and a missed silence costs a client the day their coverage was thinnest without knowing it. Teams drift toward the comfortable error because the expensive one is invisible. Writing the rule down - silence is a bug until proven otherwise - is what keeps the drift from happening quietly, and it is why the rule lives in the observability spec rather than in somebody's habits.

None of this makes a listening network immune to the world. Sources change, regions go dark, and some gaps are real absences of activity rather than failures to observe it. What is achievable is that we know which one we are looking at, quickly, and that the answer reaches the client in the record's own provenance rather than in a postmortem. That is the honest version of reliability for a network whose job is to notice things.