A manifesto
Stop the
Judge Tax.
Agent observability platforms should observe agents. Not bill you for the privilege.
Read on ↓
There is a tax on knowing whether your agent works.
It goes like this: you ship an agent. You want to know when it fails. So your observability platform tells you to run an LLM judge over your traffic: a frontier model, reasoning from scratch, over every trace.
Then it hands you the bill. Braintrust and LangChain make you pay for the judges out of your own pocket, on top of what you already pay them. The monitoring costs more than the monitored.
That is the judge tax.
So you sample.
Because judges are expensive, you judge a sliver of traffic. One percent. Half a percent. Whatever the invoice allows.
But agent failures are rare by nature. A behavioral anomaly is often less than 1% of traffic. The agent that says it transferred to a human but never called the tool; the web search that quietly returned stale results.
Sampling doesn’t reduce the cost of finding rare failures. It hides them.
And failures are no longer typos in a chatbot reply. Agents move money, write to production, talk to your customers, and act for days without supervision. The failures got more catastrophic precisely as the monitoring got more sampled.
What if there was a better way?
What if you could just describe a behavior, in plain English, and get back a custom-trained classifier? As accurate as GPT-5.6 Sol, but 1600x cheaper.
That is what we built.
cheaper than GPT-5.6 Sol xhigh per classified trace, at comparable accuracy. That is rd-signal-2, the model behind Raindrop Signals, already classifying 20 billion traces a month, with a median trace classified in 100 ms.
At that price, classification stops being a line item. Raindrop includes Signals with the platform: every trace, every customer, no judge bill.
We have been here before.
Our team invented fraud transformer models at Robinhood. We built the first recommender system at Pinterest. We deployed agents inside the federal government.
Every time, the lesson was the same: the systems that matter are the ones that watch everything, cheaply enough to never look away.
What we believe
- I.Observability that samples is not observability.
- II.The platform should pay for the watching, not the watched.
- III.Reason at build time. Execute deterministically. Judge only the ambiguous.
- IV.Rare failures are the ones that matter. Look at all the traffic.
- V.Agent observability platforms should observe agents.
Try the calculator
How much are you paying to know whether your agents work?
Stop paying for judges.
Start using Signals.