In our last blog, Inside PAXAFE’s AI Agent Harness: Building Agentic AI for Supply Chains we walked through the harness behind Ask PAXAFE: a model, a loop, a handful of read only tools, and a layer of skills on top so users end up asking the questions worth asking. That whole system has a person on the other end. Someone types a question and the agent answers it.
This post is about what happens when you take the person away. We took the same engine and pointed it at every active shipment on the platform. Internally we call it the holistic risk agent. It looks at where the shipment is, what the devices inside it are reporting, and what's happening along its route, and gives a verdict: on track, at risk, off track, or not available. Each verdict comes with a reason and a recommended action, split into two parts, one for the delivery and one for the product inside the box. A shipment can show up on time with product that's out of spec, or late with product that's perfectly fine, so one label isn't enough.
Most agents people use today are reactive. You ask, they answer. An always on agent flips that around. It runs in the background against your data and decides by itself what deserves attention.
That moves the hard problem somewhere else. With chat, the hard part is what to say, because the question is handed to you. With an always on agent, the risk judgement is the easier half, since we already had the tools and the model from the chat side. The hard part is deciding when to look. Look too often and you're paying for answers you already have. Look too rarely and you miss the shipment that's going wrong. Since we don't have unlimited token buedgets we had to be cognizant of the cost as well which lead us to a develop a really efficient system.
The first obvious answer is to check everything on a timer, say every fifteen minutes. It's simple, but most shipments don't do anything in a fifteen minute window, so nearly every check comes back with the same answer as last time. You end up paying an expensive model thousands of times a day to re-read a story that hasn't changed.
The second is to check a shipment whenever something happens to it: a carrier scan comes in, a sensor reports a temperature excursion, a risk level goes up. This is where most teams land and it's where we started. It's cheap and responsive, but in cold chain it goes blind at exactly the wrong time.
There's been a number of shipments that showed us why triggers only wouldn't work. Picked up on a Monday, one carrier update on Tuesday, then nothing. Six days later it was three days past its planned arrival with no new ETA and no delivery scan. The agent never looked at it, because nothing happened, so nothing told it to look.
In cold chain, a quiet shipment is often the one in trouble. So the rule we started with is simple and a hybrid of the two: react when there's news, and keep checking in when there isn't.
The system has three simple parts.
The first is a set of lightweight checks that constantly scan the platform for reasons to look at a shipment. Some watch for news, like a new carrier update or a change in temperature risk. One watches for silence, flagging shipments that have gone quiet for too long or are past their planned arrival without being delivered. These checks are cheap and deterministic with no LLMs in the loop, so we can afford to be generous with them. If one flags a shipment that turns out to be fine, the cost is close to nothing.
The second is a to-do list. Every flagged shipment goes on it, and it's the only way the agent gets work. Urgent items, like an active temperature excursion, always go ahead of routine check-ins, so a pile of quiet shipments never delays one that's actively going wrong.
The third works through that list a few shipments at a time. How many it takes at once is the single dial we turn to control how much of the platform the agent uses.
Asking the model is the expensive part, so we need to be sure that something's gone wrong on a shipment before asking the model.
Before every look, the agent takes a quick snapshot of the things that actually matter on the shipment and compares it to the snapshot from the last verdict. If nothing meaningful has changed, the last verdict still stands and the model is never called. Most of the time that's what happens, because most shipments are exactly where they were a few minutes ago.
The tricky part is deciding what counts as meaningful. Make it too sensitive and tiny changes, like a sensor reading moving a fraction of a degree, trigger a full look every time. Make it too loose and verdicts go stale without anyone noticing. We probably spent the most of amount of time getting this part right.
There's one deliberate exception. For quiet shipments, we skip the comparison once a day and ask the model anyway. On a shipment where nothing is happening the snapshot never changes, so without this the agent would never look again. But silence two days before delivery means something very different from silence the morning after pickup, and the model should get to weigh that.
The result is that we pay for the model only when something on a shipment has actually changed, plus one check a day on the quiet ones to make sure quiet still means fine. That's far less than checking everything on a timer, and unlike the event driven version, nothing slips through just because it went silent.
We did make two trade-offs. Occasionally the agent looks at the same shipment twice, for example if something fails halfway through a check. We're fine with that, because looking twice is harmless and not looking at all is not. The agent can also take a few seconds to notice a change, because it checks for news on a schedule instead of the instant data arrives. However, we're now spending considerable engineering efforts
An always on agent is mostly a question of when to look. React to news, keep checking in on silence, and skip the model whenever nothing has changed. That gets you an agent that watches every shipment continuously, catches the ones that go quiet, and costs a fraction of checking everything all the time.