Agent Accountability Starts With "On Whose Authority"

Part 1 of three on agent accountability. This one is about whether the record exists at all.

OpenAI rang Hugging Face on 20 July to suggest to revoke some credentials that showed up in an incident response. Hugging Face said they were already revoked. They’d been revoked because they were the ones used in Hugging Face’s own breach.

Neither side knew yet that they were looking at the same incident! That phone call is how they found out commonality between two incidents. OpenAI told that part at Black Hat a couple of weeks later.

You’ve read enough about that breach. What I want to know is what happens when it’s our turn in such situation!

Pick a tool call your agents made few weeks ago. Some operation that mattered: a payment, a merge, a message to a customer, a schema change. Answer four questions.

Which agent made the change. On whose authority. What was it allowed to do at that moment (consent/scope). What did it actually send, and what came back.

If you run agents in production you probably suspect you can’t answer all four, and you’ve filed that under instrumentation debt. Something a better vendor fixes, or maybe a spare sprint. I thought so too. Then I went and read the specs, and two of those four have no field anywhere in the conventions. A third has one, but it names what was deployed rather than who was acting. The fourth exists and ships switched off.

The complete standard record of an agent taking an action

Agent observability converged on OpenTelemetry. Most frameworks emit it, every vendor ingests it. The span that records an agent using a tool is execute_tool, and it carries nine attributes.

Two are Required: gen_ai.operation.name, whose value is the literal string execute_tool, and gen_ai.tool.name.

Two more are Conditionally Required, which in practice means they show up if something errored or if the framework happened to know the agent’s name. Three are Recommended. The last two are gen_ai.tool.call.arguments and gen_ai.tool.call.result, and both are Opt-In.

Opt-In isn’t a soft suggestion in OpenTelemetry. The spec says instrumentations should populate the attribute if and only if the user configures it, and then goes further: “Instrumentation that doesn’t support configuration MUST NOT populate Opt-In attributes.”

So a fully conformant record of an agent doing something in the world can be the word execute_tool plus a tool name. What it sent and what came back are off unless somebody went and turned them on, which for the Python instrumentation means setting OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT.

There’s also a plan span, for task decomposition. Three attributes: operation name, error type, agent name. Nothing that holds the plan. You can sometimes dig it out of the child LLM span underneath, assuming content capture is on, which tells you what the record was built for.

The field we had, then deleted

You’d expect the authority side of this to live in the identity layer and get joined in later. Most likely it doesn’t!

I grepped the whole GenAI attribute registry, both span documents and the MCP conventions for every variant of authorization, principal, delegation, permission, scope, credential, consent, approval, actor, identity. As I write this in August 2026, one hit across all of it, and it’s a sentence of prose about a runner delegating to a sub-agent. Not an attribute.

Nothing records the human the agent was acting for. Core OpenTelemetry has user.id and it appears on exactly zero GenAI spans, not even as an Opt-In. Nothing records a delegation chain, even though RFC 8693 has carried nested delegation in the act claim since 2020, which I went through in an earlier piece on agent identity and delegation, and PROV-O has had actedOnBehalfOf since 2013. Nothing records the authorization decision.

Then I found the one that made me go back and check. Core OpenTelemetry used to have enduser.scope, with the example value read:message, write:files. It’s the only attribute in this ecosystem that ever held an authorization scope. It’s deprecated, and the note says there’s no replacement at this time.

We had the field. It got removed, and the deprecation note is explicit that nothing took its place.

The fixes are all sitting there but in open state. The tracking issue for GenAI user and session conventions went up on 5 February 2025 and is still open. There’s an unmerged PR for authorization outcomes, an unmerged proposal for agent identity that could carry SPIFFE, an open issue asking how to record that a human approved a tool call. An earlier run at this in core semconv, “Semantic conventions for AI agent identity and trust,” got closed without adoption in April.

Here’s the part I wasn’t expecting. In June the GenAI conventions moved out of core semconv into a repo of their own, open-telemetry/semantic-conventions-genai, and that repo has zero releases and zero tags. The README says TODO. All 63 gen_ai.* attributes are marked Development and none is Stable. opentelemetry.io doesn’t host the span docs anymore either, it just points at the new repo.

Which means the authoritative source for how our whole industry records agent behavior has not done a release, and there’s no version string you can stamp on a trace to say which convention produced it. Read a trace back in two years and you’re guessing!

What the wire format can’t say

Two problems sit below the semantics, and neither is fixable by a vendor so far.

The OTLP Span message has sixteen fields. None of them is a signature, digest, hash, or chain link. There is not a field to put integrity, so you can’t add it without leaving the format/customize it.

The second one is subtler and I like it more. A span carries dropped_attributes_count, dropped_events_count and dropped_links_count, so it can tell you it lost bits of itself. Nothing inside a span can tell you a sibling span went missing. Default export queue is 2048 (OTEL_BSP_MAX_QUEUE_SIZE), and the spec is clear that spans get dropped once it fills. There’s a rejected_spans counter, but it lives in the transport response back to the exporter and never travels with the stored data. Forty days later the reader sees a clean trace with a hole in it and no way to know.

You can’t tell “the agent didn’t do that” apart from “we didn’t write it down.” Those are the two answers an investigation cares about most.

Crossing an org boundary has the same shape of problem. W3C Baggage is the only way to carry application context between services, and its Information Exposure section talks about either keep confidential data out of baggage or keep baggage out of requests that cross trust boundaries. It’s an either/or rather than a ban, but one branch is exactly the thing you’d want to do. It’s also still a Candidate Recommendation Snapshot. A2A doesn’t help: its Task object is id, contextId, status, artifacts, history, metadata, and authorization stays in transport headers where it’s never persisted.

Meanwhile MCP deprecated its own logging utility in the 2026-07-28 revision and pointed everyone at OpenTelemetry. What it dropped was a severity level, a logger name and some arbitrary JSON. No actor, no authority. Losing it cost nothing as such.

What the vendors built on top of it

I went through the docs of the major agent observability platforms, and the source where they publish it. Same pattern everywhere: they all ship a delete API, and none of them ships tamper-evidence.

Datadog, now offering this as Agent Observability, has a preview API that deletes LLM span data by trace ID. It takes an optional parameter documented as “Optional delay in seconds before the deletion is executed.” The docs don’t say which RBAC permission gates it. No audit event is documented for it either. Max retention on a production trace is 90 days, default 15, and unlike logs there’s no archive destination you can point spans at, only a paginated export API you’d run yourself. Their own archive docs, on the Azure tab, tell you “Do not set immutability policies because the last data needs to be rewritten in some rare cases (typically a timeout).” The Google Cloud tab says the same thing in different words. The S3 tab says nothing as such.

Langfuse is the sharpest one. Its older v3 ingestion path merges events into an existing record, and the fields a trace protects from overwrite are listed in one place in the source: id, project_id, timestamp, created_at, environment. Take a closer look at what’s absent. input and output don’t just miss that list, they’re resolved separately, taking the latest non-null value by event timestamp. So an event carrying an existing trace id and a fresh output replaces the stored model output, last write wins, and no audit entry appears, because the audit log’s trace actions are bookmark, delete and publish.

Now being fair to them: that’s documented merge semantics, not a vulnerability. v4 rejects these writes at the edge by default and nobody has filed an advisory. But the path stays live on v3, on v4 in legacy or dual mode, and on un-migrated Langfuse Cloud until 16 November 2026. Self-hosted has no deadline. Scores stay over writable even on v4. And v4’s docs say duplicate ids give “no guarantee of consistency (not even eventual consistency)”, which for forensics is worse than overwriting, because now the record is ambiguous and the vendor won’t say which version you’ll read.

Weave does something I hadn’t seen before. When a row is too big for the storage backend the insert throws, and the retry path doesn’t truncate anything. It walks the four JSON columns largest first, swapping each one wholesale for a marker, {"_weave": {"error":"<EXCEEDS_LIMITS>"}}, until the row fits under 3.5 MiB. Often that’s more than one column: their own tests assert the inputs and the output both getting replaced in a single call. So the biggest prompt in the trace, usually the interesting one, is first to go. None of this appears anywhere in W&B’s documentation. You find it by reading the server source. Their server-side sampler is documented though, and it’s honest about the consequence: “A dropped trace receives a normal success response. The client isn’t notified that its trace was dropped.”

All of these platforms also offer an admin audit log with integrity language attached, and every one of them scopes it to console actions. W&B’s captures 29 action types and not one of them is a Weave trace operation. Datadog’s does carry an LLM Observability category, but what lands in it is configuration. Nothing there records a read, an export, or a deletion of the trace data itself. The control that looks like tamper-evidence stops right where the agent record starts.

The honest line in the whole sweep comes from a post by an Arize engineer, under a heading listing what their own product doesn’t do: what they can’t give you yet is tamper-proof records. It carries a disclaimer saying it isn’t the company’s position, which is probably why it’s the straightest sentence any vendor has published on this. It’s also true of the whole category, not just them.

Follow the meter

None of this is incompetence, and I want to be careful here, because the easy version of this argument is unfair. Lets take a closer look at what each platform bills for and the defaults stop being mysterious.

Weave meters ingested bytes and defines them to include LLM inputs and outputs, at a megabyte on overage. The meter is the prompt and the completion, so your bill scales linearly with how complete your evidence is. Langfuse bills per record and counts every trace, observation and score, so deeper instrumentation costs strictly more and running evals against your own logs is itself billable. Datadog’s pricing page is blunt about it: they bill “only on LLM spans, not on tool spans, retrieval spans, or other workflow spans surrounding the model call.” The tool call, the thing that actually touched the world, isn’t the unit of record.

Then read their advice on managing spend. Langfuse: “You can cut costs quickly by sampling fewer traces or logging only essential data,” all while preserving your core insights. W&B: “your costs scale roughly with the share of traffic that you keep.”

Every documented lever for spending less removes evidence, which is ironical! The vendors are pricing a debugging product honestly, and debugging is what they built. The trouble is that the debugging product is the only thing on the market, and we’ve quietly started using it for something else.

Some arithmetic, and purely to engage in understanding the math, numbers/price/amount may vary, with the assumptions on the table because they matter. Say 100,000 tool calls a day, one model call behind each, and about 40 KB captured per call once you count the prompt, the completion, the arguments and the result. That’s three million billable LLM spans and roughly 120 GB a month. Replay the whole accumulated context on every call instead of the delta and the storage roughly doubles or triples, though the span bill doesn’t move. For example, Datadog logs at thirty-day retention: taking an example (and actual numbers may be different), about $19.50 a month at annual list, being $12 of ingest on the 120 GB plus $7.50 to index three million events. Per-span LLM pricing at 90-day retention: $2,375 a month at US annual list, and just over half of that is the retention surcharge rather than the spans. Two model calls per tool call and the same workload is $4,625. Actual estimates may be different but you get the idea.

That same 120 GB in Glacier Deep Archive is about twelve cents of storage. Batch it into five-megabyte objects and pay the lifecycle transition requests and you’re near $1.32 all in, where the storage is the rounding error. Write one object per span instead and the requests alone cost you $150.

Same bytes, same month, two bills: $2,375 against $1.32. And the two aren’t buying the same thing. The $2,375 buys you ninety days, after which the record is gone. The $1.32 buys you keeping it. Deep Archive does grow as the archive does, reaching about ten dollars a month once seven years have piled up, which is still three orders of magnitude under the alternative. The tamper-evidence primitive everyone treats as exotic is nearly free: AWS has said since 2019 that Object Lock carries no additional charge, though it needs versioning switched on and you pay for every version you keep.

The industry priced query latency and gave away immutability. Retention got capped as a side effect, and the caps are the part money won’t move. Fifteen days. Ninety. Four hundred. Fifteen months. Not one of the LLM observability vendors sells seven years at any price. The hyperscalers happily do, which is what makes those ceilings look like a product decision rather than a cost one. Google Cloud Logging prices retention out to 3,650 days, CloudWatch lists 2,557 days as a setting, and New Relic names seven years outright.

The gap, as a field list

This is the part worth stealing. If you’re designing an agent execution record, arguing with a vendor, or writing a standards proposal, these are the fields a cross-organizational reconstruction needs that no released standard requires.

1. Authenticated agent identity, separate from the deployed artifact.
2. The human principal the agent was acting for.
3. The delegation chain, hop by hop.
4. What was granted at each hop: scopes, audience, expiry.
5. The permission state at the moment of action, what it could have done rather than only what it did.
6. The authorization decision and why: permit or deny, which policy, denied by what.
7. Human approval events: approved, rejected or automatic, by whom, from where.
8. Tool arguments and results. These exist, and they’re Opt-In.
9. The plan or reasoning behind the action.
10. Record integrity: a hash, a signature, a transparency-log anchor.
11. Chain linkage to the previous record.
12. Evidence provenance: self-reported or externally attested.
13. A completeness marker, so a gap reads as a gap.
14. A convention version binding, so somebody can read this back later.
15. Carriage of all of the above across an org boundary.
16. Retention that outlasts the discovery of the incident.

Every part is already on the shelf

What gets me is how little of this needs inventing, really!

SCITT landed as RFC 9943 in June and it’s the integrity substrate items 10 through 12 are asking for: signed statements, receipts, append-only transparency logs, non-equivocation. It never mentions AI or machine learning once. “Agent” shows up a single time, in a threat diagram, meaning a build agent.

PROV-O has had prov:Delegation since 2013, defined as the assignment of authority and responsibility to an agent while the delegating agent keeps some of it. Thirteen years old, and nobody has pointed it at this.

OCSF 1.9.0 shipped on 3 August with an ai_operation profile carrying ai_agent and delegation objects across 41 event classes, which proves these fields are specifiable by people who do this for a living. The profile is optional. OCSF’s own issue tracker points out that their delegation object records that authority moved but not what was granted at each step, so no scopes, audience or expiry.

The Cloud Security Alliance has the strongest language anywhere: a MUST on logging every significant agent action against a persistent identifier, plus seven elements they say implementations should capture, one of them a request context hash. It’s an IAM paper with no certification weight behind it.

Even the vendors show it’s buildable. Claude Code emits a tool decision event carrying accept or reject and a source field that distinguishes config, hook, user_permanent, user_temporary, user_abort and user_reject. That’s item 7 on the list, shipped and working, under a proprietary attribute name that interoperates with nothing.

The guidance layer is somewhere else entirely. The operative US federal standard for log management is NIST SP 800-92, published in 2006. Its revision has been in draft for two years and ten months, and the only mention of AI in that draft is a question to reviewers asking whether the topic should be covered. NIST 800-53’s base AU-3 control specifies audit record content in five parts: what type of event, when, where, source, outcome, and the identity of individuals or objects involved. No “on whose authority.” No “with what permission.” The OMB memo from May that rewrote federal logging policy mentions AI twice, never as something that acts.

Where I’d start

The drill is worth running before anything else. Pick a tool call from forty days back, put an hour on the clock, and see how far you get on the four questions. Do it as a team exercise, same as testing a backup restore. Most teams die inside ten minutes, either because the trace aged out of a fifteen-day window or because the agent that made the call can’t be told apart from any other. That failure is the point. It turns a vague worry into a dated artifact you can walk into a budget conversation with.

Splitting hot from durable takes an afternoon. The vendor stays as it is, for debugging, at whatever retention you’re already paying for. A second copy of the raw spans goes into object storage with Object Lock in compliance mode. Storage runs about a hundredth of the price and the immutability costs nothing. That one change takes you from evidence that expires in fifteen days to evidence that exists, with no standard and no vendor cooperation required. Check your bucket layout first, because Object Lock won’t work on a server-access-log destination bucket.

The fields nobody standardized are still yours to write. Until items 1 through 7 exist, they can live in your own namespace, as long as you stay consistent. An attribute you control beats a standard attribute that doesn’t exist. The cheap moment to capture the principal, the delegation chain and the authorization decision is where your policy engine already knows them, and there isn’t a second one.

The Opt-Ins are worth turning on with your eyes open. Arguments and results are the difference between a record and a rumour, and they’re also the line item. Sampling is fine if the bill demands it, so long as it’s explicit and the rate goes in the trace, so a gap reads as a decision instead of as absence.

An agent needs an identity your authorization system can revoke, not a display name. That’s the difference between an incident where you cut off one agent and an incident where your only lever is an org-wide switch.

What the specs should do

Three changes, none of which invents anything.

Make the authority fields exist. Agent identity, principal, delegation chain, and the scope granted at each hop. The proposals are already open, one of them for eighteen months. enduser.scope existed and got deprecated with nothing behind it, so un-deprecating it would be a start.

Promote arguments and results from Opt-In to Recommended for state-changing tools. A read isn’t a write. A convention that files a database query and a wire transfer under the same event category isn’t describing the world I work in. If a tool declares itself mutating, its inputs and outputs belong in the default record.

Give the format somewhere to put integrity, and a way to mark a gap. A digest field and a link to a transparency-log receipt would let SCITT do the job it was built for. And a dropped-span marker would let a reader tell silence from absence, which is the cheapest fix on this whole list.

We’re not missing the technology. We’re missing about nine fields, a version tag, and a default.

Part 2 asks the next question. Say you do work out who did it. Can you take the authority back?