An agent plugin can be valid, installed, enabled, and still absent from the work that matters.
I have encountered this pattern in several forms. A hook script passes when called directly but never reaches a normal session. Two plugin caches contain the same number of skills but not the same content. A project summary exists, yet the receiving agent recovers an old branch or the wrong next action. A connector is configured but no longer authenticated.
Each system looks green because each check answers a narrower question than the claim being made.
The remedy is not one larger health check. It is a chain of evidence with five separate planes:
- source;
- installed state;
- live behavior;
- continuity;
- authenticated capability.
This is the conformance model I now use in synthesis engineering. It is useful well beyond Synthesis because it turns a vague compatibility claim into a set of observations another person or runtime can reproduce.
Plane 1: Is the source internally valid?
The source plane answers questions that version control can answer well.
- Does the manifest parse?
- Do declared dependencies exist?
- Are generated adapters derived from one canonical source?
- Do tests cover failure paths?
- Does the plugin version agree across its native packages?
- Does the skill catalog fit its declared metadata contract?
This plane matters. It is also the easiest place to create false confidence.
A direct unit test can prove that a SessionStart script emits the right JSON when fed a fixture. It cannot prove that Codex or Claude Code discovered the hook, trusted it, invoked it, or placed the output in model-visible context.
Source validity means the artifact is eligible to work. Nothing more.
Plane 2: What is actually installed?
The installed plane compares the intended release with the files and registries the client currently uses.
For a plugin, this includes more than checking whether a directory exists. The check should establish:
- the active version, not merely the newest directory name;
- the marketplace or source that supplied it;
- content parity, not file counts;
- enabled state;
- generated instruction integrity;
- local drift from source; and
- whether obsolete duplicate copies can still win discovery.
The distinction between source and installed state becomes especially important with versioned caches. A source checkout may be at one version while an active session still points to another. Updating the cache can also remove a path that a live session retained when it started.
Installation and update are therefore different operations. Installation can be safely repeatable. An update that changes active runtime paths should be an explicit session boundary: checkpoint work, close the affected sessions, update through the native client, and start fresh sessions.
Plane 3: Did a normal session receive the behavior?
The live plane is where many compatibility claims fail.
OpenAI’s Codex hook documentation describes multiple lifecycle events, concurrent matching hooks, and a trust step for non-managed commands. Claude Code has its own hook contract. A direct script invocation does not exercise either client contract.
A useful live receipt needs provenance. At minimum, it should include:
- the client;
- the event;
- the installed plugin version and root;
- a genuine session identifier;
- a timestamp with a bounded freshness window; and
- evidence that binds the receipt to the client’s own session record.
Without the last condition, any process that knows the receipt schema can write a convincing file. A static test could impersonate the live event it is supposed to verify.
The live plane should also separate definition, trust, and delivery. A hook may be defined but untrusted. It may be trusted but disabled. It may be enabled yet not invoked on a particular product surface. Those are different failures with different remedies.
Plane 4: Can another session continue the work?
Live delivery proves that a client received a behavior. It does not prove that work survives a handoff.
The continuity plane tests the durable record:
- current phase and status;
- stable reference facts;
- controlling plan;
- recent session history;
- exact next actions;
- repository, branch, and upstream identity;
- active writer and claim ownership; and
- freshness against git and the systems being described.
The strongest handoff test starts a fresh session in another client and asks it to reconstruct the project without access to the first conversation. The receiving agent should find the same phase, plan, decisions, and next action. It should also detect when the durable files disagree with git or live service state.
This is why chat export is not enough. A transcript records what was said. A project record identifies what remains true, what changed, what is owned, and what should happen next.
Continuity also has a concurrency dimension. If two agents can write at the same time, the record needs claim-before-write ownership, isolated worktrees, heartbeats, and a cross-machine lease that makes one atomic decision. File synchronization alone is not a distributed lock.
Plane 5: Can the client perform the required action now?
The capability plane asks the question that configuration screens often blur: is the capability available and authenticated in this session?
A connector can be present but signed out. A browser tool can be supported but lack operating-system permission. A repository client can be installed but have no access to the target organization. A feature may exist in the desktop app and not in an IDE extension.
Those states should not collapse into one boolean.
I use three outcomes in addition to pass and fail:
- UNKNOWN: the system could not establish the truth. The network may be unreachable, the client binary may be unavailable, or the authoritative API may have returned an unreadable response.
- UNSUPPORTED: the product surface intentionally does not provide the capability under test.
- WARN: the observation is real but not required for the declared support contract.
UNKNOWN must not become green. UNSUPPORTED should not be disguised as a
broken imitation of another client’s feature.
Outcome parity, not identical machinery
Claude Code and Codex should not be forced into identical file layouts or hook implementations. Their native systems are part of their value.
The compatibility contract should describe user outcomes:
- the method is discoverable;
- the relevant safety decision is enforced or its boundary is stated;
- project state survives the handoff;
- a live session receives current context; and
- evidence names the product surface that produced it.
One client may use a plugin marketplace while another uses a different registry. One hook may return a client-specific envelope. One runtime may mark a feature unsupported. That is acceptable when the evidence is explicit and the required outcome remains true.
The least capable common subset is a poor target. Native adapters with shared outcome tests are a better one.
A release gate that can fail honestly
A practical release gate can be short:
- Validate canonical source and dependency structure.
- Install only from the reviewed, merged source.
- Compare active client registries and installed content with that source.
- Review executable hook hashes through the client’s trust interface.
- Start fresh sessions in every required client and record genuine receipts.
- Recover one project in both directions from durable files alone.
- Probe required connectors and permissions through authoritative read-only interfaces.
- Publish one structured report that preserves every plane and exact outcome.
The order matters. A live receipt from an old plugin is not evidence for a new release. A fresh plugin installation does not validate an old session. A successful source test cannot override an untrusted hook.
This model changes the meaning of green. Green no longer means that one check found nothing wrong. It means the required evidence chain is complete.
That is the standard agent infrastructure needs if people are going to trust work that moves across sessions, computers, models, and competing clients.