Four-minute self-assessment

Can you tell whether your AI agents make good decisions?

Fourteen questions in three parts. One workflow and the decisions within it. Find out who owns your agents’ decisions, how your organization expects them to be made, and whether you can see when they differ. No email needed.

Which agent workflow are you thinking about?

Part 1 of 3

Ownership

Who owns the decisions and who can step in

01 Can you name the business decision your agent is making—and who owns its quality? Yes. The decision and accountable owner are explicit. Partly. The workflow is defined, but ownership is shared or unclear. No. We mainly manage the agent as a technical system.

02 If one of the agent’s decisions turned out to be wrong today, could you name who answers for it within the hour? Yes. One named person answers for each decision the agent makes. We would find out, but it would take a meeting. No. It would land on whoever built the agent, or on nobody.

03 Is there someone with the authority to stop or reverse an agent’s decision, and do they know it? Yes. Named people can hold or reverse a decision, and they know it. In principle, but nobody has been told or has tried it. No. Once the agent acts, the decision stands.

04 Do the teams involved in the workflow agree on who decides each step? Yes. It is written down and every team has agreed to it. Mostly, but some steps are claimed by two teams, or by none. We have never put it to them side by side.

05 Is there one live view, shared by every team involved, of who decides what and by which criteria? Yes. One live document, kept current, that every team works from. Partly. It exists in documents or slides, but each team has its own version. No. It lives in people’s heads and in the agent’s configuration.

06 Are thresholds, escalation conditions and intervention owners agreed in advance? Yes. The business owner has approved when to pass, flag or escalate. Partly. People know when to intervene, but rules are not consistently explicit. No. Intervention is largely improvised after a problem appears.

Part 2 of 3

Decision trajectory

How your organization would make this decision

07 Is your experienced people’s judgment captured as an agreed reference, independent of the agent’s instructions? Yes. Required considerations are written down, independent of the agent, and endorsed by the business. Some are documented; important judgment still lives in people’s heads. No. What we have sits in the agent’s prompts and configuration, or in individual expertise.

08 Can your experienced people map how they would approach the same business decision, including what they would consider, what they would question, and when they would involve someone else? Yes. We can map the considerations, judgment points and escalation conditions, and how they depend on each other. Partly. We know the main considerations, but parts of the judgment remain implicit. No. We can describe the workflow, but not the judgment behind each decision.

09 Can your people explain the organizational impact of a decision at each node—including downstream effects, handoffs and what happens if the decision is wrong? Yes. We know the downstream consequences, affected teams and escalation triggers. Partly. We know the direct outcome but not all knock-on effects. No. We do not have a reliable map of the ramifications of each node.

10 Could the decision owner carry the same assurance method to the next agent workflow? Yes. The criteria, record and ownership model are repeatable. With significant help from specialists or the original project team. No. Each new workflow starts from scratch.

Part 3 of 3

Comparison

Whether you can see when an agent’s decision differs

11 Can you reconstruct why the agent reached a particular business decision? Yes. We retain the decision basis and supporting evidence. Partly. We have logs or traces, but the business rationale is incomplete. No. We can see the action, not the basis for the decision.

12 Can you compare an agent’s decision trajectory with the decision criteria your responsible teams have agreed upon? Yes. We compare it with the criteria our responsible teams agreed. Sometimes, through sampling or manual review. No. Our checks focus on permissions, policy, outputs or system behavior.

13 Would you detect a missing business consideration before the same mistake repeats at scale? Yes. We have checks before or at the point of decision. Sometimes. We usually discover patterns in periodic reviews. No. We are more likely to find out after an incident or complaint.

14 Can you measure decision-quality gaps and the cost of reviewing them? Yes. We track reviewed decisions, gaps, false flags and review effort. Partly. We track incidents or quality indicators, but not the full gap. No. We cannot quantify the gap in agent decisions.

Your answers stay in your browser. This is self-reported; the sprint tests it on a real case.