Coverage for gco_mcp / mission / decide.py: 100.00%
87 statements
« prev ^ index » next coverage.py v7.13.5, created at 2026-09-14 22:07 +0000
1"""Pure deterministic verdict cascade for the Mission Decide_Phase.
3The cascade is the **control-path** output of the loop: given the current
4``SessionState`` (before the in-progress iteration is appended), the
5in-progress :class:`IterationRecord` (with ``strategy``, ``observation``,
6and ``criteria_evaluation`` already populated but ``verdict`` /
7``verdict_reason`` not yet set), and the wall-clock value the caller has
8already measured, :func:`decide_verdict` returns a
9``(VerdictLabel, VerdictReason)`` tuple. The function is pure: no logger
10calls, no I/O, no random sources, no clock reads. The wall-clock value is
11passed in on the call signature so tests can pin it.
13The cascade order is fixed:
151. **Budget terminations** — checked in a fixed sub-order so the
16 verdict_reason is deterministic when more than one cap is breached:
18 * ``max_iterations`` — the in-progress iteration would be the
19 ``budget["max_iterations"]``-th or later. Computed as
20 ``len(session["iterations"]) + 1 >= max_iterations``.
21 * ``max_wall_clock`` — ``now - session["started_at"] >= max_wall_clock_seconds``.
22 Returns False when ``started_at`` is missing (the session has
23 not yet transitioned out of ``pending``).
24 * ``no_progress`` — ``no_progress_counter >= stagnation_threshold``.
25 When the session has ``use_sampling=true``, the heuristic (step
26 4) gets priority so the sampler can revise the strategy before
27 the loop terminates. Without sampling, ``no_progress``
28 terminates immediately. If the heuristic doesn't fire (e.g.,
29 the tool sequence changed after a prior revision), the deferred
30 stagnation check (step 4b) terminates.
322. **Completion** — every ``required=True`` Criterion has status
33 ``met`` in the in-progress iteration's ``criteria_evaluation``, AND
34 no Criterion (required or not) has status ``inconclusive``.
363. **Cadence-skip** — when :func:`should_evaluate_now` says "this is
37 not a checkpoint", emit a synthetic ``("continue", "cadence_skip")``
38 without consulting the Strategy_Revision_Heuristic. The heuristic
39 only fires on real checkpoints so off-cadence iterations cannot
40 advance the no-progress counter or trigger an ``adjust``.
424. **Strategy_Revision_Heuristic** — :func:`_strategy_unproductive`:
43 the same ``tool_calls[*].tool_name`` sequence
44 for the last 3 iterations AND ``no_progress_counter`` at or above
45 half the stagnation threshold, OR new errors in the latest
46 Observation that didn't appear in the prior Observation. Returns
47 ``("adjust", "heuristic_unproductive")`` when either clause fires.
494b. **Deferred stagnation** — if step 1c deferred the ``no_progress``
50 check (because sampling is enabled) and the heuristic didn't fire,
51 terminate now.
535. **Default** — ``("continue", "in_progress")``.
55The ``iteration`` argument is *not* yet present in
56``session["iterations"]`` — the engine appends it after the verdict is
57decided. Anything that needs to look at "the last N iterations
58including the current one" composes the current ``iteration`` with
59``session["iterations"][-(N-1):]``.
61Determinism: same ``(session, iteration, now)`` triples produce the same
62``(VerdictLabel, VerdictReason)`` tuples. This is enforced by a property
63test in ``tests/test_mission_decide_determinism.py``.
65Cost guardrails are intentionally absent from this cascade. Real-time
66workload cost tracking is structurally inaccurate (Spot vs on-demand
67drift, EBS / EFA / egress not in the Pricing API, Cost Explorer 24h
68latency). Operators who need a cost cap should configure AWS Budgets
69and Cost Anomaly Detection at the account level — Mission caps only
70the controls the loop has direct visibility into.
71"""
73from __future__ import annotations
75import math
76from datetime import datetime, timedelta
78from .checkpoints import should_evaluate_now
79from .types import IterationRecord, SessionState, VerdictLabel, VerdictReason
81# <pyflowchart-code-diagram> BEGIN - auto-inserted, do not edit
82# Generated at (UTC): 2026-09-01T14:42:56Z
83# Generated from Git commit: 89b000378ed5a912a38c06f4feab2b029936ebcc
84# Flowchart(s) generated from this file:
85# * ``decide_verdict`` -> ``diagrams/code_diagrams/gco_mcp/mission/decide.decide_verdict.html``
86# (PNG: ``diagrams/code_diagrams/gco_mcp/mission/decide.decide_verdict.png``)
87# Regenerate with ``SOURCE_DATE_EPOCH=<unix-seconds> GCO_DIAGRAM_SOURCE_COMMIT=<40-char-sha> python diagrams/generate.py --code-only``.
88# <pyflowchart-code-diagram> END
91__all__ = [
92 "build_revision_rationale_template",
93 "decide_verdict",
94]
97def decide_verdict(
98 session: SessionState,
99 iteration: IterationRecord,
100 now: datetime,
101) -> tuple[VerdictLabel, VerdictReason]:
102 """Return the deterministic Verdict for the in-progress iteration.
104 The cascade order is fixed (see module docstring). The first matching
105 branch wins: a session that has both run out of iterations and has
106 every Criterion met returns ``("terminate", "max_iterations")``, not
107 ``("complete", "criteria_met")`` — budget caps are evaluated before
108 completion so the operator can tell the loop ended because it ran
109 out of budget rather than because the goal was reached on the
110 closing iteration.
111 Note: When the prior Execute_Phase ran a scripted Strategy and the
112 sandbox cap fired, ``_execute_script`` writes
113 ``iteration["sandbox_terminated_reason"]`` and the cascade returns
114 that reason verbatim before anything else is consulted. The
115 sandbox limit is a true budget cap — the script ran out of wall
116 clock during execution — so it routes to a ``terminate`` verdict
117 on the same path as the ``BudgetControls``-driven caps below.
118 """
119 # 0. Sandbox-cap propagation. ``_execute_script`` stashes the
120 # reason on the in-progress iteration when the sandbox runner
121 # raised :class:`SandboxTerminated`. Reading the sentinel here
122 # means the engine's Execute_Phase can complete cleanly (no phase
123 # failure) while still routing the verdict to the budget-cap path.
124 sandbox_reason = iteration.get("sandbox_terminated_reason")
125 if sandbox_reason is not None:
126 return ("terminate", sandbox_reason)
128 # 1a. max_iterations — the +1 captures "this in-progress iteration
129 # would be the Nth one to land", so a session with budget=N and N-1
130 # already-recorded iterations terminates on the Nth's Decide_Phase.
131 # ``-1`` is the explicit "uncapped" sentinel; the validator
132 # already enforced that any other non-positive value is rejected.
133 max_iter = session["budget"]["max_iterations"]
134 if max_iter != -1 and len(session["iterations"]) + 1 >= max_iter:
135 return ("terminate", "max_iterations")
136 # 1b. max_wall_clock — pure time arithmetic; missing started_at
137 # means the session has not yet recorded its first iteration's
138 # start, so no wall-clock can be measured.
139 if _wall_clock_exceeded(session, now):
140 return ("terminate", "max_wall_clock")
141 # 1c. no_progress — the counter is incremented by the engine only
142 # on evaluated iterations, so a session with all-skipped checkpoints
143 # cannot terminate for stagnation. When the session has sampling
144 # enabled, the heuristic gets priority (step 4 below) so the
145 # sampler can revise the strategy before the loop terminates.
146 # Without sampling, ``adjust`` is purely informational and
147 # ``no_progress`` terminates immediately.
148 if session["no_progress_counter"] >= session["stagnation_threshold"]:
149 if not session.get("use_sampling"):
150 return ("terminate", "no_progress")
151 # With sampling enabled, fall through to the heuristic check
152 # below. If the heuristic fires, the sampler gets one more
153 # chance. If it doesn't fire (e.g., the tool sequence changed
154 # after a prior revision), terminate for stagnation.
155 _stagnation_pending = True
156 else:
157 _stagnation_pending = False
159 # 2. Completion — every required Criterion met AND nothing inconclusive.
160 if _completion_satisfied(session, iteration):
161 return ("complete", "criteria_met")
163 # 3. Cadence-skip — bail before the heuristic fires so off-cadence
164 # iterations don't ever produce ``adjust``. The iteration_index
165 # passed to ``should_evaluate_now`` is the 0-indexed position of
166 # the in-progress iteration (which equals the count of already-
167 # persisted iterations).
168 if not should_evaluate_now(session, len(session["iterations"]), now):
169 return ("continue", "cadence_skip")
171 # 4. Strategy_Revision_Heuristic.
172 unproductive, _heuristic_reason = _strategy_unproductive(session, iteration)
173 if unproductive:
174 return ("adjust", "heuristic_unproductive")
176 # 4b. Deferred stagnation — the counter hit the threshold but the
177 # heuristic didn't fire (e.g., the tool sequence changed after a
178 # prior sampled revision). Terminate now.
179 if _stagnation_pending:
180 return ("terminate", "no_progress")
182 # 5. Default.
183 return ("continue", "in_progress")
186# ---------------------------------------------------------------------------
187# Budget helpers — pure
188# ---------------------------------------------------------------------------
191def _wall_clock_exceeded(session: SessionState, now: datetime) -> bool:
192 """True iff ``now - session["started_at"] >= max_wall_clock_seconds``.
194 Returns False when ``started_at`` is absent — a session that has
195 never been transitioned out of ``pending`` cannot have exceeded any
196 wall-clock budget. The engine writes ``started_at`` on the first
197 iteration entry, so this guard only matters for the synthetic
198 "decide called before run_iteration" path used in unit tests.
200 Returns False when ``max_wall_clock_seconds`` is the explicit
201 ``-1`` "uncapped" sentinel — the operator opted out of the wall-
202 clock cap and the cascade should fall through to the next branch
203 rather than terminate spuriously.
204 """
205 started_iso = session.get("started_at")
206 if not started_iso:
207 return False
208 max_seconds = session["budget"]["max_wall_clock_seconds"]
209 if max_seconds == -1:
210 return False
211 started = datetime.fromisoformat(started_iso)
212 return now - started >= timedelta(seconds=max_seconds)
215# ---------------------------------------------------------------------------
216# Completion check
217# ---------------------------------------------------------------------------
220def _completion_satisfied(
221 session: SessionState,
222 iteration: IterationRecord,
223) -> bool:
224 """True iff every required Criterion is met and none are inconclusive.
226 A session completes when all Criteria with ``required=True`` have
227 status ``met`` AND no Criterion (required or not) has status
228 ``inconclusive``. The ``required`` flag lives on the Criterion
229 declaration in ``session["criteria"]``; the per-iteration status
230 lives on ``iteration["criteria_evaluation"]``. The two are joined
231 by ``criterion_id``.
233 A session with zero declared Criteria can never complete on its own
234 — there are no required Criteria for the cascade to satisfy. The
235 operator drives such a session to terminal via ``mission_complete``
236 or a budget cap. We mirror that semantic here by returning False
237 when the criteria list is empty.
238 """
239 if not session["criteria"]:
240 return False
241 required_by_id = {c["criterion_id"]: c.get("required", True) for c in session["criteria"]}
242 for result in iteration["criteria_evaluation"]:
243 status = result["status"]
244 if status == "inconclusive":
245 return False
246 if required_by_id.get(result["criterion_id"], True) and status != "met":
247 return False
248 return True
251# ---------------------------------------------------------------------------
252# Strategy_Revision_Heuristic
253# ---------------------------------------------------------------------------
256def _strategy_unproductive(
257 session: SessionState,
258 iteration: IterationRecord,
259) -> tuple[bool, str]:
260 """Pure heuristic for the Strategy_Revision check.
262 Two clauses, evaluated in declaration order. The first match wins
263 so the returned reason is deterministic when both clauses fire.
265 * **Clause (a)** — the same ``tool_calls[*].tool_name`` sequence
266 has been used for the last 3 iterations (counting the in-progress
267 one) AND ``no_progress_counter >= ceil(stagnation_threshold / 2)``.
268 Needs at least 2 prior iterations to evaluate (3 total when the
269 current iteration is included). A scripted strategy contributes
270 an empty sequence so two scripts with the same body register as
271 "same sequence" — that's intentional: the heuristic flags repeats,
272 and an empty-sequence repeat across three iterations is a repeat.
273 * **Clause (b)** — the in-progress Observation contains at least
274 one ``errors`` entry that did not appear in the immediately
275 prior Iteration's Observation. Needs at least 1 prior iteration
276 to evaluate. Without a prior to compare to, "new" is undefined
277 and we return False.
279 Returns ``(False, "")`` when neither clause fires. When clause (a)
280 fires, the reason is ``"tool_sequence_repeating"``; when clause (b)
281 fires, ``"new_observation_errors"``. The reason string is
282 informational only — the Verdict's ``verdict_reason`` is always
283 ``"heuristic_unproductive"`` regardless of which clause matched.
284 """
285 # Clause (a): no_progress threshold AND tool-sequence repeat.
286 threshold = session["stagnation_threshold"]
287 half = math.ceil(threshold / 2)
288 if session["no_progress_counter"] >= half:
289 # Need at least 2 prior + the current = 3 total iterations.
290 prior = session["iterations"]
291 if len(prior) >= 2:
292 recent_three = [prior[-2], prior[-1], iteration]
293 sequences = [_tool_name_sequence(it) for it in recent_three]
294 if sequences[0] == sequences[1] == sequences[2]:
295 return (True, "tool_sequence_repeating")
297 # Clause (b): new errors in the latest Observation vs the prior one.
298 if session["iterations"]:
299 prior_observation = session["iterations"][-1].get("observation") or {}
300 prior_errors = list(prior_observation.get("errors") or [])
301 current_errors = list(iteration["observation"].get("errors") or [])
302 for err in current_errors:
303 if err not in prior_errors:
304 return (True, "new_observation_errors")
306 return (False, "")
309def _tool_name_sequence(iteration: IterationRecord) -> tuple[str, ...]:
310 """Extract the ordered tuple of ``tool_name``s from an iteration's strategy.
312 Returns an empty tuple when the strategy is a script (no
313 ``tool_calls``) or when ``tool_calls`` is missing. Two scripted
314 strategies therefore both produce ``()`` and compare equal — clause
315 (a) treats that as "same sequence", which matches the operator's
316 intent of flagging mechanical repetition regardless of mode.
317 """
318 strategy = iteration.get("strategy") or {}
319 tool_calls = strategy.get("tool_calls") or []
320 return tuple(str(call.get("tool_name", "")) for call in tool_calls if isinstance(call, dict))
323# ---------------------------------------------------------------------------
324# Revision rationale template
325# ---------------------------------------------------------------------------
328def build_revision_rationale_template(
329 session: SessionState,
330 iteration: IterationRecord,
331) -> str:
332 """Build the deterministic ``revision_rationale`` text for an ``adjust`` verdict.
334 Used both as the rationale on sessions with ``use_sampling=false``
335 and as the fallback rationale when sampling is rejected on a
336 ``use_sampling=true`` session.
337 Pure: depends only on persisted Session/Iteration fields, never
338 calls into the sampler or any other non-deterministic component.
340 The rendered text names the iteration index (1-indexed for
341 operator-friendliness), the heuristic reason, the unmet Criterion
342 ids (so the rationale points at the goal that's still moving), and
343 a one-line summary of the in-progress strategy (tool-name sequence
344 or ``"scripted strategy"``). The format is intentionally short and
345 machine-parseable — operators can grep it; no LLM is involved.
346 """
347 # Resolve the iteration index — the in-progress iteration has not
348 # been appended to session["iterations"] yet, so its 0-indexed
349 # position equals len(iterations) and the 1-indexed position is +1.
350 iteration_index_one_based = len(session["iterations"]) + 1
352 # Match the heuristic again so the rationale text matches whichever
353 # clause actually fired. Both calls are pure and cheap.
354 _, heuristic_reason = _strategy_unproductive(session, iteration)
355 if not heuristic_reason:
356 # decide_verdict only emits ``adjust`` when the heuristic fires,
357 # but the caller may invoke this template independently (e.g.
358 # the sampling-fallback path on a non-heuristic adjust) — fall
359 # back to a generic reason so the template stays usable.
360 heuristic_reason = "strategy_review_requested"
362 unmet_ids = [
363 result["criterion_id"]
364 for result in iteration["criteria_evaluation"]
365 if result["status"] == "unmet"
366 ]
367 unmet_summary = ", ".join(unmet_ids) if unmet_ids else "none"
369 strategy = iteration.get("strategy") or {}
370 if "script" in strategy:
371 strategy_summary = "scripted strategy"
372 else:
373 names = _tool_name_sequence(iteration)
374 strategy_summary = ", ".join(names) if names else "no tool calls"
376 no_progress = session["no_progress_counter"]
377 threshold = session["stagnation_threshold"]
379 return (
380 f"Strategy revised on iteration {iteration_index_one_based}: "
381 f"{heuristic_reason}. Unmet criteria: {unmet_summary}. "
382 f"Last strategy: {strategy_summary}. "
383 f"No-progress counter: {no_progress}/{threshold}. "
384 f"Adjusting approach for next iteration."
385 )