vance/codex-mmo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
codex-mmo/docs/EXTERNAL-VERACITY.md
T

100 lines
37 KiB
Markdown
Raw Permalink Normal View History

😏
2026-08-24 08:11:59 -07:00
# External Veracity Baseline
This document records the external contracts audited for Codex MMO 8.0.0 through 2026-08-23. It is a release baseline, not a promise that network services, account entitlements, prices, serving routes, or rolling catalogs will remain unchanged. The runtime has no web dependency; live discovery and probes are explicit operator actions. The app-server contract is different: 8.0.0 deliberately requires the exact reviewed Codex 0.149.0 generated schema rather than guessing compatibility. No other Codex release is accepted, migrated, or interpreted.
## Version and contract matrix
| Surface | Release baseline | Authoritative evidence | Runtime policy |
|---|---|---|---|
| Python | 3.11+ language/stdlib contract; tested locally on 3.14.7 | Python documentation and `pyproject.toml` 3.11 targets | No third-party Python packages; external executable requirements are documented separately. |
| Codex | Exact 0.149.0 executable and experimental protocol; live account catalog | Installed binary plus generated protocol schemas and tagged OpenAI Codex source/model metadata | Every new session, reattachment, and replacement root/worker host rejects an executable whose reported version is not exactly 0.149.0. `doctor --live` performs the complete contract check: exact version, 401 schema files, and digest `fcfeaf23728b96ab73916a21302eb7a16629e67ee99f7ee47b60fad6b6e5ee1a`. Built-in model availability remains account-derived; every other binary version is rejected. |
| Codex interactive resume | Codex 0.149.0 `--remote unix://… resume THREAD_ID` | Installed CLI help, generated schemas, tagged CLI/app-server source, and live local probe | MMO resolves one exact nonterminal immutable session, validates its pinned cwd/home/snapshot/executable/capability/run inventory, then attaches the stock TUI to the current canonical root-thread generation. A TUI-created fresh top-level context advances an audited lineage inside the same MMO run; predecessor IDs locate that session, while native/fork threads are excluded. Plain launch creates a new session; terminal sessions and model/cwd/profile overrides are rejected on resume. |
| Switchyard | 0.2.0 schema (`schema_version = 1`) and CLI; Rust 1.96.1 MSRV for source installation | NVIDIA NeMo Switchyard 0.2.0 source, CLI, and published crate | Emit only documented `format`, `base_url`, `api_key_env`, `extra_headers`, and `max_retries` client fields; the optional installer pins the verified top-level crate version. Validate the CLI's `IpAddr` host type early and restrict the unauthenticated managed ingress to loopback. |
| Codex image CLI | Codex 0.149 shared CLI and image crate | Tagged OpenAI Codex source plus the installed CLI | Accept the upstream PNG/JPEG/GIF/WebP set and place `--` before the stdin prompt operand so variadic `--image` cannot consume `-` as another image path. |
| Codex app-server execution and usage | Installed Codex 0.149.0 `generate-json-schema --experimental`, Unix WebSocket listener, official app-server documentation, tagged protocol/source, and Switchyard 0.2.0 translation/usage normalization | Exact generated schema set plus official OpenAI documentation and tagged OpenAI Codex/Switchyard source | Require Unix app-server hosts for root/workers; send headerless JSON-RPC-shaped messages, one per text frame, after the exact `initialize`/`initialized` lifecycle; gate exact thread/goal/turn/control/history/pending/approval requests and method-specific responses. Switchyard 0.2.0 drops Codex's Responses namespace-tool shape, so only that recorded gateway release receives a temporary flat dynamic-tool bridge routed back through Codex's own MCP API. Upstream PR #384/commit `c7beccd4891fa5cfe3a3b94fdd376f5765864507` fixes native namespace preservation after the v0.2.0 tag; its first published release triggers the documented native qualification and complete shim removal. Retry only the documented `-32001`/`Server overloaded; retry later.` condition with exponential delay and jitter. Persist cumulative token usage without double-counting notifications; price cache categories independently and include root plus workers. No stdio/fallback host exists. |
| App-server WebSocket | RFC 6455 over a local Unix socket | RFC 6455 plus official Codex Unix-socket transport documentation | Require an HTTP/1.1 `101` Upgrade with token-valid `Upgrade`/`Connection`, the exact accept hash, and no unsolicited subprotocol or extension. Mask client frames; reject masked server frames, RSV bits, non-minimal lengths, malformed control/fragment sequences, invalid close codes/reasons, binary application messages, and invalid UTF-8; perform a close handshake on normal shutdown or protocol failure. |
| Codex authentication | Codex 0.149 `login status` and auth storage | Tagged OpenAI Codex CLI/login/storage source and installed client | Doctor requires both a successful base-home login and an `auth.json` that can be linked/copied into generated homes. Codex 0.149 keys keyring entries to canonical `CODEX_HOME`, so keyring-only login is reported as unusable for isolated homes rather than accepted. Generated homes force file storage to preserve the selected link/copy behavior. |
| Provider HTTP headers | RFC 9110; Codex 0.149 and Switchyard 0.2.0 Rust HTTP clients | IETF semantics plus tagged/downstream source | Validate field names, reject case-insensitive duplicates, and reject non-HTAB C0 controls and DEL; preserve configured Codex environment-backed header values through secret filtering when present. |
| Provider base URLs | RFC 3986 HTTP(S) URI syntax; Codex 0.149 and Switchyard 0.2.0 endpoint builders | IETF URI syntax plus tagged downstream source | Require an RFC 3986 absolute HTTP(S) URI with a host and no userinfo or malformed percent encoding; reject query/fragment roots because Codex slash-normalizes and appends its path while Switchyard also normalizes known endpoint suffixes, without URI-joining embedded query/fragment components. |
| MCP | Protocol `2025-06-18`, STDIO transport | Model Context Protocol lifecycle, transport, base schema, and tools specifications | One supported revision, explicit initialization state, newline-delimited UTF-8 messages, string/integer request IDs, object parameters, protocol errors for unknown tools or invalid arguments, and `isError` tool results for execution failures. |
| JSON-RPC | 2.0 | JSON-RPC 2.0 specification as narrowed by the MCP schema | Exact version/method/params/ID validation; parse, invalid-request, invalid-params, and method-not-found error codes; no notification response. |
| JSON | RFC 8259 | IETF JSON specification and Python `json` documentation | Reject `NaN`, infinities, overflow-to-infinity numbers, duplicate object names, lone UTF-16 surrogates, and non-finite output. This chooses deterministic interoperable behavior where RFC 8259 warns that duplicate names and unpaired surrogates produce unpredictable results. |
| JSON Schema | Project subset informed by Draft 2020-12 | JSON Schema 2020-12 core/validation specifications, RFC 3339, RFC 3986, and the official test suite | No remote references/vocabularies; unsupported keywords rejected; Python `re` with portable patterns. `date` and `date-time` implement RFC 3339 syntax/ranges including offset-shifted leap seconds and lowercase `t`/`z`; `uri` implements absolute RFC 3986 syntax. |
| TOML | 1.0.0 | TOML language specification and Python `tomllib` documentation | Emit valid Unicode scalar values and escape every forbidden basic-string control character, including U+007F. |
| HTTP readiness | RFC 9110 successful-response semantics | IETF HTTP Semantics | Only status codes 200–299 establish readiness. Redirects and errors do not. |
| Release archives | GNU/POSIX tar conventions, PKZIP ZIP metadata, Python 3.11+ `tarfile`/`zipfile` APIs | Python archive documentation and format specifications | Reject traversal, links, special members, count/size excess, negative sizes, duplicates, implicit or extra directory entries, and exact mode drift; stream tar verification/extraction. |
| XDG paths | Base Directory Specification 0.8 | freedesktop.org specification | Empty or relative base-directory values are ignored. `XDG_BIN_HOME` is a documented project extension. |
| Linux process state | `/proc/<pid>/stat` ABI | Linux `proc_pid_stat(5)` documentation | Parse fields after the command name's final `)`; use state, process group, and start-time fields. Persist and recheck PID/start-token fingerprints before signalling app-server groups, publish ownership before protocol bootstrap, and retire live group descendants even when the recorded leader has exited. |
| Terminal job control and exit status | POSIX Issue 7 semantics; Python 3.11+ `process_group`; GNU Bash signal-status convention | POSIX `tcsetpgrp()`/terminal-interface specifications, Python `subprocess` documentation, and the Bash manual | With terminal stdin/stdout, keep Codex in the controlling-terminal session even if stderr is redirected; use a dedicated foreground process group; restore terminal ownership/modes; preserve the caller's color environment; translate Python's negative signal return code to `128 + signal` at command boundaries. |
| Git writer isolation | Git 2.x CLI; tested locally on 2.55.0 | Git `worktree`, `diff`, `apply`, and `rev-parse` manuals plus Python filesystem-codec documentation | Run MCP writers in isolated worktrees, capture binary-safe patches and base fingerprints, correlate artifact hashes, hard-fail/taint boundary violations, and require explicit accepted integration into the canonical tree. |
| Local Qwen/llama.cpp | Qwen3.5-9B model card and rolling llama.cpp server docs observed 2026-08-15 | Official Qwen model card and llama.cpp server source/docs | Distinguish the model's native 262,144-token multimodal capability from the bundled text-only 32K/8K resource cap; pin matching context, Jinja tool, reasoning-off, and no-projector example flags. |
| OpenCode Go | Public catalog of 29 IDs observed 2026-08-23 | Commit-pinned OpenCode Go endpoint/pricing docs, live endpoint, and OpenCode-maintained Models.dev | IDs are dynamically checked; the explicit endpoint table is transport authority while Models.dev supplies capabilities. Both source values and any disagreement are fingerprinted. Provider time-tier and dash-valued rows remain exact evidence, but no tier or absent scalar is misrepresented as a universal price. Optional capabilities and model-specific effort selectors are asserted only when present. `hy3-preview` remains visible but non-agent-compatible because it has listing evidence only; the contributor Muse route now has a complete executable join. Bundled translated routes remain text-only pending multimodal-path verification. |
| OpenCode Zen | Public live listing of 64 IDs observed 2026-08-23; shared API base and key | Commit-pinned OpenCode Zen endpoint/pricing/deprecation docs, live endpoint, OpenCode-maintained Models.dev, and Switchyard 0.2.0 real-binary local-mock transport probe | Join live presence to exact-ID capability metadata; select 25 Responses, 20 Chat, and 13 Anthropic executable routes with `OPENCODE_API_KEY`; retain 6 Google-native entries as catalog-only. Two IDs present in the live listing and Models.dev but absent from the provider documentation are marked `live-undocumented` and remain unpriced. Use provider billing rates and expose only representable efforts/prices. Time- and context-tier rows remain evidence rather than false scalar rates. Exclude the listing's request-time `created` value from normalized model records while retaining the exact raw capture digest. |
| OpenRouter | Public Models API snapshot of 422 text-input/text-output IDs observed 2026-08-23; OpenAI-compatible Chat base | OpenRouter Models API/OpenAPI, provider-routing, reasoning, tool-calling, and usage documentation; Switchyard 0.2.0 tagged configuration and real-binary local-mock transport probe | Use `https://openrouter.ai/api/v1` with `openai_chat` and `OPENROUTER_API_KEY`; derive capabilities only from explicit fields; preserve typed provider policy; record requested aggregator route separately from the actual serving provider/endpoint reported; keep translated execution text-only and omit prices the scalar schema cannot represent. Discovery consults stored credentials only for the official origin; an explicit blank key or custom source does not read them. |
| Z.AI | Reviewed General API/Coding Plan records through 2026-08-18; authenticated nine-ID Coding Plan listing observed 2026-08-19 | Z.AI model, parameter, pricing, Coding Plan, Codex, and Claude Code integration docs plus the authenticated models endpoint | Keep General API and Coding Plan as distinct access products and credentials. Bundled GLM-5.3 uses exactly the Coding Plan Responses base `https://api.z.ai/api/v1` and `ZAI_CODING_API_KEY`, with `low`, `high`, and `max` effort; the separate Chat-compatible base remains `/api/coding/paas/v4`. The three profile-selected bindings are not aliases for the six other advertised IDs. The Anthropic path remains catalog-only because its Bearer auth is not executable through Switchyard 0.2.0's fixed `x-api-key` behavior. Mutable documentation-capture drift blocks a current full-fingerprint claim until separately reviewed. |
Machine-readable route/model sources and inventory IDs are in `config/upstream-inventory.json`. The reviewed source records are common, canonical-hash-checked envelopes in `config/inventory-snapshots/` for OpenRouter, OpenCode Zen/Go, Z.AI, and Codex. Generated catalog entries keep `source`; OpenCode Zen/Go and Z.AI rows additionally separate `availability_source`, `capability_source`, and `pricing_source` where no one upstream artifact supplies all three accurately.
## Audit classifications
| Class | Finding and disposition |
|---|---|
| A — implementation defect | Fixed app-server messages that incorrectly carried the omitted `jsonrpc` header; WebSocket handshakes and frames that accepted invalid RFC 6455 state, masking, lengths, UTF-8, close payloads, fragmentation, subprotocols, or extensions; retries of unrelated `-32001` errors rather than only the exact documented overload; an installer that did not resolve, install, and post-verify the one supported Codex version; catalog discovery that consulted OpenRouter credentials for explicit blank-key or custom-source probes; and a stale generic catalog user agent. Also fixed unsupported MCP-version echo, missing lifecycle/envelope validation, acceptance of null/fractional MCP request IDs and positional parameter arrays, and conversion of unknown tools or schema-invalid arguments into execution-error tool results; removed the catalog's case-insensitive raw-upstream-ID fallback so only exact provider-qualified model keys resolve; a doctor probe that sent post-initialize traffic before validating negotiation and a `--probe` flag silently ignored without `--live`; false-success exit codes for invalid profile/suite validation, failed evaluation tasks, timed-out job waits, and signal-terminated children; catalog source options silently ignored unless a second flag was supplied and Codex catalog commands bypassing configured binary/home settings; versioned settings/profile/catalog/smoke/evaluation fields silently accepting typos, wrong TOML scalar/container types, boolean schema versions, coerced numeric IDs, explicitly named catalog/smoke files silently disappearing, pack-relative instruction/contract fields pointing outside their documented directories, and working-directory-relative base Codex homes; omission of Codex 0.149's `indexed` web-search mode and acceptance of arbitrary approval-policy strings; inert cross-driver provider fields and unchecked built-in provider/auth pairs; source installation that followed linked payload members and tolerated missing declared members; installer replacement or recursive uninstall of an unowned install root, or explicitly substituted purge roots, without manifest identity; acceptance of non-finite, duplicate-member, or invalid-scalar JSON and boolean-as-integer JSON manifest versions; Draft 2020-12 boolean/integer mismatches; permissive `date`, `date-time`, and `uri` format checks that rejected valid RFC 3339 leap seconds/lowercase markers while accepting non-RFC timestamps, non-month-end leap seconds, and malformed RFC 3986 URIs; invalid TOML emission for U+007F; redirects accepted as readiness; XDG relative-path acceptance; unsafe `/proc` tokenization; detached interactive terminal execution, forced color suppression, and false non-interactive classification when only stderr was redirected; Git rename/source-path, file-mode, and filesystem-codec corruption in workspace audits; result truncation that appended its marker beyond the hard character ceiling; stale or discarded-on-failure Codex usage plus cost estimates that conflated base input, cache reads, and cache writes or omitted root usage; OpenCode Go rows that defaulted an optional structured-output capability to true without source evidence, assigned an unsubstantiated blanket `high` reasoning effort, or copied a time-varying Models.dev price into fields attributed to provider docs; a doctor credential gate that treated unknown built-in authentication as valid, including keyring-only state inaccessible from generated homes, and catalog verification that passed when its requested Codex source was unavailable; root/worker executable divergence that ignored `settings.toml` for detached workers; a dead global auth-link setting, propagation of unrelated MCP OAuth credentials, and generated-home `auto` storage that could migrate shared auth into a home-specific keyring namespace; dead/wrong downstream header fields, incomplete header-value and case-insensitive-name validation, filtering of configured environment-backed header secrets, acceptance of unusable HTTP base roots, and late/coercive handling of Codex custo
| B — test defect | Updated the Codex stand-in to expose current context/effort, exec-usage fields, plus deprecated/internal rows and variadic image parsing; made MCP tests and the doctor perform and verify the negotiated two-phase handshake; and added boundary regressions for non-finite/duplicate-member/invalid-scalar JSON, JSON Schema boolean/fractional-integer behavior, exact integer manifest versions, TOML U+007F/surrogates, HTTP redirects, HTTP base-root syntax/join safety, Codex custom-route unsigned retry/idle types, closed and strongly typed versioned configuration/missing explicit files, pack-member and absolute-home paths, per-driver fields and built-in provider/auth pairs, closed evaluation fields and fixture safety, exact web-search/approval-policy enums, exact loopback gateway-host parsing, full gateway-route identity, current usage/cache-read/cache-write/failure accounting, exact OpenCode Go structured-output, reasoning-effort, and scalar-pricing sets, exact OpenCode Zen route/capability/pricing counts and snapshot reconstruction, common inventory snapshot integrity and exact snapshot-to-catalog drift, exact OpenRouter capability/pricing counts, pagination completeness, optional-auth discovery without secret leakage to URL overrides, OpenRouter drift comparison, exact Switchyard route generation, truthful command exit/source-option semantics, hard rejection of obsolete credential and header names, link/special/missing members, unowned/mismatched install and uninstall roots plus retained-manifest purge, built-in login/transfer status, keyring-only rejection, auth link-mode precedence/OAuth exclusion, pinned root/worker executables, real Codex strict configuration/model-catalog parsing, real Switchyard dry-run parsing, unavailable requested catalog sources, hard result truncation, pathological `/proc` names, XDG relative values, terminal foreground ownership/color/resize/restoration with redirected stderr, staged and committed Git changes, trailing-whitespace repository roots, ZIP and tar file/directory permission mutation, pinned Switchyard installation, image prompt separation, the explicit interactive Codex-option boundary, canonical v2 header fields, environment-backed secret propagation, RFC 3339/RFC 3986 format edges, case-insensitive header collisions, OpenCode time-tier parsing, and the exact dedicated Z.AI Responses route. At JSON-Schema-Test-Suite commit `6648e8194c69697b2e1a15fe76a06a480b183a51`, all 123 main-suite groups whose schemas are accepted by the project subset passed 469 tests with 0 failures; 239 groups using unsupported keywords were skipped. The annotation-only `format.json` and remote-vocabulary `vocabulary.json` files are excluded by policy. |
| C — documentation defect | Corrected an unqualified catalog-model example and ambiguous model-key placeholder, plus the false “no runtime dependencies beyond Python” claim and omitted Bash and Switchyard 0.2.0 Rust-1.96.1 build requirements, absolute secret-isolation language for a necessarily name-based environment filter, a local Qwen example that did not pin the catalog's reduced deployment contract, the package-version command, an interactive image example missing the required Codex-option separator, a smoke example made entirely of nonexistent field names, active-agent/root counting and the derived native-thread limit, plus the single-model provider description, exact base-URL joining behavior, canonical credential fields, downstream retry semantics/defaults, header names, XDG extension claim, JSON Schema scope, Codex/API context distinction, OpenCode/Z.AI provenance, interactive-terminal behavior, archive verification guarantees, and current Z.AI Codex configuration. Repaired dead or indirect source links. |
| D — stale upstream data | Updated the Codex baseline and exact app-server schema to 0.149.0; refreshed OpenCode Go to 29 IDs, OpenCode Zen to 64 IDs, and OpenRouter to 422 reviewed text-input/text-output IDs as observed 2026-08-23. The Go snapshot now includes the complete Muse contributor, DeepSeek vision, and Ox rows while keeping `hy3-preview` listing-only; complete dash-valued or tiered billing rows remain evidence without invented scalars. Two Zen IDs that are live and present in Models.dev but absent from the provider documentation remain explicitly `live-undocumented`. Updated OpenRouter field changes without weakening route policy or capability derivation. Runtime discovery remains authoritative. |
| E — breaking compatibility boundary | Every MMO-owned document requires exact schema 8 and, where applicable, exact package/profile version 8.0.0. The redundant lifecycle-version field, retired active-work/rollout/flat-control fields, lifetime-spawn fields, old provider-qualified keys, obsolete credentials/headers, and retired IDs are rejected. No migration, normalization shim, alternate execution host, automatic rewrite, state inspection fallback, or lookup fallback remains. External protocol version labels owned by Codex, MCP, providers, or Switchyard are not MMO compatibility readers. |
| F — upstream inconsistency | OpenAI API limits differ from Codex session-catalog limits. The rolling official app-server page says paginated thread creation is not yet supported, while the exact installed 0.149.0 binary accepted `thread/start` with `historyMode: "paginated"` and served the resulting `thread/turns/list`; MMO follows the exact-pinned executable and generated schema rather than projecting rolling prose onto another release. OpenCode Go's `/models` description implies metadata while its response currently supplies IDs, five live IDs have Models.dev capability evidence but no current provider endpoint/pricing-table row, and Models.dev's default OpenAI-compatible adapter disagrees with the explicit Anthropic `/messages` rows for four current Qwen models. OpenCode Zen's endpoint table and live listing are not identical, every live row's `created` field is regenerated as the request time, two live/Models.dev IDs are absent from the documentation table, its Kimi K2.5 cache-read price differs from Models.dev, and its source repeats one identical Grok 4.6 tier row. OpenRouter's Models overview documents a public listing while its API reference declares Bearer security even though the audited endpoint returned HTTP 200 without a key. Switchyard 0.2.0 exposes one generic reasoning preset list for every reasoning-capable route and equates parallel-tool support with basic tool support in its Codex discovery metadata. The project records both sides and chooses the versioned execution-path authority. |
| G — feature gap | Switchyard 0.2.0 has no Google-native client format, so six live OpenCode Zen Gemini entries are catalog-only; its Responses, Chat, and Anthropic groups remain executable. Switchyard also cannot configure an environment-backed Bearer header for an `anthropic_messages` client, so the Z.AI Anthropic-compatible path is catalog-only; the executable Z.AI Responses and Chat paths remain available. Other environment-backed arbitrary Switchyard headers are deliberately rejected; use its API credential field or a direct Codex provider. Codex 0.149 has no supported way to alias a keyring entry across canonical `CODEX_HOME` paths, so isolated MMO homes require file-backed built-in authentication. The bundled OpenCode Zen/Go and OpenRouter translation routes expose text only even where upstream metadata advertises attachments. OpenCode time- or context-tiered prices and OpenRouter's tiered, dynamic, fixed, media, web-search, one-hour-cache-write, and differently priced internal-reasoning charges cannot all fit the catalog's scalar token-rate schema and are retained as snapshot evidence rather than flattened inaccurately. |
| H — unverified | `hy3-preview` limit/capability metadata; account-wide Codex entitlements beyond the observed catalog and one Luna turn; the cause and duration of the observed OpenCode `403` and Z.AI `429` responses; authenticated provider limits, tools, streaming, cancellation, multimodal fidelity, and all selected model bindings; OpenRouter downstream provider identity for the successful generic Mistral probe; provider-wide rate limits; local OpenAI-compatible server fidelity; Tool MCP server behavior/authorization; and future billing remain external uncertainties. One-shot text inference does not establish portfolio-wide availability or role efficacy. Runtime resume does not depend on rollout-file layout; it relies on the pinned app-server protocol and exact thread history. |
The 2026-08-18 independent follow-up classified acceptance of non-month-end leap seconds as an additional A-class implementation defect and corrected it with offset-crossing regressions. It classified the prior `392`/`78` JSON Schema suite count as a C-class documentation defect because the selection rule was unstated and could not be reproduced from the pinned suite; the B-class row now records the exact reproducible selection and exclusions.
The reduced local Qwen context/modalities are verified as an intentional project deployment policy, not an external-contract discrepancy or legacy-compatibility claim. The documentation now separates that policy from the checkpoint's upstream-native capabilities.
The RFC format follow-up reran all 160 `date`, `date-time`, and `uri` cases in the pinned official Draft 2020-12 optional-format fixtures with zero mismatches. Project tests retain the defect-focused boundary cases offline; release validation does not fetch the upstream suite.
The 2026-08-23 bounded live acceptance probe disabled Switchyard retries and sent one text request with `max_output_tokens = 8` through each executable external provider/protocol pairing. OpenRouter Chat through Switchyard returned HTTP 200 for the reviewed Mistral Nemo binding. All Go and Zen Responses, Chat, and Anthropic pairings returned upstream HTTP 403; Z.AI Coding Plan Responses and Chat returned upstream HTTP 429. A separate disposable 0.149.0 app-server completed one low-effort built-in Luna turn. The probes were not retried, did not touch MMO user sessions, and do not prove tools, streaming, multimodal transport, route telemetry, or other model bindings.
## Independent adversarial verification follow-up
The final baseline-to-candidate audit found and corrected several integration defects that individual component checks had missed: interactive setup could leave a durable session in `starting`; failed atomic runner launch records consumed a budget documented as admission-only; direct-provider credential precedence differed from gateway/doctor precedence; cancellation silently truncated state but not audit data; maximum-length evaluation IDs could be created but not loaded; blank evaluation patterns/commands could pass vacuously; normal catalog regeneration printed `passed: false` after successfully repairing drift; the CLI accepted a result limit below its enforced minimum; install validation reused and deleted a fixed state path; a failed upgrade could leave the preserved credential file migrated while restoring the old payload; a retained uninstall marker could authorize unrelated content recreated at the former install path; a fresh installation could leave an older-generation retained owner marker in active config/state; and release-set publication did not reject a symlink destination before replacement. Regression tests now exercise each boundary.
The same audit reviewed every changed test. The redirected-stderr PTY change is intentional rather than a weakened assertion: stdin and stdout still own the controlling terminal, while the test proves color, foreground process-group transfer, resize delivery, terminal restoration, and successful operation with stderr captured separately. Exact route-qualified model keys remain mandatory throughout the catalog and every bundled profile; no old-key alias exists.
## Primary references
- [MCP lifecycle](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle), [base protocol schema](https://modelcontextprotocol.io/specification/2025-06-18/basic), [tools](https://modelcontextprotocol.io/specification/2025-06-18/server/tools), and [STDIO transport](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports)
- [JSON-RPC 2.0](https://www.jsonrpc.org/specification)
- [RFC 8259 JSON](https://www.rfc-editor.org/rfc/rfc8259), [RFC 3339 date/time](https://www.rfc-editor.org/rfc/rfc3339), [RFC 3986 URI syntax](https://www.rfc-editor.org/rfc/rfc3986), [JSON Schema Draft 2020-12 validation](https://json-schema.org/draft/2020-12/json-schema-validation), and the [official JSON Schema Test Suite at audited commit `6648e819`](https://github.com/json-schema-org/JSON-Schema-Test-Suite/commit/6648e8194c69697b2e1a15fe76a06a480b183a51)
- [TOML 1.0.0](https://toml.io/en/v1.0.0), [Python `tomllib`](https://docs.python.org/3/library/tomllib.html), and [HTTP Semantics RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html)
- [Python `tarfile`](https://docs.python.org/3/library/tarfile.html), [Python `zipfile`](https://docs.python.org/3/library/zipfile.html), and the [PKWARE ZIP application note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT)
- [XDG Base Directory 0.8](https://specifications.freedesktop.org/basedir/)
- [Linux `proc_pid_stat(5)`](https://man7.org/linux/man-pages/man5/proc_pid_stat.5.html)
- [POSIX `tcsetpgrp()`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetpgrp.html), [POSIX terminal process groups](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html), and [Python 3.11 `subprocess`](https://docs.python.org/3.11/library/subprocess.html)
- [GNU Bash exit status](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html), including the `128 + signal` convention for commands terminated by a fatal signal
- [Git `diff`](https://git-scm.com/docs/git-diff) and [Git `rev-parse`](https://git-scm.com/docs/git-rev-parse)
- [Python filesystem encoding helpers](https://docs.python.org/3.11/library/os.html#os.fsencode)
- [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B) and [llama.cpp server documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)
- [OpenAI Codex CLI reference](https://developers.openai.com/codex/cli/reference), [OpenAI Codex models](https://developers.openai.com/codex/models), [OpenAI API models](https://developers.openai.com/api/docs/models), and [Codex client model source](https://github.com/openai/codex/blob/main/codex-rs/models-manager/models.json)
- [Codex 0.149 shared CLI image arguments](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/utils/cli/src/shared_options.rs) and [image decoder features](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/utils/image/Cargo.toml)
- [Codex 0.149 provider header construction](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/model-provider-info/src/lib.rs)
- [Official Codex app-server protocol and lifecycle](https://learn.chatgpt.com/docs/app-server), [Codex 0.149 app-server source](https://github.com/openai/codex/tree/rust-v0.149.0/codex-rs/app-server), the locally generated Codex 0.149 experimental JSON Schemas, [RFC 6455 WebSocket](https://www.rfc-editor.org/rfc/rfc6455), [Switchyard 0.2.0 usage normalization](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/crates/switchyard-components/src/stats/usage.rs), and [Switchyard 0.2.0 cache-category cost formula](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/crates/switchyard-components/src/stats/cost.rs)
- [Codex 0.149 login-status CLI](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/cli/src/main.rs), [status exit behavior](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/cli/src/login.rs), and [CODEX_HOME-scoped auth storage](https://github.com/openai/codex/blob/rust-v0.149.0/codex-rs/login/src/auth/storage.rs)
- [NVIDIA NeMo Switchyard](https://github.com/NVIDIA-NeMo/Switchyard), [Switchyard 0.2.0 workspace version/MSRV](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/Cargo.toml), [CLI host type](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/crates/switchyard-server/src/cli.rs), [ingress routes and Codex discovery metadata](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/crates/switchyard-server/src/lib.rs), [endpoint construction](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/crates/libsy-llm-client/src/backend.rs), [tagged OpenRouter client example](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.2.0/benchmark/server-configs/tb-lite-llm-classifier-opus-kimi-gemini.toml), [published Switchyard Server 0.2.0 package](https://docs.rs/crate/switchyard-server/0.2.0), and [Cargo `install --version`](https://doc.rust-lang.org/cargo/commands/cargo-install.html#options)
- [OpenCode Zen](https://opencode.ai/docs/zen), [commit-pinned Zen documentation source](https://github.com/anomalyco/opencode/blob/03bba464d46f3eddf74195919b1344aa937f7b11/packages/web/src/content/docs/zen.mdx), [live Zen IDs](https://opencode.ai/zen/v1/models), [OpenCode Go](https://opencode.ai/docs/go/), [commit-pinned Go documentation source](https://github.com/anomalyco/opencode/blob/03bba464d46f3eddf74195919b1344aa937f7b11/packages/web/src/content/docs/go.mdx), [live Go IDs](https://opencode.ai/zen/go/v1/models), and [Models.dev](https://github.com/anomalyco/models.dev)
- [OpenRouter Models API](https://openrouter.ai/docs/api/api-reference/models/get-models), [models overview](https://openrouter.ai/docs/guides/overview/models), [OpenAPI document](https://openrouter.ai/openapi.json), [Chat Completions](https://openrouter.ai/docs/api/api-reference/chat/send-chat-completion-request?explorer=true), [reasoning](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens), [tool calling](https://openrouter.ai/docs/guides/features/tool-calling), and [usage accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting)
- [Z.AI Coding Plan](https://docs.z.ai/devpack/overview), [Codex integration](https://docs.z.ai/devpack/tool/codex), [Claude Code integration](https://docs.z.ai/devpack/tool/claude), [API authentication](https://docs.z.ai/api-reference/introduction), [core parameters](https://docs.z.ai/guides/overview/concept-param), [model overview](https://docs.z.ai/guides/overview/overview), and [pricing](https://docs.z.ai/guides/overview/pricing)
## Operator verification
Static release acceptance is offline. To observe current external state, run:
```bash
codex-mmo catalog verify --remote --codex
codex-mmo doctor --profile PROFILE --live --probe
codex-mmo profile smoke PROFILE --cwd "$PWD"
```
Missing expected Codex models may be normal account gating. Unknown public models, wrong route/protocol bindings, malformed generated downstream configuration, serving-policy mismatches, or observed IDs absent from a dynamic catalog are surfaced as drift.
Reference in New Issue Copy Permalink