codex-mmo/README.md

# Codex Multi-Model Orchestrator 8.0.0

Codex Multi-Model Orchestrator (Codex MMO) compiles provider-agnostic agent profiles into immutable snapshots used by persistent interactive sessions. A profile chooses models by exact access route, assigns roles and trust boundaries, and combines Codex native agents with a mechanically supervised Agent MCP runtime when useful.

The bundled portfolio is designed to show practical reasons for composition: useful concurrency, independent reproduction and verification, source-to-code audit, multimodal browser review, contract separation, security challenge, controlled weak-model use, and route resilience. It does not prefer one maker merely because its frontier model wins an aggregate benchmark.

## Single generation-8 format

This release is a clean break. It rejects:

- any MMO-owned document whose `schema_version` is not `8`, including settings, catalogs, Tool MCP registries, inventory snapshots, profiles, smoke/evaluation definitions, compiled snapshots/guidance, gateways, sessions, runs, jobs, and install/release manifests;
- legacy provider-qualified model keys;
- retired profile and evaluation IDs;
- flat `can_control`, active-work/renewal/hard-wall fields, lifetime `max_total_spawns` and `max_spawns`, rollout-path state, per-spawn `timeout_seconds`, role progress-checkpoint fields, and old smoke timeout keys.

There are no migration commands, compatibility readers, alternate execution hosts, aliases, fallback key lookups, automatic rewrites, or compatibility profiles. Recreate configuration and profile packs directly in generation 8, and externally archive or remove any older runtime state before installation. Profile `version` must exactly equal the package version (`8.0.0`); it is not an independent upgrade axis. External formats owned by Codex app-server, MCP, providers, and Switchyard retain their upstream version identifiers and are not MMO schema surfaces.

## What the runtime enforces

- Exact route/model/reasoning bindings with separate maker, API operator, access product, protocol, billing, and serving-provider identities.
- Content-addressed immutable snapshots inherited by every descendant.
- Native, Agent-MCP, and hybrid orchestration selected per profile.
- Authenticated lineage, directed delegation/control graphs, depth, reusable active capacity, resource groups, and cancellation. Terminal workers release admission capacity, so long sessions can delegate sequentially without exhausting a lifetime counter.
- One isolated Unix app-server host for every root and Agent-MCP worker. Workers retain one exact persistent thread; an interactive root retains an ordered, auditable lineage when the attached stock TUI deliberately starts a fresh top-level context. Durable event/terminal history, transport recovery, continuation, and bounded partial evidence remain inside the same immutable MMO session and run.
- Goal-mode token budgets with profile ceilings, turn mode without a task clock, and warning-only stall intervals. Provider/model slowness never discards work and models never track elapsed time themselves.
- Compare-and-swap inspect/trace/steer/interrupt/pause/continue/detach/stop/finalize/compact/respond/effort/fork controls constrained by per-target action grants; pending user-input, MCP-elicitation, and approval responses retain their method-specific app-server shapes. MCP pause is a durable cold pause that releases host capacity; detach is the live-background operation.
- Typed failed-turn retention, provider-limit suspension with raw/reset evidence, and exact runtime package/code fingerprints make recovery causes and stale live hosts explicit.
- Per-route availability: a missing optional worker degrades explicitly; an unavailable root prevents startup.
- MCP-only structured literal tasks, read-only workspaces, strict contracts, and mandatory verification for weak roles.
- Isolated Git worktrees for delegated writers, binary-safe patches, artifact/hash evidence, taint detection, and explicit root integration.
- Lossless cursor-paginated worker results and valid filtered traces: `agents_wait` returns compact revision-addressed state with previews opt-in, `agent_result --cursor` returns contiguous result pages, and `agent_trace_record` pages one oversized filtered event until `next_cursor` is null. Malformed trace records remain bounded because they cannot be structurally filtered. Reading, accepting/rejecting, and—only for accepted patches—integrating remain separate monotonic steps. Model-visible MCP responses do not expose supervisor result/event/stderr/socket paths.
- Complete-route modality checks, including image and tool-produced media preservation.
- Typed OpenRouter serving policies plus requested-route and actual-serving telemetry.
- Separate subscription, local-resource, actual-API, and API-equivalent cost ledgers.
- Operator-owned Tool MCP definitions that profiles may only narrow.

Structured contracts prove shape and correlated artifacts/commands, not semantic truth. Native Codex controls remain less observable than Agent MCP; all bundled featured native roles are read-only.

## Architecture

```text
immutable MMO session and Unix app-server root host
├── detachable stock Codex TUI or noninteractive client on the current root generation
├── direct critical-path work and canonical workspace
├── selected operator-owned Tool MCP servers
├── native Codex agents (optional, low overhead, advisory controls)
└── mmo_mesh Agent MCP (optional, mechanical controls)
    └── isolated persistent app-server workers
        ├── pinned route/model/reasoning
        ├── lineage, goal budgets, scopes, contracts, live control
        └── durable trace/partial/result and isolated Git patch lifecycle

catalog identity
model maker + upstream ID
          ×
API operator + access product + wire protocol + billing route
          ×
actual serving provider when the endpoint reports it
```

The runtime contains no model-family scheduling branches. Profiles express intelligence policy; the kernel enforces generic process, resource, result, and integration mechanics.

## Bundled portfolio

### Featured profiles

| Profile | Workload | Root | Composition | Engine |
|---|---|---|---|---|
| `adaptive-engineering` | General decomposable engineering | GPT-5.6 Sol via ChatGPT | Luna scout, DeepSeek V4 Pro implementer, Opus 5 reviewer | Hybrid |
| `codex-harness-team` | ChatGPT-only context isolation/control | GPT-5.6 Sol via ChatGPT | Luna scout, Terra invariant role, fresh Sol critic | Hybrid |
| `high-confidence-debugging` | Reproduce, repair, falsify | DeepSeek V4 Pro via Go | Sonnet 5 reproducer, Sol verifier | Agent MCP |
| `research-backed-engineering` | Current-source implementation | MiniMax M3 via Go | Luna researchers, Sonnet 5 source auditor | Agent MCP |
| `secure-change` | Security-sensitive change | Opus 5 via Zen | GLM-5.3 Coding Plan hunter, DeepSeek verifier | Agent MCP |
| `visual-engineering` | Reference-driven UI/browser work | GPT-5.6 Sol via ChatGPT | Terra analyst, fresh multimodal Sol verifier | Hybrid |
| `contract-first-refactoring` | Compatibility-sensitive refactor | Sonnet 5 via Zen | MiniMax invariant miner, Luna test author, DeepSeek verifier | Agent MCP |
| `incident-hypothesis-triage` | Read-only causal incident analysis | GLM-5.3 via Z.AI Coding Plan | DeepSeek Flash runner, pinned Nemotron challenger | Agent MCP |

### Experimental labs

| Profile | Hypothesis |
|---|---|
| `access-efficient-escalation-lab` | Contained local/lower-cost work can reduce scarce-tier use without unacceptable quality loss |
| `route-resilience-lab` | Matched DeepSeek work through Go and pinned OpenRouter exposes route drift and failure behavior |
| `bounded-research-organization-lab` | A depth-two research hierarchy can outperform the same flat fan-out on long corpora |
| `competing-implementations-lab` | Two isolated implementations selected by tests can beat one after total cost and integration effort |

Labs report matched evidence but make no superiority claim. See [Bundled profiles](docs/PROFILES.md) and [coverage](docs/PLAN-COVERAGE.md).

## Catalog

The reviewed baseline, current through 2026-08-19, contains 21 routes, 548 models, 452 agent-compatible models, and 9 resource groups across Codex/ChatGPT, OpenCode Go, OpenCode Zen, OpenRouter, Z.AI General API, Z.AI Coding Plan, and local routes.

Keys are route-qualified:

```text
codex_chatgpt_builtin__gpt_5_6_sol
opencode_go_openai_chat__deepseek_v4_pro
opencode_zen_anthropic_messages__claude_opus_5
zai_coding_responses__glm_5_3
openrouter_openai_chat__nvidia_nemotron_3_ultra_550b_a55b
```

Bundled GLM-5.3 roles use only the Z.AI Coding Plan Responses route and `ZAI_CODING_API_KEY`. The separate Z.AI General API remains available to users who have it, but is never an implicit fallback.

Every external inventory, from three Coding Plan records to 413 OpenRouter records, follows the same reviewed snapshot/fingerprint/generation pipeline. Live discovery reports drift; it never invents capabilities for a new ID.

```bash
codex-mmo catalog summary
codex-mmo catalog routes
codex-mmo catalog model zai_coding_responses__glm_5_3
codex-mmo catalog inventory
codex-mmo catalog verify --remote --codex
```

See [Route and model catalog](docs/CATALOG.md).

## Installation

Requirements:

- Linux, Bash, Git, and Python 3.11+;
- Codex CLI exactly 0.149.0 with the pinned experimental app-server schema;
- Switchyard 0.2.0 for translated/external routes;
- Rust/Cargo 1.96.1+ only when asking the installer to build Switchyard;
- credentials only for routes you intend to use.

```bash
./install.sh

# Or install missing Codex/Switchyard executables too:
./install.sh --install-codex --install-switchyard
```

Configure credentials and ChatGPT-backed Codex authentication:

```bash
${EDITOR:-vi} ~/.config/codex-mmo/credentials.env
codex-mmo auth login
codex-mmo auth status
```

Validate, then run:

```bash
codex-mmo validate --all-profiles
codex-mmo doctor --profile adaptive-engineering --live

cd /path/to/repository
codex-mmo
```

See [Installation](docs/INSTALLATION.md). Run `codex-mmo --help` for the complete command map;
the [CLI guide](docs/CLI.md) documents output, automation, exit-status, input, and precedence
contracts.

## Command-line behavior

Structured commands render human-readable tables or labeled details when stdout is a terminal and
strict JSON when stdout is piped or redirected. `--json` forces JSON; `--quiet` suppresses only
interactive stage progress; `--debug` adds exception diagnostics. Requested data goes to stdout,
while progress, warnings, usage guidance, and errors go to stderr. Global flags may appear before or
after a control command as long as they precede the explicit `--` separator used for Codex-owned
arguments.

`codex-mmo exec` accepts a positional prompt or non-empty piped stdin and fails immediately rather
than reading from an interactive terminal when both are absent. `codex-mmoctl` exposes the same
control tree without the primary command's implicit interactive `run`. See the [CLI guide](docs/CLI.md)
for shell examples, machine-error shape, lifecycle vocabulary, and exit statuses.

## Profiles and runs

```bash
codex-mmo profile list
codex-mmo profile show adaptive-engineering --resolved
codex-mmo profile validate adaptive-engineering
codex-mmo profile compile adaptive-engineering
codex-mmo profile use high-confidence-debugging

codex-mmo run --profile adaptive-engineering
codex-mmo run --profile visual-engineering -- --image screenshot.png
codex-mmo resume --last
codex-mmo resume SESSION_ID
codex-mmo exec --profile incident-hypothesis-triage "Analyze the supplied incident evidence"
```

`codex-mmo` starts one persistent Unix app-server session and attaches the stock Codex TUI as a remote client when no control command is supplied. `exec` uses the same host/thread model without a TUI. `resume` reattaches to the exact active immutable session, snapshot, working directory, generated `CODEX_HOME`, and current root-thread generation; any predecessor thread ID in that session's lineage resolves to the current generation. It reuses the live app-server host when its generated routing configuration remains compatible, starts a replacement when that host is gone, and deliberately recycles it when the restored gateway endpoint or compiled route availability changes. A terminal session cannot be resumed; start a deliberately new session instead. `codex-mmoctl` invokes the same control plane without implicit interactive behavior. Put `--` before Codex-owned options such as `--image` or `--search`.

When a fresh top-level context is observed while a fingerprinted stock TUI is attached and the prior root has no active turn, MMO stages and commits a new root-thread generation without creating another MMO session or run. Native child/fork threads are never eligible. If the controller loses the notification, recovery consults only the pinned app-server's session-isolated thread index, records the recovered transition, and resumes the latest canonical generation; it never searches rollout files or unrelated Codex history. A fresh interactive goal is restored before new work is treated as active.

`exec --wall-timeout SECONDS` is an external caller/harness limit, not an agent budget. Reaching it detaches the client while the root host and goal continue, preserving the exact thread and partial evidence for `resume`. A successful root turn may also return the session as detached when recoverable descendants remain active or suspended; inspect, continue, finalize, or fully stop those workers instead of silently abandoning them. Goal-mode profile limits use Codex's token accounting, not a model-authored or supervisor-guessed clock.

Codex app-server accepts at most 4,000 characters in a goal objective. MMO treats that field as a lifecycle label, not the task transport: the complete prompt remains unchanged in the initial turn, while a long goal label is a deterministic prefix plus a SHA-256 correlation note. Interactive goal sessions are seeded paused before the TUI attaches and become active only after the first accepted turn.

Profile-local rebinding remains exact and capability-checked:

```bash
codex-mmo run \
  --profile adaptive-engineering \
  --bind implementation_specialist=opencode_go_openai_chat__deepseek_v4_pro
```

User profiles live under `~/.config/codex-mmo/profiles.d/`. Packs are static TOML, Markdown, and JSON only; executable code, symlinks, credentials, and Tool MCP endpoint definitions are rejected.

## Tool MCP versus Agent MCP

`mmo_mesh` is the runtime-owned Agent MCP server for delegation. Third-party Tool MCP servers—Firecrawl, IDA, browser tools, or organization-specific services—are a separate operator boundary:

```text
~/.config/codex-mmo/tool-mcp.d/*.toml
```

Profiles reference stable server/tool IDs and can only narrow the operator’s finite allowlist and approval ceiling. A read-only agent role does not make an external MCP tool read-only in its own system.

```bash
codex-mmo tool-mcp list
codex-mmo tool-mcp show firecrawl
codex-mmo tool-mcp validate
codex-mmo profile doctor adaptive-engineering
```

See [Tool MCP](docs/TOOL_MCP.md).

## Sessions and jobs

```bash
codex-mmo session list
codex-mmo session show SESSION_ID
codex-mmo session runs SESSION_ID
codex-mmo session detach SESSION_ID
codex-mmo session pause SESSION_ID
codex-mmo session compact SESSION_ID
codex-mmo session continue SESSION_ID --input "Proceed from the retained evidence"
codex-mmo session continue SESSION_ID --goal-token-budget 800000
codex-mmo session stop SESSION_ID --grace 300
codex-mmo session cancel SESSION_ID

codex-mmo jobs list --session SESSION_ID
codex-mmo jobs list --run RUN_ID
codex-mmo jobs status JOB_ID
codex-mmo jobs result JOB_ID
codex-mmo jobs result JOB_ID --cursor NEXT_CURSOR
codex-mmo jobs wait --session SESSION_ID JOB_ID
codex-mmo jobs cancel JOB_ID --reason "superseded"

codex-mmo gateway list
codex-mmo gateway logs PROFILE
codex-mmo gateway stop-idle
```

One logical session contains exactly one immutable run and one current root-thread generation. `detach` disconnects a client while root and workers continue. `pause` durably checkpoints partial evidence, pauses/interrupts admitted turns, retires root and supervised-worker hosts, and releases their execution capacity without changing thread identity. `compact` hosts the paused root just long enough to invoke app-server compaction and then returns the session to a cold pause. `continue` re-admits and reactivates the same root generation plus the exact paused worker set, and may extend a goal only within its compiled token ceiling. `stop` asks active work to finalize from retained evidence, waits the requested operator grace, then retires every host in the run. `cancel` is immediate. A controller, provider, or transport failure produces typed recoverable state on the persisted lineage; it does not erase events, messages, tool observations, terminal history, partial results, or a bounded writable patch.

`session_kind` records whether the first client was interactive or noninteractive; it does not select an execution backend. `root_execution_host` is always `app_server`. An interactive TUI attaches with Codex `--remote` to the existing Unix socket and never becomes the owner of thread or worker lifetime.

Terminal sessions and jobs are never revived. Active admission and goal-token ceilings belong to the one immutable run; terminal jobs release their simultaneous-capacity leases, and all jobs remain inspectable after the run ends. The root performs accept/reject/integrate transitions for successfully completed results through its authenticated Agent-MCP tools. Stopped, failed, and cancelled jobs remain readable but cannot be dispositioned as successes. Reading a CLI job result for inspection does not accept it or apply its patch. When `next_cursor` is non-null, request that exact cursor and concatenate text pages in order; no head/tail elision occurs.

## Smoke tests and evaluation

```bash
codex-mmo profile smoke adaptive-engineering --cwd "$PWD"
codex-mmo profile smoke secure-change --workers-only --cwd "$PWD"

codex-mmo eval suites
codex-mmo eval validate adaptive-change
codex-mmo eval run --profile adaptive-engineering --suite adaptive-change --dry-run
codex-mmo eval run --profile adaptive-engineering --suite adaptive-change
codex-mmo eval run --profile adaptive-engineering --suite adaptive-change --trial-mode release
codex-mmo eval compare RUN_ID_A RUN_ID_B
```

Every suite uses matched controls and worker ablations. Development mode runs three trials per task; release mode runs five. Evaluations record outcomes, explicit waits, root activity events, concurrency, result disposition, integration, route/serving identity, and separate cost ledgers. They publish no aggregate score.

Bundled status is not evidence that every composition has passed live promotion. See [Evaluation](docs/EVALUATION.md).

## Development and release checks

```bash
make test
make validate
make release
```

Release validation checks the single MMO schema-8/package-8.0.0 generation; catalog reproducibility; all profiles/suites; the exact Codex 0.149.0 protocol pin and offline app-server behavior; package safety; deterministic archives; and integration tests. `doctor --live` separately proves that the installed binary generates the pinned 401-file upstream app-server schema digest. Authenticated provider availability and five-trial efficacy remain separate live acceptance evidence.

## Documentation

- [Architecture](docs/ARCHITECTURE.md)
- [Command-line interface](docs/CLI.md)
- [Profile schema](docs/PROFILE_SCHEMA.md)
- [Bundled profiles](docs/PROFILES.md)
- [Orchestration backends](docs/ORCHESTRATION_BACKENDS.md)
- [Route and model catalog](docs/CATALOG.md)
- [Provider/route drivers](docs/PROVIDER_DRIVERS.md)
- [Tool MCP](docs/TOOL_MCP.md)
- [Security model](docs/SECURITY.md)
- [Evaluation](docs/EVALUATION.md)
- [Release acceptance](docs/ACCEPTANCE.md)
- [Troubleshooting](docs/TROUBLESHOOTING.md)

## Scope

Codex MMO is an orchestration and enforcement runtime, not a workflow programming language. Profiles use agents, exact bindings, directed edges, bounded nesting, budgets, resources, permissions, scopes, contracts, trust rules, and instructions. They do not embed Python, arbitrary loops, expression DSLs, or scheduler plugins.

## License

See [LICENSE](LICENSE).