😏
This commit is contained in:
+389
@@ -0,0 +1,389 @@
|
||||
# Changelog
|
||||
|
||||
## 2.4.0
|
||||
|
||||
- Stabilized the pre-release 2.4.0 implementation and qualification evidence in
|
||||
place. These review corrections do not add a Soma environment variable or package
|
||||
dependency, and 2.3.0 remains the unchanged rollback. Evaluator-only provenance
|
||||
arguments and llama.cpp server flags are not runtime configuration additions.
|
||||
- Replaced isolated latest-turn rewriting with a bounded full-task context. Transform
|
||||
calls can receive the original role-preserving `system`, `developer`, `user`,
|
||||
`assistant`, and `tool` messages; tool results and definitions; `tool_choice`;
|
||||
`parallel_tool_calls`; response-format, modality, audio, and stop controls; the
|
||||
immutable target tool calls; and a clearly separated failed assistant draft.
|
||||
Model and sampling knobs, transport headers, endpoint credentials, and rejected
|
||||
transform candidates remain excluded. System/developer instructions remain
|
||||
authoritative below Soma's JSON and native-tool invariants.
|
||||
- Replaced independent whole-field regeneration with one fixed joint-repair object
|
||||
containing required nullable `reasoning` and `content` members. Soma classifies all
|
||||
present fields before repair and requests values only for fields marked for repair,
|
||||
allowing accepted reasoning to ground content and jointly generated reasoning to
|
||||
precede jointly generated content.
|
||||
- Isolated each classification envelope to the named draft field while retaining the
|
||||
complete original request, controls, and immutable calls. Repair generation may
|
||||
inspect fields it must replace, but integrity verification sees only retained draft
|
||||
fields and the current candidate, preventing rejected sibling text from influencing
|
||||
either decision.
|
||||
- Specialized the existing repair schema per call so requested members are strings
|
||||
and every other required member is null. Added local JSON syntax/top-level checks
|
||||
and literal stop-sequence checks for both target and repaired output, a fixed
|
||||
alternate focus for candidate three, and bounded privacy-safe candidate rejection
|
||||
reasons. Exact nonblank punctuation, symbol, and empty-container outputs are not
|
||||
rejected by an English-text heuristic. These corrections add no JSON-schema
|
||||
dependency or temperature branch.
|
||||
- Added explicit field outcomes. A rejected optional reasoning field is dropped when
|
||||
usable content remains; a reasoning-only response without content or a native tool
|
||||
call is not accepted as terminal success. Immutable native `tool_calls` survive
|
||||
exhausted adjacent-prose repair, with only the unusable prose cleared. Returned
|
||||
function calls must match supplied definitions, and multiple calls are rejected
|
||||
when the request sets `parallel_tool_calls=false`.
|
||||
- Defined the normal semantic route as primary reasoning-off classification,
|
||||
candidate one, and integrity verification, followed—after a completed semantic
|
||||
rejection—by up to two fresh secondary reasoning-on candidates from the original
|
||||
context. Each secondary candidate is verified on secondary. Without a secondary,
|
||||
recovery is limited to two primary candidates. Availability failover remains
|
||||
request-local. Configuration now enforces primary `off` and secondary `on` so the
|
||||
deployed route cannot silently differ from the route the evaluator qualifies.
|
||||
- Added a reasoning-off verifier fallback for a reasoning-on verification result that
|
||||
is truncated or structurally invalid. The identical candidate is checked once on
|
||||
the same endpoint/model with thinking disabled; Soma regenerates only after a
|
||||
completed verifier returns `rewrite`. Semantic repairs never receive a rejected
|
||||
candidate or prior-failure prose; structural JSON recovery receives only a closed
|
||||
reason. Any `finish_reason=length` result is treated as truncated even when its
|
||||
partial text happens to be syntactically valid JSON.
|
||||
- Added per-endpoint `TRANSFORM_MEDIA_MODE=placeholder|forward|reject` and
|
||||
`TRANSFORM_SECONDARY_MEDIA_MODE`. Placeholder mode preserves typed part positions
|
||||
and metadata while omitting binary payload; forward mode sends the original media
|
||||
as native OpenAI multimodal parts; reject mode requires a compatible alternate
|
||||
route or fails explicitly. A forward-capability rejection is never silently
|
||||
downgraded to placeholder mode. Malformed scalar content parts are rejected before
|
||||
the target call, and streamed assistant-audio data/transcripts are accumulated
|
||||
rather than overwritten. Mixed text/tool plus audio turns preserve their audio;
|
||||
audio-only target turns fail explicitly because the text repair contract cannot
|
||||
inspect or repair that payload.
|
||||
- Added `TRANSFORM_ALLOW_CLARIFICATION`, default `false`. Missing task inputs no
|
||||
longer license arbitrary examples, placeholders, invented results, or generic
|
||||
promises; a clarification is an acceptable repair only when the operator opts in.
|
||||
- Added the opt-in `TARGET_RETRY_ON_UNREPAIRABLE`, default `false`. When the complete
|
||||
target turn leaves neither usable content nor a native tool call, Soma may make
|
||||
exactly one fresh target request with a fixed corrective system message inserted
|
||||
after the leading system/developer block. It preserves the original conversation
|
||||
and tool contract and never includes rejected target or transform output. Optional
|
||||
failed reasoning does not trigger this retry when content remains usable.
|
||||
- Added the opt-in `TARGET_LOOP_BACK_ON_VERIFIED_REPAIR`, default `false`. When an
|
||||
integrity-verified repair rewrites the target's refusal reasoning, Soma may make
|
||||
exactly one additional target request consisting of the original conversation plus
|
||||
a single assistant message carrying only that verified repaired reasoning in a
|
||||
`reasoning_content` field, so the target re-ingests the relaxed context and
|
||||
produces the task output itself. The second response is processed under the same
|
||||
bounded rules and never loops again; genuine refusals never loop because they
|
||||
classify as `pass`. Loop-back and `TARGET_RETRY_ON_UNREPAIRABLE` remain mutually
|
||||
exclusive per request under the shared two-target-call ceiling and aggregate
|
||||
transform deadline. Backends that reject `reasoning_content` in input messages
|
||||
fail explicitly.
|
||||
- Raised the defaults to `TRANSFORM_CONTEXT_MAX_CHARS=131072`,
|
||||
`TRANSFORM_FIELD_MAX_CHARS=32768`, `TRANSFORM_DECISION_MAX_TOKENS=1536`,
|
||||
`TRANSFORM_REWRITE_MAX_TOKENS=16384`, and
|
||||
`TRANSFORM_TOTAL_TIMEOUT=1200`. Semantic context is never silently truncated; the
|
||||
configured context limit is capped at 4000000 characters and the field limit must
|
||||
not exceed it. The limit is enforced after each phase removes discarded draft
|
||||
fields, so oversized optional reasoning cannot block an otherwise bounded content
|
||||
repair.
|
||||
- Removed `TRANSFORM_CONFIRM_REWRITES`. Any presence of the obsolete variable now
|
||||
fails configuration validation so a stale profile cannot appear to enable behavior
|
||||
that no longer exists.
|
||||
- Set hard ceilings of two target calls, 20 transform calls per target response, and
|
||||
40 transform calls for one client request. Default configuration still makes one
|
||||
target call because target retry is disabled. Fail-open no longer permits a
|
||||
reasoning-only or otherwise empty terminal response.
|
||||
- Expanded privacy-safe diagnostics, offline coverage, and live qualification around
|
||||
full context, media routes, joint field policy, semantic candidate routing,
|
||||
verifier fallback, and optional target retry. The frozen evaluator now covers 240
|
||||
classification cases, 80 balanced joint message repairs, all three media modes,
|
||||
staged primary-off/secondary-on routing, retained/forbidden facts, schema-off and
|
||||
repeat sentinels, and reproducibility. Live target smoke remains explicitly opt-in.
|
||||
Soma 2.4.0 ships unqualified until the exact transform profile passes every
|
||||
automated gate; the report retains hash-bound evidence for all 80 repairs, and the
|
||||
evaluator never downloads, loads, switches, starts, stops, or restarts a model.
|
||||
Error responses after dispatch now carry the same privacy-safe counters as success
|
||||
responses, target elapsed time includes malformed target responses, and exact
|
||||
qualification records a dedicated secondary reasoning-budget value.
|
||||
- The live evaluator can record llama.cpp's unrestricted reasoning-budget sentinel
|
||||
as `--reasoning-budget -1` for an honest exploratory run. This never satisfies the
|
||||
positive bounded-budget qualification gate; it only prevents current deployments
|
||||
from being mislabeled as budget `0` or `512` while collecting repair evidence.
|
||||
- Defined the first qualifying local route as temperature zero with a bounded
|
||||
llama.cpp `--reasoning-budget 512`, recorded by the evaluator with the matching
|
||||
`--reasoning-budget 512`. Temperature zero removes intentional sampling variance
|
||||
from repeat gates but is not a guarantee of byte-identical output; other
|
||||
temperatures and budgets remain separate profiles that need their own reports.
|
||||
- Made the automated gates the sole qualification stage. A complete pass records
|
||||
`qualified: true` and `qualification_status: qualified`; a failed gate records an
|
||||
unqualified profile. Hash-bound retained outputs remain audit evidence and do not
|
||||
form a separate approval stage.
|
||||
- Added an explicit provider-managed evaluator profile for hosted-model experiments.
|
||||
It records a retained, hash-bound public model metadata record and non-secret
|
||||
endpoint/model identities, rejects fabricated local GGUF or reasoning-budget
|
||||
provenance, and is always marked exploratory rather than qualified. Target smoke
|
||||
now requires the exact requested `OK` token instead of accepting arbitrary
|
||||
non-empty content.
|
||||
- Added an explicitly unqualified hybrid evaluator profile for a local GGUF primary
|
||||
and provider-managed secondary. It retains both provenance records without
|
||||
pretending the hosted reasoning budget is known. Target-smoke requests now allow
|
||||
128 output tokens while still requiring the response to be exactly `OK`.
|
||||
- Kept the unrestricted-budget Qwen3.5-9B Q6_K report unqualified until a complete
|
||||
replacement run exists. In-place evaluator stabilization changes its source/corpus
|
||||
evidence identity, so the evaluator atomically installs a fresh temperature-zero,
|
||||
512-token report before the obsolete unrestricted artifact is removed; the old JSON
|
||||
is never edited or promoted.
|
||||
|
||||
## 2.3.0
|
||||
|
||||
- Replaced exact excerpt extraction, Unicode-equivalence recovery, unique-anchor
|
||||
resolution, bounded fragment generation, masked retries, and local splicing with a
|
||||
decision-only `pass | rewrite` classifier and complete-field regeneration. The
|
||||
simpler contract removes the highest-fragility requirements for small transform
|
||||
models while giving every model enough user context to produce a substantive
|
||||
answer rather than a generic bridge.
|
||||
- Added mandatory post-rewrite classification. Every locally valid candidate must
|
||||
independently classify as `pass`; a residual or paraphrased refusal,
|
||||
nonresponsive bridge, malformed verification result, or verification failure
|
||||
rejects the candidate.
|
||||
- Added one bounded fresh-regeneration path from the pristine source after a
|
||||
received-output or post-check failure. A configured secondary performs the second
|
||||
candidate when available; otherwise primary is reused. Rejected candidate text is
|
||||
never fed back to either model, and the final candidate is verified exactly once.
|
||||
- Added `TRANSFORM_REWRITE_MAX_TOKENS`, defaulting to 4096 with a valid range of
|
||||
256–16384. Complete replacements are bounded after decoding to
|
||||
`min(16,384, max(4,096, 2 * source characters))`; the rewrite schema deliberately
|
||||
omits a dynamic `maxLength` for structured-output compatibility.
|
||||
- Added explicit 16,384-character rewrite-input limits without semantic truncation.
|
||||
Missing textual user context and oversized source/user inputs are deterministic,
|
||||
field-scoped failures. Mixed-media turns carry an `unseen_media` flag; pure
|
||||
non-text turns cannot be regenerated.
|
||||
- Limited transform context to the complete field, latest textual user turn,
|
||||
`tool_turn`, `unseen_media`, the deterministic output bound, and an optional closed
|
||||
retry reason. Earlier messages, system instructions, processed reasoning, tools and
|
||||
arguments, headers, credentials, and rejected candidates remain excluded.
|
||||
- Raised the per-completion transform-call ceiling from eight to twelve to cover two
|
||||
complete `(regeneration + post-check)` candidate paths for each reasoning/content
|
||||
field while retaining at most two classification votes, the aggregate transform
|
||||
deadline, one target call, and request-local backend stickiness.
|
||||
- Removed the DSML parser, shim, counters, diagnostics, and documentation. Soma now
|
||||
preserves native OpenAI `tool_calls` only; proprietary text-to-tool conversion is
|
||||
owned by the target's OpenAI-compatible gateway. Native calls now require a
|
||||
nonblank unique ID and `type: function`, preventing stream/non-stream envelope
|
||||
drift and uncorrelatable tool results.
|
||||
- Tightened the post-`[DONE]` SSE extension to at most one object with `choices: []`,
|
||||
no keys outside `choices`, `cost`, and `usage`, and at least one non-null metadata
|
||||
value. The postlude may end at EOF or one closing `[DONE]`; all later data is
|
||||
rejected, and upstream error objects retain the redacted stream-error path.
|
||||
Normalized outbound SSE emits one accepted `cost`/`usage` metadata event instead
|
||||
of repeating one-shot cost data on every delta.
|
||||
- Added `X-Soma-Postcheck-Rejections` and `classify`, `rewrite`, and `postcheck`
|
||||
phase diagnostics with candidate/backend/reasoning/purpose metadata. Removed
|
||||
rewrite-mode and DSML diagnostics while retaining strict privacy-safe logging,
|
||||
field decisions, repair counts, rejected-rewrite counts, and fail-open visibility.
|
||||
- Reworked live qualification around the exact imported 2.3.0 runtime, decision-only
|
||||
classification, complete-field replacements, and verified final output. Local
|
||||
reports belong under excluded `qualification-local/`; Soma never downloads,
|
||||
starts, stops, switches, or otherwise manages a model.
|
||||
- Distinguished automated-gate success from final qualification: reports remain
|
||||
`pending_manual_review` until the 40 retained source/replacement pairs receive
|
||||
independent semantic review. Exact-pair runs with a configured secondary inject
|
||||
one labeled primary-unavailable event and require real secondary classification,
|
||||
regeneration, and post-check calls; reports include per-backend phase counts.
|
||||
- Made live provenance fail closed if either the imported Soma source or evaluator
|
||||
changes during a run, preventing a report from hashing different code than it
|
||||
actually exercised.
|
||||
|
||||
## 2.2.7
|
||||
|
||||
- Fixed the 2.2.6 streaming regression that rejected OpenAI-compatible gateway
|
||||
usage or billing postludes emitted after an upstream `[DONE]`. Empty-choice
|
||||
metadata is now preserved, including an optional final `[DONE]`, while premature
|
||||
delimiters, malformed postludes, and any post-delimiter choice data remain invalid.
|
||||
- Added explicit `off`, `on`, and `default` transform reasoning modes. `off`
|
||||
preserves the 2.2.6 request shape, `on` uses llama.cpp's
|
||||
`chat_template_kwargs.enable_thinking=true`, and `default` leaves reasoning
|
||||
controls to the server. The setting applies consistently to classification and
|
||||
rewriting.
|
||||
- Added one optional, fully explicit secondary transform endpoint/model profile. It
|
||||
may point to the primary server with a different reasoning mode or to an
|
||||
independently operated server/model, but it shares Soma's prompt, temperature,
|
||||
JSON contract, validation, and fixed generation bounds.
|
||||
- Kept recovery bounded to the existing second attempt. The secondary can replace a
|
||||
failed primary attempt after connection/timeouts, HTTP 408/429/5xx, invalid outer
|
||||
completions, or locally rejected classification/rewrite output. Configuration
|
||||
errors and other non-transient 4xx responses remain visible.
|
||||
- Added optional secondary confirmation for valid primary `rewrite` classifications.
|
||||
A secondary `pass` conservatively preserves the field; two valid `rewrite`
|
||||
decisions retain the primary anchor. Confirmation is disabled by default and does
|
||||
not add a third classification attempt.
|
||||
- Added request-local failover: after a transient primary availability failure, only
|
||||
the remaining transform phases in that Soma request begin on secondary. The next
|
||||
client request starts on primary, so Soma retains no cross-request circuit-breaker
|
||||
or harness/session state.
|
||||
- Added a 600-second aggregate transform budget, starting after the single target
|
||||
call, so retries and multi-field processing cannot multiply the per-call timeout
|
||||
without bound. The one-target-call invariant and eight-transform-call ceiling are
|
||||
unchanged.
|
||||
- Extended safe diagnostics with backend, reasoning mode, call purpose, secondary
|
||||
call/failover/disagreement counts, and transform-deadline failures without logging
|
||||
prompts, generated text, anchors, credentials, or raw upstream request IDs.
|
||||
- Extended live qualification to exercise explicit reasoning modes and composed
|
||||
primary/secondary behavior without managing models. Deployment guidance requires
|
||||
a secondary to qualify alone and the exact pair to qualify before an operator
|
||||
enables confirmation; the evaluator does not change runtime configuration.
|
||||
- An exploratory test of the operator-selected Qwen3.5-9B Q6_K with reasoning enabled
|
||||
at temperature 0 removed the two reasoning-off false rewrites but regressed overall
|
||||
refusal recall from 99% to 96%, introduced four invalid/missed partial-refusal
|
||||
anchors and ten repeat failures, and remained unqualified. Targeted 64/128/256-token
|
||||
reasoning budgets were non-monotonic, so Soma adds no model-specific
|
||||
reasoning-budget knob.
|
||||
|
||||
## 2.2.6
|
||||
|
||||
- Preserved the 2.2.5 one-target-call flow, phase prompts, bounded transform call
|
||||
counts, anchored/masked rewrite behavior, DSML normalization, `FAIL_OPEN` policy,
|
||||
endpoints, and runtime environment variables while simplifying the implementation.
|
||||
- Removed unused transform compatibility helpers, prefiltered protected
|
||||
`ENABLE_REASONING` fields before overlay, reduced target-response mutation to one
|
||||
working deep copy, removed custom HTTP pool sizing, and consolidated rewrite
|
||||
contract validation under its owning parser/semantic validator.
|
||||
- Made client flag validation exact: `stream` and `parallel_tool_calls` must be
|
||||
booleans, while `n` is null or integer `1` and no longer accepts a boolean.
|
||||
- Added target assistant-message validation before DSML or transform calls, including
|
||||
role/content types, one non-empty reasoning alias, native tool-call shape,
|
||||
non-blank function names, JSON-string arguments, and unusable empty turns. Unknown
|
||||
valid target metadata remains preserved.
|
||||
- Limited latest-user transform context to text: plain strings and the textual parts
|
||||
of multimodal arrays are joined, while image, audio, file, base64, tool, and other
|
||||
non-text metadata is omitted.
|
||||
- Required refusal anchors to contain at least one letter or number and begin and end
|
||||
on source boundaries; rewrites must likewise contain a letter or number.
|
||||
Context-copy checks now ignore only splice-edge whitespace, closing the
|
||||
copied-neighbor bypass without changing the local exact-splice behavior.
|
||||
- Replaced generic streamed-response merging with explicit OpenAI delta assembly.
|
||||
Text and every tool-call fragment append in arrival order, including identical
|
||||
fragments; stable metadata is checked explicitly and unknown metadata uses
|
||||
last-value semantics. Cumulative snapshot providers are explicitly unsupported.
|
||||
- Require a terminal non-null SSE `finish_reason`; reject `[DONE]` or EOF before it,
|
||||
meaningful deltas after it, malformed event JSON, and non-finite values. Normalized
|
||||
outbound SSE remains unchanged.
|
||||
- Reject non-finite configuration numbers, malformed endpoint ports, endpoint
|
||||
userinfo/query/fragment components, duplicate or framing-sensitive configured
|
||||
headers, duplicate transform-contract object members, and non-finite JSON across
|
||||
client, target, transform, SSE, and DSML boundaries. Wire JSON now uses a strict,
|
||||
surrogate-safe encoder.
|
||||
- Tightened HTTP request framing to reject simultaneous `Content-Length` and
|
||||
`Transfer-Encoding`, non-sole chunked encodings, negative chunks, and unterminated
|
||||
chunk framing. No in-process body-size or buffering cap was added; those limits
|
||||
remain the responsibility of a trusted front proxy.
|
||||
- Preserved leading and trailing whitespace for DSML `string=true` arguments while
|
||||
continuing to trim non-string JSON arguments. Sanitized upstream request IDs before
|
||||
reflection and prevented upstream headers from colliding with Soma diagnostics.
|
||||
- Reduced client-facing detail for streamed target/internal failures while retaining
|
||||
server-side diagnostics, and removed query strings and upstream error text from
|
||||
routine proxy-error logs.
|
||||
- Added an opt-in standard-library live qualification evaluator with 200 reviewed
|
||||
classifier fixtures, 40 category-balanced rewrite fixtures, schema-off sentinels,
|
||||
deterministic parallel/repeat gates, reproducibility metadata, synthetic rewrite
|
||||
output for human audit, and an optional 50-case one-shot target gate. Its reviewed
|
||||
anchors must cover the curated refusal core, its JSON/channel selection mirrors the
|
||||
runtime, and reports fingerprint the exact prompt/source/configuration snapshot.
|
||||
It never downloads, switches, loads, or restarts a model.
|
||||
- Recorded the tested aggressive Qwen3.5-9B Q8_0 deployment as unqualified after
|
||||
false-positive and rewrite-stability failures. The official Qwen3.5-9B Q8_0 remains
|
||||
a candidate that must independently pass the same gate; Soma performs no model
|
||||
management.
|
||||
- Recorded the operator-selected DavidAU Qwen3.5-9B Q6_K artifact as reproducibly
|
||||
unqualified at temperature 0, seed 1, the temperature sampler, and one server slot;
|
||||
preserved its exact machine-readable qualification report with the release. A
|
||||
per-request temperature-0.7 comparison was also preserved and was substantially
|
||||
worse, with 43 false rewrites and 93% refusal recall. Temperature 0 remains the
|
||||
better tested setting, but neither configuration qualifies.
|
||||
- Added two copyable deployment profiles using only existing variables, with one Soma
|
||||
process and distinct listener port per harness. Documented isolated profile startup,
|
||||
the trusted-local/reverse-proxy boundary, front-proxy resource limits, and the
|
||||
non-loopback warning. No runtime harness branching was introduced.
|
||||
|
||||
## 2.2.5
|
||||
|
||||
- Fixed the observed 2.2.4 semantic-echo failure in which classification recovered to a valid literal anchor, but both rewrite attempts returned complete, schema-valid, `finish_reason=stop` JSON whose replacement was unchanged from that anchor; this was not a JSON-mode, truncation, or token-budget failure.
|
||||
- Split the built-in classification and rewrite guidance so classification remains observational while the editor is explicitly permitted to produce a bounded non-refusal fragment without independently solving the user request.
|
||||
- Strengthened classification recovery guidance so a corrected excerpt must remain a unique, exact, syntactically replaceable refusal clause containing the withholding decision rather than merely any literal source text.
|
||||
- Kept the initial rewrite and ordinary repairs anchored, but masked `refusal_anchor` from the sole second attempt after an `unchanged` or `anchor_retained` result. The masked request carries only the field, bounded contexts, `anchor_omitted: true`, and the closed failure reason; the exact anchor and offsets remain local for validation and splicing.
|
||||
- Added field-specific masked guidance that relies on the local splice to preserve both contexts and forbids copying, paraphrasing, summarizing, or otherwise reproducing them in the replacement.
|
||||
- Made the field fallback instruction unconditional on every `masked_anchor` retry after a live Qwen run ignored a buried conditional fallback and copied immutable context. The final system instruction, appended after the masked contract, directs the transform to return exactly `{"replacement":"Proceed with the requested work."}` for reasoning or `{"replacement":"I'll proceed with the requested task."}` for content. This gives the semantic retry a fixed, unambiguous target while preserving normal server-output parsing, local validation, candidate-exhaustion, and `FAIL_OPEN` behavior; the shared schema and validator still accept any otherwise valid replacement fragment.
|
||||
- Added privacy-safe `rewrite_mode=anchored|masked_anchor` diagnostics without logging the anchor, replacement, target text, user context, or rejected output; non-rewrite phases use `none`.
|
||||
- Preserved the 64-character anchor cap, 512-character contexts, 256-character replacement cap, 384-token rewrite budget, two rewrite attempts, one target call, and eight-transform-call ceiling. Rewriting still receives no latest user turn, unbounded target field, or processed reasoning, and still has no complete-field fallback.
|
||||
- Preserved `FAIL_OPEN` behavior: strict mode returns the exhausted transform error, while fail-open mode preserves only the affected field, marks it `failed_open`, and continues.
|
||||
|
||||
## 2.2.4
|
||||
|
||||
- Fixed the observed 2.2.3 failure in which classification JSON was valid, an initial complete-field rewrite retained the refusal, and its repair exhausted the token budget before closing its JSON object; JSON mode was enabled but could not guarantee completion of the unbounded complete-field edit.
|
||||
- Replaced complete-field refusal rewriting with a deterministic local splice: classification must resolve a unique source anchor of at most 64 decoded characters, and the transform receives only that anchor plus at most 512 source characters on each side.
|
||||
- Require the rewrite model to return only an anchor replacement of at most 256 decoded characters, with the fixed cap expressed in the llama.cpp-compatible JSON schema, repeated in the prompt, and enforced authoritatively in Python.
|
||||
- Reject and repair complete-field candidates that repeat immutable context across a splice boundary or embed a substantial boundary segment inside wrapper text.
|
||||
- Bound rewrite generation to 384 tokens and preserve the original prefix and suffix exactly; only the bounded source window is sent, never the latest user turn or processed reasoning, and there is no complete-field rewrite fallback.
|
||||
- Apply one failure-specific repair to any received but unusable rewrite candidate, without echoing candidate text or exception details into the retry.
|
||||
- Make exhausted rewrite candidates consistently honor `FAIL_OPEN`: strict mode returns 502 with the specific transform failure, while fail-open mode preserves only the affected field and marks it `failed_open`.
|
||||
- Retained all endpoints, environment variables, response headers, client formats, the one-target-call invariant, and the eight-transform-call maximum.
|
||||
- Documented that fixed small `maxLength` constraints work with the tested llama.cpp structured-output path, while previously considered large dynamic limits did not; because the schema is not model-visible, prompts also state each cap and local validation remains authoritative.
|
||||
|
||||
## 2.2.3
|
||||
|
||||
- Reduced classification excerpts from 128 to 64 decoded characters and directed the classifier to choose a compact, normally 2–8-word refusal anchor.
|
||||
- Required exact contiguous provenance after JSON decoding, preserving case, Unicode punctuation, CRLF and other line endings, tabs, repeated spaces, and non-breaking spaces.
|
||||
- Resolve only unique CRLF, no-break-space, or curly-quote normalization matches back to the exact original source slice with a streaming, memory-bounded matcher before rewriting; ambiguous matches, lone-CR changes, case changes, omissions, and general whitespace changes remain invalid.
|
||||
- Clarified that llama.cpp JSON grammar constrains structure and string length but cannot enforce the dynamic rule that an excerpt occurs in the supplied source text; Soma continues to enforce that relationship locally.
|
||||
- Gave the existing single classification recovery call failure-specific guidance using a closed failure reason, without echoing rejected transform output into the retry.
|
||||
- Added privacy-safe diagnostic failure categories for invalid JSON or contract shape, pass-with-excerpt, empty excerpt, excessive length, and nonliteral excerpts; source text, excerpts, and transform output remain excluded from logs.
|
||||
- Preserved the public `invalid_transform_output` error code, JSON-mode default and behavior when disabled, endpoints, environment variables, response headers, field-scoped `FAIL_OPEN` semantics, 768-token classification budget, and existing target and transform call bounds.
|
||||
|
||||
## 2.2.2
|
||||
|
||||
- Simplified classification to the strict `pass | rewrite` contract and removed the `safety` outcome.
|
||||
- Route every refusal or withholding decision, including decisions formerly classified as safety boundaries, through the existing context-isolated rewrite stage.
|
||||
- Continue using the latest user turn only for classification; rewriting still receives only the target field, refusal excerpt, and optional processed reasoning.
|
||||
- Removed the safety-preserved counter, response header, and field-decision value from current diagnostics.
|
||||
- Updated transform call bounds: `pass` stops field processing, while every refusal or withholding classification invokes the bounded rewrite path.
|
||||
|
||||
## 2.2.1
|
||||
|
||||
- Recover once from empty, truncated, malformed, or contract-invalid classification output before applying the configured field-scoped `FAIL_OPEN` policy.
|
||||
- Preserve transform completion metadata and report incomplete classification generation as `transform_output_truncated` instead of an undifferentiated invalid-JSON error.
|
||||
- Bound classification output to 768 tokens and require `rewrite` and `safety` excerpts to be literal, nonempty substrings no longer than 128 characters; `pass` requires an empty excerpt.
|
||||
- Added `X-Soma-Classification-Retries` and safe classification-recovery diagnostics without logging transform output or request content.
|
||||
- Treat decoded empty, missing, non-string, or structurally invalid rewrite replacements as `rewrite_rejected` while retaining operational handling for malformed JSON.
|
||||
- Require literal equality for reasoning/content duplicate routing so case- or whitespace-normalized variants cannot replace the original field.
|
||||
- Reject short and medium processed-reasoning copies in rewritten content using normalized equality, bidirectional containment, and bounded similarity checks.
|
||||
- Documented classification and rewrite call bounds and clarified that JSON mode is enabled by default, requires a process restart after environment changes, and can be verified with `--check-config`, startup diagnostics, or `/health`.
|
||||
|
||||
## 2.2.0
|
||||
|
||||
- Split transform processing into contextual classification and context-isolated rewriting.
|
||||
- Restored the exact latest user turn only for classification; user context never reaches rewriting.
|
||||
- Added `pass`, `rewrite`, and `safety` classification outcomes.
|
||||
- Preserve legitimate safety boundaries instead of treating every refusal as editable.
|
||||
- Require an exact refusal excerpt for rewrite decisions.
|
||||
- Added one bounded rewrite-only repair when an edit is unchanged or retains the classified excerpt.
|
||||
- Removed strict reasoning compression and lexical-similarity floors; retained catastrophic growth and reasoning-copy protections.
|
||||
- Process duplicate final text as content and duplicate tool-turn text as reasoning.
|
||||
- Added detected-refusal, safety-preserved, and rewrite-repair diagnostics.
|
||||
- Retained stateless operation, one target call, native tool-call fidelity, DSML normalization, buffered SSE, and field-scoped fail-open behavior.
|
||||
|
||||
## 2.1.2
|
||||
|
||||
- Rebalanced refusal handling with more sensitive classification and conservative edits.
|
||||
- Added independent field processing and `rewrite_rejected` diagnostics.
|
||||
|
||||
## 2.1.1
|
||||
|
||||
- Ignored unnecessary replacement text when `refusal:false`.
|
||||
- Simplified llama.cpp JSON grammar to avoid nested `maxLength` failures.
|
||||
|
||||
## 2.1.0
|
||||
|
||||
- Removed user context from transforms, separated reasoning/content guidance, added structured JSON, rewrite bounds, and duplicate handling.
|
||||
Reference in New Issue
Block a user