TokenPak v1.3.8

Ships the full PM/GTM v2 initiative — ten landed goals across public-surface truth (Axis A), commercial enablement (Axis B), and trust artifacts (Axis C), plus a narrow security-hardening delta. Five maintainer decisions (A purchase subdomain, B metrics deferred, C comparison + status surfaces, D security scope, E distribution model) were ratified during the initiative and applied inline. Three new blocking CI gates added (headline-benchmark, cli-docs-in-sync, bandit). SC + SC+1 conformance matrix 84/84 green throughout. Companion artifacts that shipped outside this repo — tokenpak-paid-stub==0.1.0 on PyPI, 3 compliance pages + comparison + status pages live on tokenpak.ai — are referenced per-entry below.

Added — B2 tokenpak upgrade CLI (M-B2)

Opens the canonical Pro upgrade page — https://app.tokenpak.ai/upgrade per an internal decision (2026-04-23) — in the user's default browser.

• tokenpak/cli/_impl.py — new cmd_upgrade(args) + _build_upgrade_parser(sub). Registered in _COMMAND_GROUPS["Getting Started"] and in argparse dispatch. Supports --print-url for non-interactive paths + TOKENPAK_UPGRADE_URL env override.
• docs/reference/cli.md — regenerated by scripts/generate-cli-docs.py to reflect the new command. cli-docs-in-sync gate stays green.
• tests/cli/test_upgrade_cmd.py — 4 tests: default URL matches an internal decision canonical, env override works, command registered in Getting Started, argparse dispatch resolves.

Added — D-delta bandit blocking CI gate (M-D1)

Narrow security hardening per an internal decision (2026-04-23): delta-only. Bandit added as a baseline-based blocking CI gate; CodeQL + Trivy + SBOM intentionally deferred to a post-v2 security initiative.

• pyproject.toml — bandit>=1.7 added to [dev].
• .bandit-baseline.json — snapshot of existing high-severity + high-confidence findings (6 items: 5× B602 shell=True subprocess calls in tokenpak/agent/* + tokenpak/cli/_impl.py, 1× B324 insecure-hash in tokenpak/telemetry/cache.py). Tracked in git.
• .github/workflows/ci.yml — new bandit (blocking) job runs bandit -r tokenpak -lll -iii --baseline.bandit-baseline.json -q. Only NEW findings beyond the baseline fail the gate; process-enforced per standard 21 §9.8.
• Baseline refresh command documented in ci.yml for future remediation work.

Added — B1a + B1b tokenpak-paid-stub discovery package + README Pro-tier paragraph (M-B1a, M-B1b)

Implements the maintainer's E2 ratification (2026-04-23): public stub on PyPI for discoverability + real tokenpak-paid on pypi.tokenpak.ai for licensed install. Two distinct PyPI names so pip's index resolution is unambiguous.

• New repo tokenpak-paid-stub (not pushed to GitHub yet — awaits repo creation + Trusted Publisher setup, the maintainer admin items):

• pyproject.toml — tokenpak-paid-stub v0.1.0, Apache 2.0, requires-python >= 3.10, no runtime deps.

• tokenpak_paid_stub/__init__.py — prints one-liner on import (stderr, non-blocking).

• tokenpak_paid_stub/__main__.py — python -m tokenpak_paid_stub prints full install guidance.

• README.md, LICENSE, .gitignore.

• .github/workflows/publish.yml — mirrors tokenpak v1.3.6 hardened release pattern (validate-pins preflight, SHA-peeled pins, SHA256SUMS outside dist/, Trusted Publisher OIDC).

• tests/test_stub_contract.py — 3 contract tests (version, ProTierRequired export, full guidance contents). Green locally against installed wheel.

• README.md — new "Pro tier" section (≤150 words) names the tier (Pro, marketing voice), the package (tokenpak-paid, technical voice), the private index, the discovery stub, and the two install channels per standards 07 + 08.

No production code in tokenpak changed — B1b is README-only. SC + SC+1 conformance matrix unchanged.

Pre-publish prerequisites (the maintainer admin):

1. Create github.com/tokenpak/tokenpak-paid-stub repo; push the local commit.
2. Configure PyPI Trusted Publisher for that repo.
3. Tag v0.1.0; the workflow publishes to public PyPI.

Manual recovery fallback: a maintainer-held PYPI_TOKEN + twine upload dist/*. Post-release update (2026-04-23): tokenpak-paid-stub==0.1.0 is now live on public PyPI (pypi.org/project/tokenpak-paid-stub/0.1.0/); initial publish went via manual twine due to a Trusted Publisher config mismatch — next release (v0.1.1) expected to publish via OIDC cleanly.

Added — C4 CLI reference autogen + onboarding doc (M-C4)

Auto-generated CLI reference locked to argparse truth + onboarding doc grounded in actually-shipped commands.

• scripts/generate-cli-docs.py — walks _COMMAND_GROUPS in tokenpak/cli/_impl.py, introspects every subcommand via build_parser, emits deterministic markdown to docs/reference/cli.md. Supports --check (CI gate mode), --stdout.
• docs/reference/cli.md — committed generator output (842 lines across all current command groups).
• docs/onboarding.md — Day 1 / 3 / 7 / 14 / 30 narrative grounded in real commands (tokenpak setup, tokenpak cost --week, tokenpak recipe list, tokenpak plan, etc.). Pro upsell kept to a single ≤50-word paragraph on Day 14 per standard 06. Links to tokenpak.ai/compliance/* trust artifacts from Day 30 ops section.
• .github/workflows/ci.yml — new cli-docs-in-sync (blocking) job per standard 21 §9.8. Runs the generator in --check mode; any drift between committed file and generator output fails the gate.
• Makefile — cli-docs + cli-docs-check targets for local iteration.

Neither the CLI generator nor the onboarding doc modifies production code. Both live in docs/ and scripts/ — shipped as package data for wheel installs, and easily synced to github.com/tokenpak/docs → docs.tokenpak.ai once that sync workflow activates.

Added — B3 license CLI 4-path verification (M-B3)

Preflight (2026-04-23) confirmed tokenpak/cli/commands/license.py ships activate / deactivate / plan commands. This packet locks the user-facing behavior across 4 paths against regression.

• tests/cli/test_license_cli_verify.py — 5 tests:

1. No license installed → tokenpak plan shows OSS tier.
2. Valid Pro license (validator mocked — signing key lives in MTC license-server, not OSS repo) → PRO in human output; standard-08 terminology discipline (human copy never contains the tokenpak-paid package slug).
3. Expired license (validator raises) → fallback to OSS tier; warning surfaces via stdout or log.
4. Corrupt license bytes (real validator exercises the garbage-in path) → fallback to OSS, no exception propagates.
5. Cross-path: get_plan never raises on malformed license-key path (directory instead of file) — the never-fail-closed contract.

No production code changed. tokenpak/agent/license/ shim remains intact (TIP-2.0 cleanup owns migration).

Added — A2 + A4 verification drift guards (M-A2, M-A4)

Preflight (2026-04-23) confirmed both compression-defaults-on and dashboard-mount were already shipped — no code change needed. These smokes catch drift between preflight and Phase 0 closeout, so a silent regression cannot reach production.

• tests/proxy/test_phase0_verification.py — 3 tests:

1. compression.enabled is True by default (flat-key accessor get_all with TOKENPAK_COMPACT scrubbed from env).
2. compression.threshold_tokens is a positive integer by default (regression guard against zero/unset).
3. serve_dashboard_file("/") returns non-empty HTML + text/html mime — dashboard is still mounted at tokenpak/proxy/server.py:346-358.

No production code changed.

Added — A6 proxy-level auth via TOKENPAK_PROXY_AUTH_TOKEN (M-A6)

Non-localhost access to a running tokenpak serve was previously ungated — any machine that could reach the proxy port could use it without authentication. A6 adds an opt-in middleware that requires a matching X-TokenPak-Auth header from non-localhost clients when the operator sets TOKENPAK_PROXY_AUTH_TOKEN.

• tokenpak/proxy/server.py — new _auth_gate + _send_json_error helpers on _ProxyHandler. Called from do_GET / do_POST / do_PUT / do_DELETE. /health bypasses the gate so liveness probes keep working.
• Auth paths (4-way gate):

1. Localhost (127.0.0.1, ::1, ::ffff:127.0.0.1) — always allowed (backwards compat).
2. Non-localhost + TOKENPAK_PROXY_AUTH_TOKEN unset → 403 forbidden.
3. Non-localhost + env set + missing/wrong X-TokenPak-Auth → 401 unauthorized (timing-safe hmac.compare_digest).
4. Non-localhost + env set + correct header → allow, header stripped, stable SHA-256 short identity retained on the handler for future telemetry attribution.

• Header choice: X-TokenPak-Auth, NOT Authorization: Bearer. Anthropic AND OpenAI both use Authorization: Bearer for their own API keys — using that header for proxy auth would collide with the client's upstream credential. X-TokenPak-Auth falls under the x-tokenpak-* prefix already excluded from PERMITTED_HEADERS_PROXY, so the SC+1 I5 header-allowlist invariant will catch any leak automatically. Deviation from the original packet spec is documented here for traceability.
• I5 belt-and-suspenders: on successful auth the gate deletes X-TokenPak-Auth from self.headers so no downstream code (passthrough, routing, compression) can forward it upstream.
• tests/proxy/test_proxy_auth.py — 9 unit tests covering all 4 auth paths + I5 static allowlist alignment.
• README.md — new "Non-localhost access" section documenting the env var, header name, and stripping guarantee.

Telemetry plumbing deferred. The packet originally asked for telemetry-row.user_id population. The TIP-1.0 telemetry-row schema has no user_id field, and schema changes are frozen for v2 per the initiative context. A stable short identity is computed and retained on the handler for later plumbing; follow-up work (TIP registry MINOR bump) will add the field and wire Monitor.log.

Conformance matrix (SC + SC+1 I1/I2/I3/I4/I5): 84/84 green with A6 in place.

Added — A5 headline benchmark pinned as blocking CI gate (M-A5)

README's "30–50% reduction" claim was previously unenforced — a PR that silently regressed compression to 25% would have shipped. Pinned to a deterministic fixture + blocking CI job per standard 21 §9.8.

• tests/fixtures/headline_corpus.txt — new deterministic 7.3 kB agent-style fixture (system prompt + 6 tool definitions + verbose user turn). Bytes-stable; representative of Claude Code / Cursor / Aider inputs.
• tests/benchmarks/test_headline_claim.py — three tests:

1. Fixture exists and is ≥ 5 kB.
2. Compression reduction ≥ 30% (the claim's minimum promise).
3. Deterministic — running twice yields identical token counts.

• pyproject.toml — registered benchmark pytest marker.
• Makefile — new benchmark-headline target runs the test and prints the measured reduction.
• .github/workflows/ci.yml — new headline-benchmark (blocking) job runs on every push/PR; process-enforced per standard 21 §9.8.
• README.md — added Reproduce the 30–50% headline claim locally: make benchmark-headline. under Quick Start.

Notable measurement: on the current fixture + HeuristicEngine, measured reduction is ~96% (1471 → 55 tokens). README's "30–50%" is a conservative claim; actual delivery substantially exceeds it. The test asserts the minimum promise (≥30%), which is the defensible gate. The maintainer reviews in the Phase 0 closeout evidence bundle whether to widen the README range to reflect measured reality.

Changed — A1 zero-config claim resolution (M-A1)

Preflight (2026-04-23) found README line 1 claimed "zero config" while line 10 disclosed manual client configuration. The cmd_setup interactive wizard existed at tokenpak/cli/_impl.py:186 (API-key detection + profile selection + proxy start) and was dispatch-registered in argparse, but was absent from _COMMAND_GROUPS, so tokenpak help did not surface it.

• _COMMAND_GROUPS["Getting Started"] — added setup entry so the wizard is discoverable via tokenpak help.
• sub.add_parser("setup", ...) — added a description= block so tokenpak setup --help now explains the wizard instead of printing a bare usage line.
• README.md — rewrote line 1 from zero config (aspirational until tokenpak integrate ships) to One command to configure your LLM proxy. (defensible today). Updated Quick Start to lead with tokenpak setup. Reframed the early-preview disclosure.
• tests/cli/test_setup_registration.py — new smoke covering: (a) setup present in Getting Started group, (b) argparse subparser dispatches to cmd_setup, (c) tokenpak setup --help exits zero and advertises the wizard.

No behavior change to cmd_setup itself; migration off the deprecated tokenpak/agent/license/ shim remains out of scope (TIP-2.0 cleanup initiative).