worklyn / zukapublic
Agent-first git hosting. One Rust binary: git over HTTP and SSH, a REST API, MCP, CI, and multi-tenant isolation.
Get a copy:
git clone https://zuka.worklyn.com/worklyn/zuka.git
| 1 | # zuka — roadmap |
| 2 | |
| 3 | Status: **complete.** M0–M7 shipped 2026-08-01. Standalone and multi-tenant both |
| 4 | work; 253 unit and 59 live tests green, clippy clean at `-D warnings`. |
| 5 | |
| 6 | M6 was verified on a real Linux host with Incus |
| 7 | ([`infra/README.md`](../../infra/README.md)): an account provisions its own |
| 8 | container, code pushes through the control plane into it, CI runs inside it, and one |
| 9 | tenant cannot reach another's repositories. |
| 10 | |
| 11 | Technical detail lives in [`SPEC.md`](SPEC.md); this file owns sequencing, risk and |
| 12 | open decisions. Product framing: [`README.md`](README.md). |
| 13 | |
| 14 | --- |
| 15 | |
| 16 | ## Milestones |
| 17 | |
| 18 | Trunk stays green at every boundary. Each milestone is independently shippable and |
| 19 | each proof is falsifiable — "it works" is not a proof. |
| 20 | |
| 21 | | M | Name | Ships | Proof | |
| 22 | |---|---|---|---| |
| 23 | | **M0** | Skeleton | Crate, `config.rs`, http shell, `error.rs` + problem types, `/healthz`, CI workflow. | **Shipped.** `cargo test` + `clippy -D warnings` + `cargo fmt --check` green in CI. | |
| 24 | | **M1** | Storage + wire | Repo create/delete, name validation, hook symlinks, `git http-backend`, Basic auth + `WWW-Authenticate`, local token file, `zuka token`. | **Shipped.** Real `git clone` + `git push` against a repo created via `POST /v1/repos`, protocol v2 negotiated; a `--force` non-fast-forward is rejected by the server-side hook and the ref does not move; a foreign account gets `404` on read and cannot clone. | |
| 25 | | **M2** | Read API | `GET` tree, raw, commits. Ref as a query parameter; blob ETags and immutable caching by object id; bounded page sizes. | **Shipped.** A pushed repository reads end to end over REST; traversal, `.git` components and smuggled git arguments are all refused. Compare and search deferred — see below. | |
| 26 | | **M3** | MCP + discovery | `/mcp` JSON-RPC (`initialize`, `ping`, `tools/list`, `tools/call`), 15 admin and discovery tools, plus `compare`, `search` and `blame` on the REST side. `git-upload-archive` enabled over SSH. | **Shipped.** A live test drives the whole loop: `repo_create` → `key_add` → the key authenticates a real `git clone`/`push` → `tree_list`/`file_read` see what was pushed. Access rules and traversal refusals are proven identical to REST. | |
| 27 | | **M4** | Limits + jobs | Per-account repo/byte quotas, token-bucket rate limiting on `/v1` and `/mcp`, `jobs/` module with its own service unit, GC + repack + `pack-refs`, deleted-repo sweep, `fsck --repair`. | **Shipped.** Over-quota create returns `413` with `limit`/`used` while reads keep working; GC concurrent with a clone of a 12 MB repo leaves it byte-identical; `fsck` finds a planted orphan in both directions and repairs only the safe one. | |
| 28 | | **M5** | CI | `.zuka.toml`, durable run queue derived from the records themselves, executor with rlimits + scrubbed env + process-group kill, runs API, MCP tools. | **Shipped.** A push runs and captures output; a failing step stops the run; a timeout kills the process group with nothing orphaned; a step cannot read the service's environment; a spec cannot raise its timeout past the host ceiling. | |
| 29 | | **M7** | Import | `POST /v1/repos {import_url}`, two-stage SSRF check (URL shape, then every resolved address), `transfer.fsckObjects`, adoption into our hooks and config. | **Shipped.** A real repository imports with history intact and is indistinguishable from one we created; loopback, private, link-local, `file://`, `ssh://` and credential-bearing URLs are all refused. | |
| 30 | | **M6** | Control plane | Incus provisioning + reconciler, Ed25519 request assertions, streaming proxy, idle stop/start, `setup --standalone\|--isolated`. | **Shipped and verified on real infrastructure.** `POST /v1/accounts/alice` returns in 64 ms while a container is provisioned behind it; a push through the control plane lands in alice's container and nowhere on the host; CI reports `uname -n` = `zuka-alice`; bob gets `404` on alice's repository and an empty repo list. | |
| 31 | |
| 32 | Deferred past v1: SSH transport, webhooks, mirroring, large-file offload, public repos, |
| 33 | commit-signature enforcement, branch protection beyond fast-forward. |
| 34 | |
| 35 | ### What the M1 review changed |
| 36 | |
| 37 | An external review of M0/M1 found ten defects, several verified against a running |
| 38 | instance. All are fixed and each has a regression test. The ones that changed the |
| 39 | design rather than a line: |
| 40 | |
| 41 | - **Cross-account theft on a case-insensitive filesystem.** `Identity` compared the |
| 42 | raw path segment, so `GET /v1/repos/Alice/site` passed the ownership check and |
| 43 | resolved to `alice`'s repository on APFS. Names are now a canonical `Name` type, |
| 44 | case-folded at the edge; the raw string never becomes a path component or a |
| 45 | comparison operand. |
| 46 | - **A repo-confined admin token could delete repositories it was denied read on.** |
| 47 | `require_admin` skipped the confinement check `check` applied. Split into |
| 48 | `require_repo_admin` (confined) and `require_account_admin` (requires an |
| 49 | unconfined token). |
| 50 | - **The streaming design in SPEC §1.2 was specified and never built.** Both |
| 51 | directions buffered: a 600 MB push cost 1.0 GB of RSS and was rejected *after* the |
| 52 | allocation, and four concurrent clones of a 257 MB repository cost 1.5 GB. There |
| 53 | was no semaphore, and the disconnect kill the file's own header comment claimed |
| 54 | reached only the direct child while `pack-objects` ran on. Now streamed end to |
| 55 | end, semaphore-bounded, process-group killed: 5 MB for both cases. |
| 56 | - **A duplicated `service` parameter split authorization from behaviour.** This |
| 57 | server took the first value, `http-backend` takes the last. A read-scoped token |
| 58 | could obtain the receive-pack advertisement. Duplicates are refused and the query |
| 59 | handed to the CGI is rebuilt rather than forwarded. |
| 60 | - **The ref rule existed only in shell**, contradicting both the spec and |
| 61 | `AGENTS.md`. It is now `git::store::check_ref_move`, and the hook is a one-line |
| 62 | `exec` into the binary. |
| 63 | - **Comments described controls that did not exist** — the disconnect kill, "bounded |
| 64 | before they are buffered", "the API refuses to mint" a non-expiring token. That |
| 65 | pattern is worse than the individual bugs, because it is what made them survive |
| 66 | review. Each comment now describes code that is there. |
| 67 | |
| 68 | Also fixed: `ZUKA_ME_URL` was validated, logged and ignored (now refuses to boot); |
| 69 | the CLI could only mint unscoped non-expiring admin tokens; `tokens.json` was 0644; |
| 70 | `ensure_space` was never called on the push path and `receive.maxInputSize` was never |
| 71 | set; git paths answered `problem+json` on every status except 401; there were no |
| 72 | connection timeouts. |
| 73 | |
| 74 | Found while fixing, not in the review: caching credentials at boot meant a token |
| 75 | minted by the CLI did not work until restart. Now re-read on change, keeping the last |
| 76 | good contents when a file stops parsing. |
| 77 | |
| 78 | ### What M1 changed in the plan |
| 79 | |
| 80 | - **`/v1/repos/{account}/{repo}/refs` shipped early**, inside M1 rather than M2 — the |
| 81 | force-push test needs to read a ref back to prove it did not move, and a test that |
| 82 | cannot observe the invariant it asserts is not a test. |
| 83 | - **Tree-path validation was written, tested, removed, and returned with M2**, which |
| 84 | is the milestone that gave it a caller. Blob-mode validation was removed again for |
| 85 | the same reason and returns with M3's write path. |
| 86 | - **`libc::statvfs` replaced a hand-declared struct.** The layout differs between |
| 87 | macOS and Linux; declaring it by hand reads garbage rather than failing loudly. |
| 88 | - **The disk reserve is enforced at repo create**, not deferred to M5. A repository |
| 89 | created without room for its object store is corrupt rather than empty. |
| 90 | |
| 91 | ### The write API was cut, 2026-08-01 |
| 92 | |
| 93 | An earlier draft of this roadmap and of `SPEC.md` made "commit without cloning" the |
| 94 | load-bearing capability, with `POST /commits`, `base_sha` optimistic concurrency, ref |
| 95 | compare-and-swap and idempotency keys. **That was wrong and it is cut.** No code was |
| 96 | written for it; the correction is documentation only. |
| 97 | |
| 98 | The reasoning that replaced it: agents already run git perfectly well. What blocks a |
| 99 | non-developer is the forge — provisioning, keys, permissions, PRs, CI config — not the |
| 100 | committing. Building a second write path would have meant two homes for the |
| 101 | fast-forward rule, two concurrency models and a merge story, all of which git already |
| 102 | has and does correctly. |
| 103 | |
| 104 | So the boundary is now strict: **git writes, the API administers and reads.** That |
| 105 | deletes `POST /commits`, `PUT /files`, `PUT`/`DELETE` on refs, and the MCP write |
| 106 | tools; it also deletes the `gix-ref` transaction requirement from SPEC §1.3, because |
| 107 | with `receive-pack` as the only writer there is no second writer to race. |
| 108 | |
| 109 | `compare` and `search` survive the cut and move into M3. They were justified as |
| 110 | "an agent must reconcile a 409" — that reason is gone, but a better one remains: |
| 111 | answering "what is in here" without a full clone is worth an endpoint on its own. |
| 112 | |
| 113 | ### What M4 found |
| 114 | |
| 115 | - **git's auto-gc was on.** `gc.auto` was unset, so `receive.autogc` (default true) |
| 116 | fired a repack *inside* pushes once loose objects passed 6700 — uncoordinated, at |
| 117 | a moment nobody chose. SPEC §9.4 had said to disable it; it had never been done. |
| 118 | Now off at creation and reasserted by the job. |
| 119 | - **The accumulation is loose objects, not packs.** Measured at ~3 per commit: small |
| 120 | pushes fall under `receive.unpackLimit` and get exploded rather than kept as packs. |
| 121 | The earlier assumption in this doc was wrong. |
| 122 | - **A delete reclaimed nothing.** `soft_delete` was built without its drain, so |
| 123 | `tmp/deleted/` grew forever on the same disk as live repositories. |
| 124 | - **The grace window, not a lock, is what makes GC safe.** A lock cannot span the |
| 125 | server and the jobs process. `upload-pack` can reference an object between |
| 126 | resolving and streaming it, so the prune expiry is the real control — which is |
| 127 | also why `ZUKA_GC_PRUNE_GRACE` is configuration rather than a constant. |
| 128 | - **`fsck --repair` deletes records, never repositories.** A repository with no |
| 129 | metadata is reported and kept: deleting real bytes on a mismatch is how a |
| 130 | reconciler turns a small inconsistency into data loss. |
| 131 | |
| 132 | ### Deferred |
| 133 | |
| 134 | - **`gix`.** Reads shell out to `git` today. That is a process spawn per request, |
| 135 | which SPEC §1.2 explicitly argued against; it is mitigated by running them on a |
| 136 | blocking pool rather than a runtime worker. Migrating the read path to `gix` is |
| 137 | worth doing on measurement, not on principle, and no measurement has been taken. |
| 138 | |
| 139 | --- |
| 140 | |
| 141 | ## Risks |
| 142 | |
| 143 | | # | Risk | Mitigation | |
| 144 | |---|---|---| |
| 145 | | R1 | **The wire protocol is subtle.** Sideband, protocol v2, shallow, `--filter`, gzipped bodies, half-closed connections. Easy to get almost right and ship a hang. | Delegate to `git http-backend`, named explicitly (SPEC §1.2). M1's proof is a real clone and push, not a unit test. | |
| 146 | | R2 | **Ref-name injection is RCE.** `../../config` writes git config; `core.pager`/`core.sshCommand`/`core.fsmonitor` are execution keys. | `gix-validate`, never hand-rolled; post-canonicalization prefix check. Normative in SPEC §2.5, not a test bullet. | |
| 147 | | R3 | **Tenant secret compromise.** CI runs arbitrary code in the container that holds the service's credentials. A symmetric key or shared KV token there breaks multi-tenancy entirely. | Ed25519 with the private key never distributed; per-tenant KV credentials; separate runner uid; `env -i`. SPEC §2.4, §6.3. | |
| 148 | | R4 | **Ref TOCTOU.** Two writers per ref — gix in-process and `receive-pack` in a subprocess. Checking then writing is a race. | `gix-ref` transaction with `PreviousValue::MustExistAndMatch`; preconditions enforced *inside* it. SPEC §1.3. | |
| 149 | | R5 | **`gix` API churn.** Pre-1.0. | Pin exactly. All use behind `git/read.rs` + `git/write.rs`. Add `cargo deny` — pinning stops build breakage, not advisories, and parsing untrusted git objects is where they will land. | |
| 150 | | R6 | **Control-plane proxying.** Multi-gigabyte bidirectional streams with backpressure, plus unbuffered SSE. Shelling out at the engine does not help at the proxy hop. | Its own milestone scope, its own proof. Not treated as free. | |
| 151 | | R7 | **CI is arbitrary code execution.** In standalone there is no container. | Off by default in standalone, said plainly in the README. uid separation, cgroup kill, rlimits, netns. No step allowlist — `sh -c` defeats it. | |
| 152 | | R8 | **The differentiator is thinner than it looks.** With git doing the writing, what is left is administration — which Gitea also has, behind more UI. | The bet is that *absence* is the feature: no orgs, teams, PRs or review, one binary, MCP-native. M3's gate is what tests that bet, and it is a real test because it can fail. | |
| 153 | | R9 | **Destructive operations.** An agent will eventually delete the wrong repo. | Soft delete to `tmp/deleted/` with a 7-day sweep. Reflog on every ref move. Disk-move-then-KV ordering with `fsck` reconciliation. | |
| 154 | | R10 | **Storage exhaustion.** Loose objects per API commit, packs per push, CI workspaces, soft-deleted repos on the same disk. | GC + repack + sweeps in M5, before CI in M6 adds workspace churn. `507` at a reserve threshold, before ENOSPC corrupts a pack mid-receive. | |
| 155 | |
| 156 | --- |
| 157 | |
| 158 | ## Open decisions |
| 159 | |
| 160 | Each blocks the milestone named. |
| 161 | |
| 162 | **~~D1 — Name.~~** Resolved: **zuka**, renamed from the `sanduku` placeholder |
| 163 | 2026-08-02. The edit was one line in `Cargo.toml` plus a directory move, because |
| 164 | `brand.rs` derives the env prefix, data directory, problem-type URIs, auth realm, git |
| 165 | config namespace, header names and CI filename from `CARGO_PKG_NAME`. A test asserts |
| 166 | the name appears nowhere else in `src/`, and it caught three literals during the |
| 167 | rename. |
| 168 | |
| 169 | **~~D9 — The control plane is deferred.~~** Resolved by provisioning a Linux host |
| 170 | rather than by relaxing the standard. Everything above the Incus shim is tested |
| 171 | against a fake provisioner on any machine; the shim itself was verified on the VM in |
| 172 | [`infra/README.md`](../../infra/README.md). |
| 173 | |
| 174 | **~~D2 — Commit authorship.~~** Resolved by cutting the write API: zuka never |
| 175 | authors a commit, so there is nothing to attribute. Whatever the agent's local git |
| 176 | identity is, is what lands — which is also the honest answer. |
| 177 | |
| 178 | **D3 — Public repos.** Built, after rate limiting landed in M5. Deliberately narrower |
| 179 | than the original framing: public grants anonymous **read** — `git clone` and the |
| 180 | browser views — and never an anonymous REST write or an anonymous principal threaded |
| 181 | through the identity model. `Identity` was left alone; anonymous access lives in one |
| 182 | function, `open_public_repo`, that can only be asked for read. The DoS surface is |
| 183 | bounded by the existing per-IP rate limiter. |
| 184 | |
| 185 | **D4 — Large files.** The original design carried a separate SHA-256 content-addressed |
| 186 | blob store with a SQLite path→hash index. Cut: git's object database is already |
| 187 | content-addressed with dedup, so a second store only earns its place for large |
| 188 | binaries that should stay out of the pack. Revisit as LFS-shaped offload when a real |
| 189 | repo hurts. Recorded so it is not rebuilt by reflex. |
| 190 | |
| 191 | **D5 — CI concurrency.** Runs per account, queue depth, and overflow behaviour — |
| 192 | reject, queue, or evict oldest. The token-bucket limiter from M4 is the shape to |
| 193 | reuse. Blocks M5. |
| 194 | |
| 195 | **D8 — Quota defaults.** Shipped at 100 repos / 2 GB per repo / 10 GB per account, |
| 196 | 600 requests per minute. Those are guesses with no usage data behind them; revisit |
| 197 | once anything real runs on this. |
| 198 | |
| 199 | **D6 — Observability transport.** Whether this emits OTLP into |
| 200 | `product/observability/` or stays with structured logs plus counters. Blocks nothing; |
| 201 | decide before M6 makes multi-tenant debugging urgent. |
| 202 | |
| 203 | **D7 — Hosting shape.** Incus is chosen over Firecracker because these are persistent |
| 204 | servers with SSH, networking and storage rather than short-lived functions. Not |
| 205 | revisited unless the idle stop/start cost model in SPEC §7.1 fails. |