v2: on-call micro benchmark against v1 (jobFor 1.69x in v1, 1.12x in v2; dedupe 6.88x in v1, 0.92x in v2), README v2 section, LEARNINGS for v2 including the read-only Rebasely audit (82 findings, 7 tr
bench/oncall rebuilds the pilot's benchmark: the pre-pilot jobFor and dedupe test (Rebasely fcf3b6a9), the pilot's compiled rules and runtime (c5a11d4f, unchanged) and a v2 port, with a differential check over 20,000 events first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Changed files
11 files changed808 additions2 deletionsagainst 9c48a39
- ChangedLEARNINGS.md+37−0
- ChangedREADME.md+98−2
- Addedbench/oncall/RESULTS.txt+22−0
- Addedbench/oncall/old.ts+39−0
- Addedbench/oncall/run.ts+116−0
- Addedbench/oncall/types.ts+16−0
- Addedbench/oncall/v1/_runtime/boundary.js+32−0
- Addedbench/oncall/v1/_runtime/index.js+117−0
- Addedbench/oncall/v1/rules.js+223−0
- Addedbench/oncall/v1.ts+20−0
- Addedbench/oncall/v2/rules.ts+88−0
ChangedLEARNINGS.md+37−0
| @@ -104,3 +104,40 @@ | ||
| - Fix edits have to be expressed against the original `.tsus` text, but the rules see macro-expanded code. The rules use the preprocessor's offset map when it is passed in, and otherwise align the two texts themselves (`rules/sourcemap.js`: a line diff, then a token diff inside changed hunks). Either way a fix is only offered when its range covers text that is identical in both versions, so an edit can never land on generated glue. Example 54 puts fixes inside match arms to keep this honest. | ||
| - Floating promises and unused Results are detectable without full type information for the common case: a call to a function whose declaration (in this file or a relative import) is `async`, annotated `Promise<...>` or `Result<...>`, or built with `wrapAsync`/`wrapSync`. When a `ts.Program` is available the same rules switch to real types. | ||
| - "Safe" fixes are defined by runtime behavior, not by type-checking: `any` to `unknown` never changes what runs, but it can surface type errors `any` was hiding. That is the point of the rule, so it stays a safe fix. | ||
| + | ||
| +# Learnings from v2: plain `.ts`, a checker, and security types | ||
| + | ||
| +v2 started from four facts. On a feature built from a detailed spec, v1 tied plain strict TypeScript on correctness (118/118 hidden tests each, identical on 20,400 fuzz pairs) while costing 28% more tool calls, 65% more code, a 51-line adapter and idioms like `parts.slice(0, 1).join("")`, and its checker reported nothing in eight runs. The pilot's wrapper objects cost 2 to 6x on the on-call hot path. Rewriting existing logic was where v1 found real bugs. And the problems reviews of AI-written web code kept finding (SQL injection, permission scoping, secrets, untrusted input) were ones v1 had no words for. | ||
| + | ||
| +## Erasable is a hard constraint, and it is cheap to meet | ||
| + | ||
| +Dropping the macros removed the compiler, the preprocessor, the source maps and the bundler plugins in one move, and the type system still carries what the macros did. `match(value, "kind", arms)` is exhaustive because the arms parameter is a mapped type over `V[K]`, and it narrows each arm because the key is a `const` type parameter fixed before the arms are checked (v1 needed currying to get the same inference). Two details mattered. The arms' constraint returns `any`, because when the arms are wrong TypeScript falls back to the constraint and an `unknown` there adds a second error at every use (the same lesson v1 learned). `Ok` and `Err` take `const` type parameters, which keeps `Err({ kind: "missing" })` literal without the contextual-typing machinery v1 built into its match. | ||
| + | ||
| +## Option as `T | undefined` moves the safety into the checker | ||
| + | ||
| +With no wrapper, the type is free (the dedupe test went from 6.88x in v1 to 0.92x), but `T | undefined` has one gap a wrapper does not: truthiness. TSUS503 rejects a truthiness test on an optional string or number, where `""` or `0` would read as absent, and nothing else; objects and booleans test fine. `noUncheckedIndexedAccess` plus `=== undefined` replaces v1's de facto indexing ban. Banning only the production of `null` (TSUS502), not comparisons with it, is what removed the adapter: `re.exec(s) === null` is fine, and D1 rows come through `safeDb`, which turns SQL NULL into `undefined`. | ||
| + | ||
| +`match()` is not free. Its arms are closures, so a four-way literal match costs about 14 ns a call with the arms inline and 6 ns hoisted, against under 1 ns for a switch. TSUS501 checks switches over unions for exhaustiveness, so a hot path loses nothing by using one. | ||
| + | ||
| +## A brand is only as strong as the cast ban | ||
| + | ||
| +`Sql`, `Permit`, `Secret` and `Untrusted` are interfaces keyed by `unique symbol`s that `index.d.ts` declares and does not export, so no code can write a literal of one. That makes them unforgeable except by a cast, which is the real reason the cast ban (TSUS506), `any` (TSUS508) and `!` (TSUS507) stay in v2: the experiment's 11 casts were all safe, so on correctness alone the rule would have gone. The checker recognizes the brands structurally (TypeScript names the property `__@SqlBrand@<id>`), so they survive aliases, intersections and generics. | ||
| + | ||
| +The types do most of the work. A query string passed to `SafeDb` is TS2345, a missing Permit is TS2554 or TS2345, a Secret where a string is expected is TS2322. The checker names those errors by what they mean (TSUS600, TSUS601, TSUS604) and covers what TypeScript accepts from any type: template literals, `+`, `String()`, `console.log(obj)`, `JSON.stringify(obj)` (with a depth-limited search for a Secret inside an object). Untrusted needed one more idea: the same `${text}` is XSS in markup, injection in SQL and prompt injection in a prompt, so the rule classifies the template by its static text and where its value goes, and gives each sink its own code and its own fix. | ||
| + | ||
| +## The planted-bug demo is the honest test | ||
| + | ||
| +The same module written twice, with six bugs planted the way they would be written in each style, says more than a list of rules. Plain strict TypeScript, with `noUncheckedIndexedAccess`, reported none of the six. v2 reported all six, each at its line, and the runner fails on any diagnostic off a planted line, so the demo also measures false positives (none). Running the attacks against both versions on the same SQLite made the bugs concrete: `' OR 1=1 --` returned another user's private rows, a `CASE WHEN` in the sort parameter answered a yes/no question about a private table, and a fenced prompt kept `<system>` out of the model's input. `tsus audit` on the plain version found the two injections and the missing check, and could not see the other three: knowing that a value is a secret or came from a user is exactly what the types add. | ||
| + | ||
| +## Report-only analysis is where the false positives live | ||
| + | ||
| +`tsus audit` on Rebasely's shipped search and triage modules reported 82 findings. Seven were real, all in `triage/`: per-repository Actions failures in the Inbox are dropped without marking the list partial (`inbox.ts:184`), and six failures are swallowed with no log while the calls next to them log theirs (`inbox.ts:222`, `223`, `224`; `handlers.ts:54`, `55`, `101`). 75 were false positives. Three were wrong: SQL fragments in `search/sql.ts` whose column names arrive as parameters that are literals at every call site (the constant analysis does not follow parameters), and a bug-report read in `triage/sources.ts` reached through a dispatch table (`SOURCES[i].items`), which the call graph cannot follow. The other 72 described the pattern correctly and were harmless on inspection: index reads guarded by a length or loop bound, regex groups after a match, documented best-effort catches, casts after a check. The search module, already fuzzed by the experiment's judge, had no true positives. | ||
| + | ||
| +The permission analysis needed three passes to stop lying. Counting a check anywhere among a function's transitive callees marked everything checked, because a sibling call deep in the tree always reaches `authorize`. Not descending into other data functions fixed that, and requiring the check to come before the call in each caller fixed the case of a harness that calls a checked and an unchecked handler side by side. Inside the reading function itself a check anywhere has to count, because filtering the rows it just read with `authorize` is a check (Rebasely's `issueRows` does exactly that). It remains a name-based heuristic; in v2 code the Permit type makes it unnecessary. | ||
| + | ||
| +## Known limitations | ||
| + | ||
| +1. The security types are only as good as the marking: text is Untrusted because a boundary said so. A prompt built under a name the rule does not recognize falls to the general TSUS607. | ||
| +2. Raw driver calls in application code are still possible; TSUS601 catches their string-built queries but not their missing checks. | ||
| +3. The audit's permission analysis is name-based, ordered by position in the source, and blind to dispatch tables and dynamic calls; its constant-string analysis does not follow function parameters. | ||
| +4. `match()` allocates its arms; hot paths should use a checked switch. | ||
ChangedREADME.md+98−2
| @@ -4,12 +4,99 @@ | ||
| > **This repo is a thought experiment, not a production tool.** The compiler works, type-checks, and every example passes, but the macro syntax is lowered by a preprocessor with known edge cases, the bundler plugins skip semantic checking, there are no published packages, and there is no stability promise. It exists to make the proposal concrete enough to argue with. Do not build anything you care about on it yet. | ||
| -**Status:** working prototype. `packages/cli` implements `tsus check` and `tsus build`: a tokenizer built on `ts.createScanner`, a preprocessor that lowers `match` and the `?` operator with a source map back to the `.tsus` file, and a full `ts.createProgram` type-check over the project. `match` is exhaustiveness-checked at compile time, `tsus build` emits `.js.map` files that point at the `.tsus` source, `packages/runtime` implements `Option`/`Result`, the typed match helpers and the boundary layer, `packages/plugin` provides webpack/Vite/esbuild/Bun adapters, the checker enforces 18 strictness rules with machine-applicable fixes (`tsus fix`, `tsus explain`, `tsus migrate`, `tsus init`; see [For agents](#for-agents)), and all 53 projects under `examples/` pass (`npm install && npm test`). `LEARNINGS.md` documents what broke along the way, how that shaped the design, and the known limitations. | ||
| +**Status:** working prototype. This branch adds [v2](#v2-plain-ts-a-checker-and-security-types), which replaces the macros and the new extension with plain `.ts`, a checker and a small type library, and adds security types. The v1 description follows it. | ||
| +v1 status: `packages/cli` implements `tsus check` and `tsus build`: a tokenizer built on `ts.createScanner`, a preprocessor that lowers `match` and the `?` operator with a source map back to the `.tsus` file, and a full `ts.createProgram` type-check over the project. `match` is exhaustiveness-checked at compile time, `tsus build` emits `.js.map` files that point at the `.tsus` source, `packages/runtime` implements `Option`/`Result`, the typed match helpers and the boundary layer, `packages/plugin` provides webpack/Vite/esbuild/Bun adapters, the checker enforces 18 strictness rules with machine-applicable fixes (`tsus fix`, `tsus explain`, `tsus migrate`, `tsus init`; see [For agents](#for-agents)), and all 57 projects under `examples/` pass, 53 for v1 and 4 for v2 (`npm install && npm test`). `LEARNINGS.md` documents what broke along the way, how that shaped the design, and the known limitations. | ||
| + | ||
| TypeScript's early design decisions assumed humans would write and maintain it, so a lot of features exist to make a human's life easier. We valued speed over quality. With AI writing most of our code, we can flip that tradeoff and force the language to be declarative and verifiable, like Rust. | ||
| -## What changes | ||
| +## v2: plain `.ts`, a checker, and security types | ||
| +v2 lives on the `v2` branch. It keeps what v1 proved and drops what it cost. Everything is ordinary, erasable TypeScript: no macros, no `.tsus` extension, no compile step. A file that uses v2 is a `.ts` file that tsc, esbuild, wrangler and node's own type stripping handle unchanged (example 61 runs the same file through an esbuild bundle and through `node main.ts`; the root `tsconfig.json` checks the v2 files with `erasableSyntaxOnly`). v2 is three pieces: | ||
| + | ||
| +- **`@tsus/v2`** (`packages/v2`), a type library with a small runtime: `Option`, `Result`, `match`, `Sql`, `Permit`, `Secret`, `Untrusted`. | ||
| +- **`tsus check --v2 [dir]`**, a checker over the project's own TypeScript program (its tsconfig, plus `strict` and `noUncheckedIndexedAccess`). Codes TSUS500 to TSUS607, documented with the reason each one exists in [docs/v2/rules.md](docs/v2/rules.md) and by `tsus explain`. | ||
| +- **`tsus audit <file.ts...>`**, a report-only pass over existing plain TypeScript. | ||
| + | ||
| +### What the experiment taught, and what v2 does about it | ||
| + | ||
| +On a new feature built from a detailed spec (the search query language, judged on 118 hidden tests and 20,400 fuzz pairs), v1 tied plain strict TypeScript on correctness and cost 28% more tool calls, 65% more code, a 51-line adapter and workarounds such as `parts.slice(0, 1).join("")` for `parts[0]`. Its checker reported nothing in 8 runs. In the on-call pilot, the Option and Result wrapper objects cost 2 to 6 times on the hot path, while rewriting existing logic exposed real bugs. And the risks that reviews of AI-written web code kept finding were SQL injection, permission scoping, secrets and untrusted input, for which v1 had nothing. | ||
| + | ||
| +| Lesson | v2 | | ||
| +|---|---| | ||
| +| Wrappers cost 2 to 6x | `Option<T>` is `T \| undefined`; `Result<T, E>` is `{ ok: true, value } \| { ok: false, error }`; `match` is one lookup and a call | | ||
| +| The adapter was mostly null conversion | Comparing with `null` is allowed, producing it is not (TSUS502); `safeDb` turns D1's SQL NULL into `undefined` | | ||
| +| Banning `undefined` and `!` banned indexing | `noUncheckedIndexedAccess` and an `=== undefined` check | | ||
| +| Friction rules caught nothing | Dropped: no class, var, throw, enum, `==`, default export, parameter reassignment, unused `let`, tagged-errors-only. Each kept or dropped rule is listed with its evidence in docs/v2/rules.md | | ||
| +| Exhaustive matching was the one prevention reviewers credited | Kept: `match()` and `matchLiteral()` are typed so a missing arm is a type error, and TSUS501 checks switches | | ||
| +| No security model | `Sql`, `Permit`, `Secret`, `Untrusted`, with checker rules for what the types cannot express | | ||
| +| Rewriting found bugs | `tsus audit` lists where the bugs hide in code nobody is rewriting | | ||
| + | ||
| +### The library | ||
| + | ||
| +```ts | ||
| +import { match, matchLiteral, matchResult, Ok, Err, sql, safeDb, html, fence, secret, reveal, untrusted, validate } from "@tsus/v2"; | ||
| +import type { Option, Result, Sql, Permit, Secret, Untrusted } from "@tsus/v2"; | ||
| + | ||
| +const area = (s: Shape): number => | ||
| + match(s, "kind", { circle: (c) => Math.PI * c.r ** 2, square: (q) => q.side ** 2 }); // a missing arm is a type error | ||
| + | ||
| +const rows = await safeDb(env.DB).all<IssueRow>( | ||
| + sql`SELECT number, title FROM issues WHERE repo_id = ${permit.resource} AND title LIKE ${pattern}`); // values are bound params | ||
| + | ||
| +const page = html`<li>${issue.title}</li>`; // Untrusted text is escaped | ||
| +const prompt = `Summarize:\n${fence(issue.body, "issue body")}`; // marked as data for the model | ||
| +const headers = { authorization: `Bearer ${reveal(apiKey)}` }; // the one explicit way out of a Secret | ||
| +``` | ||
| + | ||
| +- **Sql** is a branded type that only the `sql` tag makes; every interpolation becomes a `?` and a bound parameter, nested `Sql` splices in, `sql.ident` takes only a string literal type. `SafeDb` accepts only `Sql`, so a query built with `+` or an untagged template is a compile error (TS2345, reported as TSUS601). For raw D1 calls the checker flags `prepare(text)` whenever the text is not a compile-time constant. | ||
| +- **Permit<"repo.read", RepoId>** is an opaque capability. Only `issuePermit` from `@tsus/v2/authority` makes one, and only files named `*.access.ts` may import it (TSUS602). A data-access function that reads private data takes a Permit, so calling it without the check is a missing argument and a hand-made object is a missing brand (TSUS600). It should read the id from `permit.resource`; a separate id of the same type is flagged (TSUS603), since that is how a check on repo A guards a read of repo B. | ||
| +- **Secret<T>** is not a string. `reveal()` is the explicit way out. The checker rejects a Secret, or an object containing one, in templates, `+`, `String()`, `JSON.stringify`, `console.*` and other log calls, URLs, `fetch` and `Response` (TSUS604). At runtime it prints as `[secret]` as a backstop. | ||
| +- **Untrusted** is the raw string at runtime (marking costs nothing) with a type that is not a string. It reaches HTML through `html`/`escapeHtml`, SQL through `sql`, a prompt through `fence`, and code that needs a plain string through `validate(text, /pattern/)`. Untagged templates and concatenation are rejected by context: HTML (TSUS605), SQL (TSUS601), an LLM prompt (TSUS606, a template assigned to a name like `prompt`, `system` or `messages`, or passed to a model call), anything else (TSUS607). `fence` wraps the text as `<untrusted source="...">` and neutralizes the closing tag and role markers inside it, the markers Rebasely's injection screen looks for. | ||
| +- Files named `*.boundary.ts` may cast, use `!`, `any` and `null`, and call raw drivers: they are where outside data is checked and converted. Their names make them reviewable. | ||
| + | ||
| +### Proof: the same module twice, with planted bugs | ||
| + | ||
| +`examples/60-v2-security-demo` is a Workers-style module (D1 queries, a permission check, a webhook secret, user text into HTML and into an LLM prompt) written in plain strict TypeScript (`plain/`) and with v2 (`v2/`), with the same six bugs planted in both, and a fixed v2 version (`fixed/`). The runner asserts all of it: plain passes `tsc --strict --noUncheckedIndexedAccess`; v2 reports every planted bug with the expected code and nothing on any other line; the fixed version passes, bundles with esbuild, and a harness runs the attacks against both versions on the same in-memory SQLite. | ||
| + | ||
| +| Planted bug | plain strict TS | `tsus audit` on the plain file | v2 (`tsus check --v2`) | At runtime, plain / fixed v2 | | ||
| +|---|---|---|---|---| | ||
| +| 1. Search text pasted into the query | nothing | string-built SQL | TSUS601 | `' OR 1=1 --` returns the private repo's rows / no rows | | ||
| +| 2. Sort column pasted into the query | nothing | string-built SQL | TSUS601 | a `CASE WHEN` sort leaks who owns a private repo / ignored | | ||
| +| 3. List handler skips the permission check | nothing | unchecked permission | TSUS600 (TS2345) | private rows listed signed out / 404 | | ||
| +| 4. Webhook key in a log line | nothing | nothing | TSUS604 | key in the logs / not | | ||
| +| 5. Issue titles into HTML unescaped | nothing | nothing | TSUS605 | `<script>` served / escaped | | ||
| +| 6. Issue body into an LLM prompt unmarked | nothing | nothing | TSUS606 | `<system>` reaches the model / fenced and neutralized | | ||
| + | ||
| +The audit finds the two classes it can see without types; the other four need to know which values are secrets and which are user text, which is what the v2 types add. | ||
| + | ||
| +### Runtime | ||
| + | ||
| +`npm run bench` rebuilds the on-call micro benchmark from the pilot (`bench/oncall`; the pilot's own scripts were not committed, so the event generator is new and absolute numbers differ from the pilot's). The v1 rules are the pilot's compiled `rules.js` and runtime, unchanged. A differential check first confirms all versions agree on 20,000 events. Node 24.17, darwin arm64, median of 7 runs (raw output in `bench/oncall/RESULTS.txt`): | ||
| + | ||
| +| Path | before (plain .ts) | v1 | v2 | | ||
| +|---|---|---|---| | ||
| +| `jobFor`, per event (page of 1,000) | 310 ns | 523 ns (1.69x) | 348 ns (1.12x); 339 ns (1.09x) with the rank as a switch | | ||
| +| dedupe test, per job | 8.3 ns | 56.9 ns (6.88x) | 7.6 ns (0.92x) | | ||
| +| absent-or-value, 1,000 reads | 775 ns (`x ?? 0`) | 14,368 ns (`fromNullable(x).unwrapOr(0)`) | 1,115 ns (`fromNullable(x) ?? 0`) | | ||
| +| 4-way literal match, 1,000 calls | 934 ns (switch) | 47,445 ns (match macro) | 13,813 ns (`matchLiteral`, arms inline); 6,016 ns (arms hoisted) | | ||
| + | ||
| +Option is free. `match()` is not: its arms are closures, so a hot loop should use a switch, which TSUS501 checks for exhaustiveness all the same. About half of what is left on `jobFor` is `tryCatch` (a closure and an `Ok` object per event) around `JSON.parse`: with a plain try/catch there, v2's `jobFor` measured 326 ns (1.05x) in the same run. | ||
| + | ||
| +### `tsus audit` | ||
| + | ||
| +`tsus audit <file.ts...> [--json] [--scope dir] [--permission-fn name,...]` reads plain TypeScript with the project's own tsconfig and writes nothing. It ranks: string-built SQL (a query whose text is not a compile-time constant, or a template that assembles SQL text from non-constant parts), unchecked permission paths (a database read reachable from an entry point, through the call graph of the package, with no call to something named like a check before it), ignored promises, swallowed errors, non-exhaustive switches over unions, implicit null paths (index reads that `noUncheckedIndexedAccess` would reject, truthiness tests on optional strings and numbers, `!`) and casts. Every finding is a lead with the reason it was flagged; `--json` also lists each database read it considered checked and the check it found. Example 63 pins the findings on a legacy file and asserts the file is unchanged. | ||
| + | ||
| +Run read-only against Rebasely (origin/main 438e678a, apps/web/src/search/*.ts and apps/web/src/triage/*.ts), it reported 17 and 65 findings. Of the 82, 7 were true positives, all in triage: per-repository Actions failures in the Inbox are dropped without marking the result partial (`inbox.ts:184`), and six failures are swallowed without a log while the calls next to them log theirs (`inbox.ts:222-224`, `handlers.ts:54-55`, `handlers.ts:101`). The other 75 were false positives: 3 wrong (two SQL fragments whose column names are literals at every call site, and one data read reached through a dispatch table) and 72 correct about the pattern but harmless on inspection (bounds-checked index reads, documented fallbacks, safe casts). The search module, which the experiment's judge had already fuzzed, had none. LEARNINGS.md has the breakdown. | ||
| + | ||
| +### Limits | ||
| + | ||
| +- The security rules are as good as the marking: text is Untrusted because a boundary said so, and a value that never got marked is a plain string. The prompt rule recognizes prompts by name and by call; a prompt built under another name falls to TSUS607. | ||
| +- The permission rule in v2 is a type, so it holds wherever the data-access function is the only way to the rows. Raw D1 calls in application code are still possible; TSUS601 catches their string-built queries, not their missing checks. The audit's permission analysis is a name-based heuristic with call-site ordering, and it cannot follow calls through dispatch tables. | ||
| +- `match()` costs closures; use a switch on hot paths. | ||
| + | ||
| +## v1: what changes | ||
| + | ||
| 1. Removes `null` and `undefined`. In their place, `Option<T>` and `Result<T, E>` become first-class types. This is the centerpiece: the removals below only work because these replacements exist. | ||
| 2. Removes classes. Data is plain objects; behavior lives in functions. | ||
| 3. Removes `var`. | ||
| @@ -237,6 +324,15 @@ | ||
| | `27-nextjs` | Next.js 14 via the webpack loader; `next build` renders `.tsus` output into static HTML | | ||
| | `28-astro` | Astro 4 via the Vite plugin, verified in built HTML | | ||
| | `29-bun` | Bun runs `.tsus` imports natively via a `bunfig.toml` preload, no build step | | ||
| + | ||
| +**v2** (plain `.ts`; see [v2](#v2-plain-ts-a-checker-and-security-types)) | ||
| + | ||
| +| Example | Shows | | ||
| +|---|---| | ||
| +| `60-v2-security-demo` | The same Workers module in plain strict TS and in v2 with six planted bugs: plain passes tsc, v2 flags all six (and nothing else), the fixed version bundles with esbuild, and a harness runs the attacks against both | | ||
| +| `61-v2-basics` | `match`, `matchLiteral`, `matchResult`, Option as `T \| undefined`, `sql`, `html`, `validate`; checked, bundled with esbuild and run with node's type stripping, same output | | ||
| +| `62-v2-rejected` | Every v2 code from TSUS500 to TSUS607, each pinned to the line it must land on, with nothing reported anywhere else | | ||
| +| `63-v2-audit` | `tsus audit` on a legacy file: the findings pinned to their lines, and the file unchanged | | ||
| **Agent tooling** | ||
Addedbench/oncall/RESULTS.txt+22−0
| @@ -0,0 +1,22 @@ | ||
| +events: page of 1000, 278 jobs; differential over 20000 events and 278 dedupe tests: 0 disagreements | ||
| +before (plain .ts) jobFor, one page of 1000 events median 310247.9 ns/op (min 307642.8, max 318352.2) | ||
| +v1 (pilot) jobFor: classify in tsus + .ts adapter median 523059.7 ns/op (min 522090.0, max 530044.4) | ||
| +v2 jobFor: Option is T | undefined, no adapter median 347682.2 ns/op (min 343605.7, max 355151.0) | ||
| +v2 jobFor, rankOf as an exhaustive switch median 339392.9 ns/op (min 334642.5, max 342588.1) | ||
| + -> per event: before 310 ns, v1 523 ns (1.69x), v2 348 ns (1.12x), v2 with switch 339 ns (1.09x) | ||
| +before dedupe test (inline), 278 jobs median 2301.1 ns/op (min 2284.7, max 2381.4) | ||
| +v1 isDuplicate(fromNullable(...)), 278 jobs median 15825.3 ns/op (min 15565.0, max 16013.1) | ||
| +v2 isDuplicate(fromNullable(...)), 278 jobs median 2106.8 ns/op (min 2097.8, max 2241.1) | ||
| + -> per job: before 8.3 ns, v1 56.9 ns (6.88x), v2 7.6 ns (0.92x) | ||
| +Primitives (1000 per op): | ||
| +plain x ?? 0 median 774.8 ns/op (min 774.0, max 843.0) | ||
| +v1 fromNullable(x).unwrapOr(0) (Some: object + 8 closures) median 14368.0 ns/op (min 14342.8, max 14462.6) | ||
| +v1 Some(x).value median 1791.6 ns/op (min 1782.0, max 1934.3) | ||
| +v2 fromNullable(x) ?? 0 (no object) median 1115.4 ns/op (min 1112.4, max 1193.3) | ||
| +plain RANK[cls] ?? 2 (lookup table) median 4805.1 ns/op (min 4775.9, max 4821.8) | ||
| +plain switch (cls) median 933.6 ns/op (min 933.4, max 1004.5) | ||
| +v1 match macro, literal arms (__tsus_match_lit) median 47445.2 ns/op (min 47167.1, max 52489.4) | ||
| +v2 matchLiteral, arms inline median 13813.3 ns/op (min 13689.3, max 14506.3) | ||
| +v2 matchLiteral, arms hoisted to a const median 6016.2 ns/op (min 5999.1, max 6089.9) | ||
| +(sink 0) | ||
| +node v24.17.0, darwin arm64 | ||
Addedbench/oncall/old.ts+39−0
| @@ -0,0 +1,39 @@ | ||
| +// Before the pilot: jobFor and the dedupe test as plain TypeScript, from Rebasely | ||
| +// ops/oncall/lib/watcher.ts at fcf3b6a9 (the dedupe test was inline in Watcher.enqueue). | ||
| +import type { ApiEvent, HandledEntry, Job, WatchConfig } from "./types.ts"; | ||
| + | ||
| +const RANK: Record<string, number> = { critical: 0, page: 1, ticket: 2, direct: 3 }; | ||
| +const parse = (s: string | null): Record<string, unknown> => { | ||
| + if (!s) return {}; | ||
| + try { | ||
| + const v = JSON.parse(s) as unknown; | ||
| + return v && typeof v === "object" && !Array.isArray(v) ? (v as Record<string, unknown>) : {}; | ||
| + } catch { | ||
| + return {}; | ||
| + } | ||
| +}; | ||
| + | ||
| +export const TRIAGE_ONLY_PREFIXES = ["bug_report:", "bug_report_reply:", "bug_reports:", "ux_issues:"] as const; | ||
| + | ||
| +export function jobFor(e: ApiEvent, cfg: WatchConfig, now: number): Job | null { | ||
| + if (e.environment && !cfg.environments.includes(e.environment)) return null; | ||
| + const data = parse(e.data); | ||
| + if ((e.kind === "alert_opened" || e.kind === "alert_severity_rose") && e.key) { | ||
| + if (e.key.startsWith("oncall:") && !e.key.startsWith("oncall:drill:")) return null; | ||
| + if (TRIAGE_ONLY_PREFIXES.some((p) => e.key!.startsWith(p))) return null; | ||
| + const cls = typeof data.class === "string" ? data.class : "ticket"; | ||
| + if (cls === "direct") return null; | ||
| + return { id: `e${e.id}`, kind: "incident", key: e.key, alertKey: e.key, incident: e.kind === "alert_opened" ? e.at : null, rank: RANK[cls] ?? 2, event: { ...e, data }, queuedAt: now }; | ||
| + } | ||
| + if (e.kind === "abuse_flag") { | ||
| + const subject = typeof data.subject_id === "string" ? data.subject_id : String(e.id); | ||
| + return { id: `e${e.id}`, kind: "incident", key: `abuse:${subject}`, alertKey: null, incident: null, rank: 2, event: { ...e, data }, queuedAt: now }; | ||
| + } | ||
| + return null; | ||
| +} | ||
| + | ||
| +export const isDuplicate = (handled: Record<string, HandledEntry>, j: Job, now: number, dedupeHours: number): boolean => { | ||
| + const h = handled[j.key]; | ||
| + const fresh = h && now - h.at < dedupeHours * 3_600_000; | ||
| + return Boolean(fresh && (j.incident === null || h.incident === null || j.incident <= h.at)); | ||
| +}; | ||
Addedbench/oncall/run.ts+116−0
| @@ -0,0 +1,116 @@ | ||
| +// The on-call micro benchmark, rebuilt from Rebasely docs/v2/tsus-pilot.md (its scripts were not | ||
| +// committed): jobFor over a realistic page of 1,000 events and the dedupe test over the jobs it | ||
| +// yields, in three versions: before the pilot (plain .ts), the pilot (TSUS v1, wrapper objects), | ||
| +// and TSUS v2 (Option is T | undefined). A differential check first proves all three agree. | ||
| +// | ||
| +// node bench/oncall/run.ts (node 22.6+ strips the types itself) | ||
| +import * as old from "./old.ts"; | ||
| +import * as v1 from "./v1.ts"; | ||
| +import * as v2 from "./v2/rules.ts"; | ||
| +// @ts-expect-error -- compiled JS without declarations | ||
| +import { Some as v1Some, __tsus_match_lit } from "./v1/_runtime/index.js"; | ||
| +// @ts-expect-error -- compiled JS without declarations | ||
| +import { fromNullable as v1FromNullable } from "./v1/_runtime/boundary.js"; | ||
| +import { matchLiteral, fromNullable as v2FromNullable } from "@tsus/v2"; | ||
| +import type { ApiEvent, HandledEntry, Job, WatchConfig } from "./types.ts"; | ||
| + | ||
| +// A seeded generator (mulberry32), so every run sees the same events. | ||
| +const rng = (seed: number) => () => { | ||
| + let t = (seed += 0x6d2b79f5); | ||
| + t = Math.imul(t ^ (t >>> 15), t | 1); | ||
| + t ^= t + Math.imul(t ^ (t >>> 7), t | 61); | ||
| + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; | ||
| +}; | ||
| +const pick = <T,>(r: () => number, xs: readonly T[]): T => xs[Math.floor(r() * xs.length)] as T; | ||
| + | ||
| +const KINDS = ["alert_opened", "alert_opened", "alert_opened", "alert_opened", "alert_severity_rose", "alert_resolved", "alert_resolved", "alert_resolved", "abuse_flag", "milestone", "heartbeat"]; | ||
| +const PREFIXES = ["deploy:", "d1:", "queue:", "probe:", "oncall:", "oncall:drill:", "bug_report:", "ux_issues:", "billing:"]; | ||
| +const CLASSES = ["critical", "page", "ticket", "ticket", "ticket", "direct"]; | ||
| +const ENVS = ["production", "production", "production", "staging", null, ""]; | ||
| + | ||
| +const makeEvents = (n: number, seed: number): ApiEvent[] => { | ||
| + const r = rng(seed); | ||
| + return Array.from({ length: n }, (_, i): ApiEvent => { | ||
| + const kind = pick(r, KINDS); | ||
| + const roll = r(); | ||
| + const data = | ||
| + roll < 0.05 ? null : roll < 0.08 ? "{not json" : roll < 0.1 ? "" : JSON.stringify(kind === "abuse_flag" ? { subject_id: r() < 0.8 ? `org_${Math.floor(r() * 40)}` : 7, reason: "spam" } : { class: r() < 0.9 ? pick(r, CLASSES) : 3, severity: "error", title: `thing ${i} broke`, body: "x".repeat(Math.floor(r() * 200)) }); | ||
| + return { id: 1000 + i, at: 1_760_000_000_000 + i * 1000, kind, key: r() < 0.05 ? null : r() < 0.03 ? "" : `${pick(r, PREFIXES)}${Math.floor(r() * 60)}`, environment: pick(r, ENVS), actor: null, data }; | ||
| + }); | ||
| +}; | ||
| + | ||
| +const cfg: WatchConfig = { environments: ["production"], dedupeHours: 6 }; | ||
| +const NOW = 1_760_000_500_000; | ||
| +const norm = (j: unknown) => JSON.stringify(j, (_k, v) => (v === undefined ? null : v)); | ||
| + | ||
| +// 1. Differential: the three versions agree on 20,000 events, and on the dedupe test. | ||
| +const many = makeEvents(20_000, 7); | ||
| +let disagreements = 0; | ||
| +for (const e of many) { | ||
| + const a = norm(old.jobFor(e, cfg, NOW)); | ||
| + if (a !== norm(v1.jobFor(e, cfg, NOW)) || a !== norm(v2.jobFor(e, cfg, NOW)) || a !== norm(v2.jobForSwitch(e, cfg, NOW))) disagreements += 1; | ||
| +} | ||
| +const page = makeEvents(1000, 42); | ||
| +const jobs = page.map((e) => old.jobFor(e, cfg, NOW)).filter((j): j is Job => j !== null); | ||
| +const handled: Record<string, HandledEntry> = {}; | ||
| +const hr = rng(3); | ||
| +for (const j of jobs) if (hr() < 0.5) handled[j.key] = { at: NOW - Math.floor(hr() * 10) * 3_600_000, incident: hr() < 0.5 ? null : NOW - 4_000_000 }; | ||
| +const v2Jobs = page.map((e) => v2.jobFor(e, cfg, NOW)).filter((j): j is v2.Job => j !== undefined); | ||
| +for (let i = 0; i < jobs.length; i += 1) { | ||
| + const j = jobs[i] as Job; | ||
| + const k = v2Jobs[i] as v2.Job; | ||
| + const a = old.isDuplicate(handled, j, NOW, 6); | ||
| + if (a !== v1.isDuplicate(handled, j, NOW, 6) || a !== v2.isDuplicate(v2FromNullable(handled[k.key]), k.incident, NOW, 6)) disagreements += 1; | ||
| +} | ||
| + | ||
| +// 2. Timing: median of 7 runs, each long enough to be stable. | ||
| +let sink = 0; | ||
| +const time = (label: string, ops: number, f: () => number): number => { | ||
| + for (let i = 0; i < 3; i += 1) sink += f(); | ||
| + const runs: number[] = []; | ||
| + for (let r = 0; r < 7; r += 1) { | ||
| + const t0 = process.hrtime.bigint(); | ||
| + for (let i = 0; i < ops; i += 1) sink += f(); | ||
| + runs.push(Number(process.hrtime.bigint() - t0) / ops); | ||
| + } | ||
| + runs.sort((a, b) => a - b); | ||
| + const median = runs[3] as number; | ||
| + console.log(`${label.padEnd(62)} median ${median.toFixed(1).padStart(10)} ns/op (min ${(runs[0] as number).toFixed(1)}, max ${(runs[6] as number).toFixed(1)})`); | ||
| + return median; | ||
| +}; | ||
| + | ||
| +const pageOld = () => { let n = 0; for (const e of page) if (old.jobFor(e, cfg, NOW) !== null) n += 1; return n; }; | ||
| +const pageV1 = () => { let n = 0; for (const e of page) if (v1.jobFor(e, cfg, NOW) !== null) n += 1; return n; }; | ||
| +const pageV2 = () => { let n = 0; for (const e of page) if (v2.jobFor(e, cfg, NOW) !== undefined) n += 1; return n; }; | ||
| +const pageV2s = () => { let n = 0; for (const e of page) if (v2.jobForSwitch(e, cfg, NOW) !== undefined) n += 1; return n; }; | ||
| +const dedupeOld = () => { let n = 0; for (const j of jobs) if (old.isDuplicate(handled, j, NOW, 6)) n += 1; return n; }; | ||
| +const dedupeV1 = () => { let n = 0; for (const j of jobs) if (v1.isDuplicate(handled, j, NOW, 6)) n += 1; return n; }; | ||
| +const dedupeV2 = () => { let n = 0; for (const j of v2Jobs) if (v2.isDuplicate(v2FromNullable(handled[j.key]), j.incident, NOW, 6)) n += 1; return n; }; | ||
| + | ||
| +console.log(`events: page of ${page.length}, ${jobs.length} jobs; differential over ${many.length} events and ${jobs.length} dedupe tests: ${disagreements} disagreements`); | ||
| +const o1 = time("before (plain .ts) jobFor, one page of 1000 events", 300, pageOld); | ||
| +const p1 = time("v1 (pilot) jobFor: classify in tsus + .ts adapter", 300, pageV1); | ||
| +const q1 = time("v2 jobFor: Option is T | undefined, no adapter", 300, pageV2); | ||
| +const s1 = time("v2 jobFor, rankOf as an exhaustive switch", 300, pageV2s); | ||
| +console.log(` -> per event: before ${(o1 / 1000).toFixed(0)} ns, v1 ${(p1 / 1000).toFixed(0)} ns (${(p1 / o1).toFixed(2)}x), v2 ${(q1 / 1000).toFixed(0)} ns (${(q1 / o1).toFixed(2)}x), v2 with switch ${(s1 / 1000).toFixed(0)} ns (${(s1 / o1).toFixed(2)}x)`); | ||
| +const o2 = time(`before dedupe test (inline), ${jobs.length} jobs`, 20_000, dedupeOld); | ||
| +const p2 = time(`v1 isDuplicate(fromNullable(...)), ${jobs.length} jobs`, 20_000, dedupeV1); | ||
| +const q2 = time(`v2 isDuplicate(fromNullable(...)), ${jobs.length} jobs`, 20_000, dedupeV2); | ||
| +console.log(` -> per job: before ${(o2 / jobs.length).toFixed(1)} ns, v1 ${(p2 / jobs.length).toFixed(1)} ns (${(p2 / o2).toFixed(2)}x), v2 ${(q2 / jobs.length).toFixed(1)} ns (${(q2 / o2).toFixed(2)}x)`); | ||
| + | ||
| +console.log("Primitives (1000 per op):"); | ||
| +const xs = Array.from({ length: 1000 }, (_, i) => (i % 3 === 0 ? null : i)); | ||
| +const cls = Array.from({ length: 1000 }, (_, i) => (["critical", "page", "ticket", "direct"] as const)[i % 4] as "critical" | "page" | "ticket" | "direct"); | ||
| +time("plain x ?? 0", 20_000, () => { let n = 0; for (const x of xs) n += x ?? 0; return n; }); | ||
| +time("v1 fromNullable(x).unwrapOr(0) (Some: object + 8 closures)", 20_000, () => { let n = 0; for (const x of xs) n += v1FromNullable(x).unwrapOr(0); return n; }); | ||
| +time("v1 Some(x).value", 20_000, () => { let n = 0; for (const x of xs) n += v1Some(x ?? 0).value; return n; }); | ||
| +time("v2 fromNullable(x) ?? 0 (no object)", 20_000, () => { let n = 0; for (const x of xs) n += v2FromNullable(x) ?? 0; return n; }); | ||
| +const RANK: Record<string, number> = { critical: 0, page: 1, ticket: 2, direct: 3 }; | ||
| +time("plain RANK[cls] ?? 2 (lookup table)", 20_000, () => { let n = 0; for (const c of cls) n += RANK[c] ?? 2; return n; }); | ||
| +time("plain switch (cls)", 20_000, () => { let n = 0; for (const c of cls) { switch (c) { case "critical": n += 0; break; case "page": n += 1; break; case "ticket": n += 2; break; case "direct": n += 3; break; } } return n; }); | ||
| +time("v1 match macro, literal arms (__tsus_match_lit)", 20_000, () => { let n = 0; for (const c of cls) n += __tsus_match_lit(c)([["critical", () => 0], ["page", () => 1], ["ticket", () => 2], ["direct", () => 3]], () => 2); return n; }); | ||
| +time("v2 matchLiteral, arms inline", 20_000, () => { let n = 0; for (const c of cls) n += matchLiteral(c, { critical: () => 0, page: () => 1, ticket: () => 2, direct: () => 3 }); return n; }); | ||
| +const ARMS = { critical: () => 0, page: () => 1, ticket: () => 2, direct: () => 3 } as const; | ||
| +time("v2 matchLiteral, arms hoisted to a const", 20_000, () => { let n = 0; for (const c of cls) n += matchLiteral(c, ARMS); return n; }); | ||
| +console.log(`(sink ${sink % 7})`); | ||
| +console.log(`node ${process.version}, ${process.platform} ${process.arch}`); | ||
Addedbench/oncall/types.ts+16−0
| @@ -0,0 +1,16 @@ | ||
| +// The on-call watcher's API and job types, as Rebasely's ops/oncall/lib declares them (null is the | ||
| +// API's and the state file's own representation). | ||
| +export type ApiEvent = { id: number; at: number; kind: string; key: string | null; environment: string | null; actor: string | null; data: string | null }; | ||
| +export type JobKind = "incident" | "api_unreachable" | "digest"; | ||
| +export type Job = { | ||
| + id: string; | ||
| + kind: JobKind; | ||
| + key: string; | ||
| + alertKey: string | null; | ||
| + incident: number | null; | ||
| + rank: number; | ||
| + event: Record<string, unknown> | null; | ||
| + queuedAt: number; | ||
| +}; | ||
| +export type HandledEntry = { at: number; incident: number | null }; | ||
| +export type WatchConfig = { readonly environments: readonly string[]; readonly dedupeHours: number }; | ||
Addedbench/oncall/v1/_runtime/boundary.js+32−0
| @@ -0,0 +1,32 @@ | ||
| +// The interop boundary: converts the outside world's nulls, undefineds, and | ||
| +// throws into Option and Result at the edge of tsus code. | ||
| +import { Some, None, Ok, Err } from "./index.js"; | ||
| + | ||
| +export const fromNullable = (value) => | ||
| + value === null || value === undefined ? None : Some(value); | ||
| + | ||
| +export const wrapSync = (fn) => (...args) => { | ||
| + try { | ||
| + return Ok(fn(...args)); | ||
| + } catch (error) { | ||
| + return Err(error); | ||
| + } | ||
| +}; | ||
| + | ||
| +export const wrapAsync = (fn) => async (...args) => { | ||
| + try { | ||
| + return Ok(await fn(...args)); | ||
| + } catch (error) { | ||
| + return Err(error); | ||
| + } | ||
| +}; | ||
| + | ||
| +// Narrowing helpers for values typed unknown (parsed JSON, request bodies). | ||
| +// They exist so .tsus code can inspect outside data without `as` casts | ||
| +// (TSUS402) or null checks (TSUS002). | ||
| +export const isRecord = (value) => typeof value === "object" && value !== null && !Array.isArray(value); | ||
| + | ||
| +export const asRecord = (value) => (isRecord(value) ? Some(value) : None); | ||
| + | ||
| +export const field = (value, key) => | ||
| + isRecord(value) && Object.hasOwn(value, key) ? fromNullable(value[key]) : None; | ||
Addedbench/oncall/v1/_runtime/index.js+117−0
| @@ -0,0 +1,117 @@ | ||
| +// The tsus runtime. Plain ESM, no classes, usable from Node and Cloudflare Workers. | ||
| + | ||
| +export const Some = (value) => ({ | ||
| + tag: "Some", | ||
| + value, | ||
| + isSome: () => true, | ||
| + isNone: () => false, | ||
| + map: (f) => Some(f(value)), | ||
| + andThen: (f) => f(value), | ||
| + unwrapOr: () => value, | ||
| + okOr: () => Ok(value), | ||
| +}); | ||
| + | ||
| +export const None = Object.freeze({ | ||
| + tag: "None", | ||
| + isSome: () => false, | ||
| + isNone: () => true, | ||
| + map: () => None, | ||
| + andThen: () => None, | ||
| + unwrapOr: (fallback) => fallback, | ||
| + okOr: (error) => Err(error), | ||
| +}); | ||
| + | ||
| +export const Ok = (value) => ({ | ||
| + tag: "Ok", | ||
| + value, | ||
| + isOk: () => true, | ||
| + isErr: () => false, | ||
| + map: (f) => Ok(f(value)), | ||
| + mapErr: () => Ok(value), | ||
| + andThen: (f) => f(value), | ||
| + unwrapOr: () => value, | ||
| + ok: () => Some(value), | ||
| +}); | ||
| + | ||
| +export const Err = (error) => ({ | ||
| + tag: "Err", | ||
| + error, | ||
| + isOk: () => false, | ||
| + isErr: () => true, | ||
| + map: () => Err(error), | ||
| + mapErr: (f) => Err(f(error)), | ||
| + andThen: () => Err(error), | ||
| + unwrapOr: (fallback) => fallback, | ||
| + ok: () => None, | ||
| +}); | ||
| + | ||
| +export const tag = (t, value) => ({ tag: t, value }); | ||
| + | ||
| +// --- compiled forms of the match macro --- | ||
| +// | ||
| +// The preprocessor lowers `match (subject) { ... }` to one of these, curried | ||
| +// so TypeScript fixes the subject's type before it checks the arms: | ||
| +// | ||
| +// __tsus_match(subject)({ Some: (x) => ..., None: () => ... }) | ||
| +// __tsus_match_or(subject)({ Some: (x) => ... }, () => wildcard) | ||
| +// __tsus_match_lit(subject)([["/", () => ...], ...], () => wildcard) | ||
| +// | ||
| +// A variant arm receives the variant's payload: `value` if it has one (Some, | ||
| +// Ok, Tagged), else `error` (Err), else the variant object itself. | ||
| + | ||
| +const payload = (v) => ("value" in v ? v.value : "error" in v ? v.error : v); | ||
| + | ||
| +const noArm = (value) => | ||
| + new Error(`TSUS100: non-exhaustive match, no arm matched ${JSON.stringify(value)}. Add a _ arm.`); | ||
| + | ||
| +const dispatch = (value, arms, otherwise) => { | ||
| + const t = value !== null && typeof value === "object" ? value.tag : undefined; | ||
| + if (typeof t === "string" && Object.hasOwn(arms, t)) { | ||
| + return arms[t](payload(value)); | ||
| + } | ||
| + if (otherwise) { | ||
| + return otherwise(value); | ||
| + } | ||
| + throw noArm(value); | ||
| +}; | ||
| + | ||
| +export const __tsus_match = (value) => (arms) => dispatch(value, arms); | ||
| + | ||
| +export const __tsus_match_or = (value) => (arms, otherwise) => dispatch(value, arms, otherwise); | ||
| + | ||
| +export const __tsus_match_lit = (value) => (cases, otherwise) => { | ||
| + for (const [literal, handler] of cases) { | ||
| + if (value === literal) { | ||
| + return handler(); | ||
| + } | ||
| + } | ||
| + if (otherwise) { | ||
| + return otherwise(value); | ||
| + } | ||
| + throw noArm(value); | ||
| +}; | ||
| + | ||
| +// --- compiled form of the ? operator --- | ||
| +// | ||
| +// const x = load()?; | ||
| +// becomes | ||
| +// const t = __tsus_try(load()); if (__tsus_isBail(t)) return __tsus_bail(t); const x = t.value; | ||
| +// | ||
| +// __tsus_try and __tsus_bail are identities at runtime; they exist for the | ||
| +// type checker. An Err or None carries no payload of the success type, so the | ||
| +// same object is returned as-is from the enclosing function. | ||
| + | ||
| +export const __tsus_try = (value) => value; | ||
| + | ||
| +export const __tsus_isBail = (value) => value.tag === "Err" || value.tag === "None"; | ||
| + | ||
| +export const __tsus_bail = (value) => value; | ||
| + | ||
| +// Emitted only for a ? the compiler already rejected (TSUS300/TSUS303), so a | ||
| +// checked build never contains it; reaching it means the checker was bypassed. | ||
| +export const __tsus_unwrap = (value) => { | ||
| + if (value.tag === "Ok" || value.tag === "Some") { | ||
| + return value.value; | ||
| + } | ||
| + throw new Error(`TSUS300: ? propagated ${value.tag} outside a Result/Option function`); | ||
| +}; | ||
Addedbench/oncall/v1/rules.js+223−0
| @@ -0,0 +1,223 @@ | ||
| +import { __tsus_match_lit, __tsus_match } from "./_runtime/index.js"; // The on-call watcher's decision rules, in TypeScript-Ultra-Strict (tsus; docs/v2/tsus-pilot.md): | ||
| +// which events are actionable and how urgent, dedupe per incident, the queue's order and cap, the | ||
| +// hourly rate limit, the skip and escalation rules after a run, the API outage run and the morning | ||
| +// digest's schedule and fallback text. Everything here is pure: lib/watcher.ts (plain .ts) does the | ||
| +// I/O, holds the state, and converts at the boundary (null in the API's and the state file's types | ||
| +// is read through fromNullable here; Option comes back and the .ts side turns it into null). | ||
| +// | ||
| +// Built by `npm run tsus:build` into ops/oncall/dist (committed: the watcher runs it with plain node). | ||
| +import { Some, None } from "./_runtime/index.js"; | ||
| +import { field, fromNullable, isRecord, wrapSync } from "./_runtime/boundary.js"; | ||
| +export const DEFAULT_CONFIG = { | ||
| + environments: ["production"], | ||
| + pollMs: 20_000, | ||
| + heartbeatMs: 60_000, | ||
| + runTimeoutMs: 15 * 60_000, | ||
| + maxRunsPerHour: 12, | ||
| + dedupeHours: 6, | ||
| + apiDownMs: 3 * 60_000, | ||
| + digest: true, | ||
| + digestHour: 9, | ||
| + digestTz: "America/Los_Angeles", | ||
| + queueMax: 50, | ||
| +}; | ||
| +const HOUR = 3_600_000; | ||
| +const DAY = 24 * HOUR; | ||
| +// Alerts that never start a run: user feedback (bug reports, a reporter's reply, bursts of reports, | ||
| +// the daily user-facing issue digest). Their text comes from users and is untrusted, and the owner's | ||
| +// rule is that triage of it never makes an agent write code: they are logged and reach the morning | ||
| +// digest, which acts on nothing (DIGEST.md). The owner or an agent the owner starts reads them with | ||
| +// scripts/bug-reports.ts (.rebasely/knowledge/guides/bug-triage.md). | ||
| +export const TRIAGE_ONLY_PREFIXES = ["bug_report:", "bug_report_reply:", "bug_reports:", "ux_issues:"]; | ||
| +// --- classification and severity --- | ||
| +// The queue order of an alert class: critical first, then pages, then tickets. An unknown class | ||
| +// ranks as a ticket. | ||
| +export const rankOf = (cls) => __tsus_match_lit(cls)([ | ||
| + ["critical", () => (0)], | ||
| + ["page", () => (1)], | ||
| + ["ticket", () => (2)], | ||
| + ["direct", () => (3)], | ||
| +], () => (2)); | ||
| +const parseJson = wrapSync(JSON.parse); | ||
| +// An event's data column: a JSON object, or {} for anything else. | ||
| +export const eventData = (raw) => __tsus_match(raw)({ | ||
| + Some: (text) => ((text === "" ? {} : parseJson(text).ok().andThen(recordOf).unwrapOr({}))), | ||
| + None: () => (({})), | ||
| +}); | ||
| +const recordOf = (v) => (isRecord(v) ? Some(v) : None); | ||
| +// The API's columns are null or a string; an empty string counts as absent, as it always has. | ||
| +const present = (s) => s.andThen((v) => (v === "" ? None : Some(v))); | ||
| +// An event without an environment is not filtered out. | ||
| +const watched = (environment, environments) => __tsus_match(present(environment))({ | ||
| + Some: (env) => (environments.includes(env)), | ||
| + None: () => (true), | ||
| +}); | ||
| +// The on-call system's own alerts (the agent's tickets, the watcher-offline page) are not runs. | ||
| +// Drills are. | ||
| +const ownAlert = (key) => key.startsWith("oncall:") && !key.startsWith("oncall:drill:"); | ||
| +const triageOnly = (key) => TRIAGE_ONLY_PREFIXES.some((p) => key.startsWith(p)); | ||
| +const alertJob = (e, key, data) => { | ||
| + if (ownAlert(key) || triageOnly(key)) | ||
| + return None; | ||
| + const cls = typeof data.class === "string" ? data.class : "ticket"; | ||
| + if (cls === "direct") | ||
| + return None; | ||
| + return Some({ key, alertKey: Some(key), incident: e.kind === "alert_opened" ? Some(e.at) : None, rank: rankOf(cls), data }); | ||
| +}; | ||
| +// The org's suspension ticket is abuse:<orgId>: the flag and the ticket are one incident. | ||
| +const abuseJob = (e, data) => { | ||
| + const subject = typeof data.subject_id === "string" ? data.subject_id : String(e.id); | ||
| + return { key: `abuse:${subject}`, alertKey: None, incident: None, rank: 2, data }; | ||
| +}; | ||
| +// Which events start a run: an alert opened or whose severity rose, and an abuse flag, in the | ||
| +// watched environments. | ||
| +export const classify = (e, environments) => { | ||
| + if (!watched(fromNullable(e.environment), environments)) | ||
| + return None; | ||
| + const data = eventData(fromNullable(e.data)); | ||
| + const key = present(fromNullable(e.key)); | ||
| + if ((e.kind === "alert_opened" || e.kind === "alert_severity_rose") && key.isSome()) | ||
| + return alertJob(e, key.value, data); | ||
| + if (e.kind === "abuse_flag") | ||
| + return Some(abuseJob(e, data)); | ||
| + return None; | ||
| +}; | ||
| +// The first poll picks up the open alerts nobody has acknowledged. | ||
| +export const pickUpOpenAlert = (a, environments) => present(fromNullable(a.ack_by)).isNone() && environments.includes(a.environment); | ||
| +// --- dedupe and the queue --- | ||
| +// A job without an incident time, or a handled run without one, is the same incident; otherwise an | ||
| +// incident that opened after the key was handled is a new one. | ||
| +const sameIncident = (incident, handled) => __tsus_match(incident)({ | ||
| + Some: (opened) => (fromNullable(handled.incident).isNone() || opened <= handled.at), | ||
| + None: () => (true), | ||
| +}); | ||
| +// Whether a key handled within dedupeHours makes this job a repeat of the same incident. | ||
| +export const isDuplicate = (handled, incident, now, dedupeHours) => __tsus_match(handled)({ | ||
| + Some: (h) => (now - h.at < dedupeHours * HOUR && sameIncident(incident, h)), | ||
| + None: () => (false), | ||
| +}); | ||
| +export const alreadyQueued = (key, queued, running) => { | ||
| + const isRunning = __tsus_match(running)({ | ||
| + Some: (j) => (j.key === key), | ||
| + None: () => (false), | ||
| + }); | ||
| + return isRunning || queued.some((q) => q.key === key); | ||
| +}; | ||
| +export const queueFull = (length, queueMax) => length >= queueMax; | ||
| +export const queueFullMessage = (queueMax, key) => `The on-call agent's queue is full (${queueMax} events); it did not look at ${key}.`; | ||
| +// Critical first, then pages, then the rest, oldest first. | ||
| +export const byPriority = (a, b) => a.rank - b.rank || a.queuedAt - b.queuedAt; | ||
| +// Handled keys are kept for 7 days. | ||
| +export const pruneHandled = (handled, now) => { | ||
| + const cutoff = now - 7 * DAY; | ||
| + return Object.fromEntries(Object.entries(handled).filter(([, v]) => !(v.at < cutoff))); | ||
| +}; | ||
| +// --- the rate limit --- | ||
| +export const recentRuns = (runs, now) => runs.filter((t) => now - t < HOUR); | ||
| +// Past maxRunsPerHour runs in the last hour an incident is not run: a ticket waits for the digest, | ||
| +// anything else reaches the owner. The digest is not counted and never limited. | ||
| +export const rateDecision = (job, runsLastHour, maxRunsPerHour) => { | ||
| + if (job.kind === "digest" || runsLastHour < maxRunsPerHour) | ||
| + return "run"; | ||
| + return present(fromNullable(job.alertKey)).isSome() && job.rank >= 2 ? "wait_for_digest" : "escalate"; | ||
| +}; | ||
| +export const rateLimitMessage = (maxRunsPerHour, title) => `The on-call agent reached its limit of ${maxRunsPerHour} runs an hour, so nobody is looking at this: ${title}`; | ||
| +// The alert's title from the event, else the job's key. | ||
| +export const jobTitle = (job) => __tsus_match(fromNullable(job.event).andThen((e) => field(e, "data")).andThen((d) => field(d, "title")))({ | ||
| + Some: (t) => ((typeof t === "string" ? t : job.key)), | ||
| + None: () => (job.key), | ||
| +}); | ||
| +// --- skip and escalation rules around a run --- | ||
| +// An alert that resolved before its run starts is not run, unless it was critical. | ||
| +export const skipResolved = (job, alert) => { | ||
| + const resolved = __tsus_match(alert)({ | ||
| + Some: (a) => (a.state === "resolved"), | ||
| + None: () => (false), | ||
| + }); | ||
| + return resolved && job.kind === "incident" && job.rank > 0; | ||
| +}; | ||
| +// The API unreachable for apiDownMs starts one run about it. | ||
| +export const outageRunDue = (alreadyRun, downSince, now, apiDownMs) => !alreadyRun && now - downSince >= apiDownMs; | ||
| +// Stopped by the watcher's own shutdown (launchctl, uninstall): not the agent's failure. An open | ||
| +// page still escalates on the server's timer. | ||
| +export const stoppedByShutdown = (stopped, out) => stopped && !out.timedOut; | ||
| +const failure = (out, runId, timeoutMs) => { | ||
| + if (out.timedOut) | ||
| + return Some(`The on-call agent did not finish within ${Math.round(timeoutMs / 60_000)} min (run ${runId}), so it was stopped.`); | ||
| + if (out.exitCode !== 0) { | ||
| + const code = fromNullable(out.exitCode).map((c) => String(c)).unwrapOr(fromNullable(out.signal).unwrapOr("?")); | ||
| + return Some(`The on-call agent's run ${runId} failed (exit ${code}).`); | ||
| + } | ||
| + return None; | ||
| +}; | ||
| +// Claude posted its summary on the alert during this run (act.ts note). | ||
| +const reportedSince = (a, startedAt) => fromNullable(a.note_by).unwrapOr("") === "claude" && fromNullable(a.note_at).unwrapOr(0) >= startedAt; | ||
| +// After a run: a timeout, a crash (non-zero exit), or a run that ended without its summary on an | ||
| +// open alert is a failure. The run's final message goes on the alert as a note, so the owner and | ||
| +// the digest see it even when the agent could not post it. A ticket is never escalated for it (the | ||
| +// policy: tickets are Claude's and the digest's; only Claude itself may escalate one); pages, | ||
| +// critical alerts and alert-less jobs (abuse flags, the API outage) are. | ||
| +// `alert` is the alert as it is after the run (None: no alert key, or the API did not answer). | ||
| +export const verdict = (job, out, alert, startedAt, runId, timeoutMs) => { | ||
| + const alertKey = present(fromNullable(job.alertKey)); | ||
| + const why = failure(out, runId, timeoutMs); | ||
| + if (why.isNone()) { | ||
| + if (alertKey.isNone()) | ||
| + return { tag: "Fine" }; | ||
| + const settled = __tsus_match(alert)({ | ||
| + Some: (a) => (reportedSince(a, startedAt) || a.state !== "open"), | ||
| + None: () => (true), | ||
| + }); | ||
| + if (settled) | ||
| + return { tag: "Fine" }; | ||
| + } | ||
| + const where = out.logPath ? ` Log: ${out.logPath}` : ""; | ||
| + const message = `${why.unwrapOr(`The on-call agent's run ${runId} ended without posting its summary to the alert.`)}${where}`; | ||
| + const found = out.result.trim(); | ||
| + const note = alertKey.isSome() && alert.isSome() ? Some(`${message}${found ? `\nThe run's final message: ${found}` : ""}`.slice(0, 2000)) : None; | ||
| + const fallbackClass = job.rank >= 2 && alertKey.isSome() ? "ticket" : ""; | ||
| + const cls = __tsus_match(alert)({ | ||
| + Some: (a) => (fromNullable(a.class).unwrapOr(fallbackClass)), | ||
| + None: () => (fallbackClass), | ||
| + }); | ||
| + return { tag: "Failed", message, note, escalate: cls !== "ticket" && cls !== "direct" }; | ||
| +}; | ||
| +// --- the morning digest --- | ||
| +// The local date and hour in a time zone (the digest's 09:00 America/Los_Angeles). | ||
| +export const localParts = (t, tz) => { | ||
| + const fmt = new Intl.DateTimeFormat("en-CA", { timeZone: tz, year: "numeric", month: "2-digit", day: "2-digit", hour: "2-digit", hourCycle: "h23" }); | ||
| + const p = Object.fromEntries(fmt.formatToParts(new Date(t)).map((x) => [x.type, x.value])); | ||
| + return { day: `${p.year}-${p.month}-${p.day}`, hour: Number(p.hour) }; | ||
| +}; | ||
| +// Due from digestHour until 12 hours later, once per local day: a Mac that slept through 09:00 sends | ||
| +// it when it wakes, but not in the evening. | ||
| +export const digestDue = (now, last, cfg) => { | ||
| + if (!cfg.digest) | ||
| + return None; | ||
| + const { day, hour } = localParts(now, cfg.digestTz); | ||
| + const sentToday = __tsus_match(last)({ | ||
| + Some: (d) => (d === day), | ||
| + None: () => (false), | ||
| + }); | ||
| + return hour >= cfg.digestHour && hour < cfg.digestHour + 12 && !sentToday ? Some(day) : None; | ||
| +}; | ||
| +// The digest Claude wrote, when the run succeeded with more than a line. | ||
| +export const digestText = (out) => out.exitCode === 0 && !out.timedOut && out.result.trim().length > 20 ? Some(out.result.trim()) : None; | ||
| +const list = (v) => (Array.isArray(v) ? v : []); | ||
| +const record = (v) => (isRecord(v) ? v : {}); | ||
| +const prop = (v, k) => (isRecord(v) ? String(v[k]) : "undefined"); | ||
| +// A plain digest from the API's summary, for when the Claude run cannot write one. | ||
| +export const fallbackDigest = (s, day) => { | ||
| + const open = list(s.open); | ||
| + const opened = list(s.opened); | ||
| + const events = record(s.events); | ||
| + const count = (k) => String(fromNullable(events[k]).unwrapOr(0)); | ||
| + const accounts = Object.entries(record(s.counts)).map(([env, c]) => `Accounts (${env}): ${Object.entries(record(c)).filter(([k]) => k !== "at").map(([k, v]) => `${k} ${String(v)}`).join(", ")}`); | ||
| + return [ | ||
| + `Rebasely, last 24 hours to ${day} (the on-call agent could not write the digest; these are the API's numbers).`, | ||
| + ...accounts, | ||
| + `Alerts opened or resolved: ${opened.length}${opened.length ? ` (${opened.slice(0, 8).map((a) => `${prop(a, "key")} ${prop(a, "state")}`).join("; ")})` : ""}`, | ||
| + `Open now: ${open.length}${open.length ? ` (${open.slice(0, 8).map((a) => prop(a, "key")).join(", ")})` : ""}`, | ||
| + `Escalations: ${count("escalated")}. Abuse flags: ${count("abuse_flag")}. Milestones: ${count("milestone")}.`, | ||
| + ].join("\n"); | ||
| +}; | ||
Addedbench/oncall/v1.ts+20−0
| @@ -0,0 +1,20 @@ | ||
| +// The pilot (TSUS v1): the rules compiled from ops/oncall/src/rules.tsus at Rebasely c5a11d4f | ||
| +// (v1/rules.js and its runtime, copied unchanged), with the .ts adapter lib/watcher.ts uses. | ||
| +import type { ApiEvent, HandledEntry, Job, WatchConfig } from "./types.ts"; | ||
| +// @ts-expect-error -- compiled JS without declarations | ||
| +import { classify, isDuplicate as isDuplicateRule } from "./v1/rules.js"; | ||
| +// @ts-expect-error -- compiled JS without declarations | ||
| +import { fromNullable } from "./v1/_runtime/boundary.js"; | ||
| + | ||
| +type V1Option<T> = { isSome(): boolean; value: T; unwrapOr(f: T | null): T | null }; | ||
| +const orNull = <T>(o: V1Option<T>): T | null => o.unwrapOr(null); | ||
| + | ||
| +export function jobFor(e: ApiEvent, cfg: WatchConfig, now: number): Job | null { | ||
| + const c = classify(e, cfg.environments); | ||
| + if (!c.isSome()) return null; | ||
| + const v = c.value; | ||
| + return { id: `e${e.id}`, kind: "incident", key: v.key, alertKey: orNull(v.alertKey), incident: orNull(v.incident), rank: v.rank, event: { ...e, data: v.data }, queuedAt: now }; | ||
| +} | ||
| + | ||
| +export const isDuplicate = (handled: Record<string, HandledEntry>, j: Job, now: number, dedupeHours: number): boolean => | ||
| + isDuplicateRule(fromNullable(handled[j.key]), fromNullable(j.incident), now, dedupeHours); | ||
Addedbench/oncall/v2/rules.ts+88−0
| @@ -0,0 +1,88 @@ | ||
| +// The same rules in TSUS v2: plain .ts, Option is T | undefined, match is a typed function. No | ||
| +// adapter: the watcher's .ts code reads `undefined` the way it already reads optional values. | ||
| +// `tsus check --v2 bench/oncall/v2` passes on this file. | ||
| +import { fromNullable, isSome, matchLiteral, tryCatch, type Option } from "@tsus/v2"; | ||
| +import type { ApiEvent, HandledEntry, JobKind, WatchConfig } from "../types.ts"; | ||
| + | ||
| +const HOUR = 3_600_000; | ||
| +export const TRIAGE_ONLY_PREFIXES = ["bug_report:", "bug_report_reply:", "bug_reports:", "ux_issues:"] as const; | ||
| + | ||
| +export type Job = { | ||
| + readonly id: string; | ||
| + readonly kind: JobKind; | ||
| + readonly key: string; | ||
| + readonly alertKey: Option<string>; | ||
| + readonly incident: Option<number>; | ||
| + readonly rank: number; | ||
| + readonly event: Record<string, unknown>; | ||
| + readonly queuedAt: number; | ||
| +}; | ||
| + | ||
| +type AlertClass = "critical" | "page" | "ticket" | "direct"; | ||
| +const isClass = (s: string): s is AlertClass => s === "critical" || s === "page" || s === "ticket" || s === "direct"; | ||
| +// The queue order of an alert class: critical first, then pages, then tickets. An unknown class | ||
| +// ranks as a ticket. Written the natural way, with the arms inline (allocated per call). | ||
| +export const rankOf = (cls: string): number => | ||
| + isClass(cls) ? matchLiteral(cls, { critical: () => 0, page: () => 1, ticket: () => 2, direct: () => 3 }) : 2; | ||
| + | ||
| +// The same order as an exhaustive switch (TSUS501 checks it covers every class): what a hot path | ||
| +// should use, since a match call allocates its arms. | ||
| +export const rankOfSwitch = (cls: string): number => { | ||
| + if (!isClass(cls)) return 2; | ||
| + switch (cls) { | ||
| + case "critical": | ||
| + return 0; | ||
| + case "page": | ||
| + return 1; | ||
| + case "ticket": | ||
| + return 2; | ||
| + case "direct": | ||
| + return 3; | ||
| + } | ||
| +}; | ||
| + | ||
| +const isRecord = (v: unknown): v is Record<string, unknown> => typeof v === "object" && v !== null && !Array.isArray(v); | ||
| + | ||
| +// An event's data column: a JSON object, or {} for anything else. | ||
| +export const eventData = (raw: Option<string>): Record<string, unknown> => { | ||
| + if (raw === undefined || raw === "") return {}; | ||
| + const parsed = tryCatch((): unknown => JSON.parse(raw)); | ||
| + return parsed.ok && isRecord(parsed.value) ? parsed.value : {}; | ||
| +}; | ||
| + | ||
| +// The API's columns are null or a string; an empty string counts as absent, as it always has. | ||
| +const present = (s: Option<string>): Option<string> => (s === "" ? undefined : s); | ||
| + | ||
| +const watched = (environment: Option<string>, environments: readonly string[]): boolean => | ||
| + environment === undefined || environments.includes(environment); | ||
| + | ||
| +const ownAlert = (key: string): boolean => key.startsWith("oncall:") && !key.startsWith("oncall:drill:"); | ||
| +const triageOnly = (key: string): boolean => TRIAGE_ONLY_PREFIXES.some((p) => key.startsWith(p)); | ||
| + | ||
| +export const jobFor = (e: ApiEvent, cfg: WatchConfig, now: number): Option<Job> => jobWith(rankOf, e, cfg, now); | ||
| +export const jobForSwitch = (e: ApiEvent, cfg: WatchConfig, now: number): Option<Job> => jobWith(rankOfSwitch, e, cfg, now); | ||
| + | ||
| +const jobWith = (rank: (cls: string) => number, e: ApiEvent, cfg: WatchConfig, now: number): Option<Job> => { | ||
| + if (!watched(present(fromNullable(e.environment)), cfg.environments)) return undefined; | ||
| + const data = eventData(fromNullable(e.data)); | ||
| + const key = present(fromNullable(e.key)); | ||
| + if ((e.kind === "alert_opened" || e.kind === "alert_severity_rose") && key !== undefined) { | ||
| + if (ownAlert(key) || triageOnly(key)) return undefined; | ||
| + const cls = typeof data.class === "string" ? data.class : "ticket"; | ||
| + if (cls === "direct") return undefined; | ||
| + return { id: `e${e.id}`, kind: "incident", key, alertKey: key, incident: e.kind === "alert_opened" ? e.at : undefined, rank: rank(cls), event: { ...e, data }, queuedAt: now }; | ||
| + } | ||
| + if (e.kind === "abuse_flag") { | ||
| + const subject = typeof data.subject_id === "string" ? data.subject_id : String(e.id); | ||
| + return { id: `e${e.id}`, kind: "incident", key: `abuse:${subject}`, alertKey: undefined, incident: undefined, rank: 2, event: { ...e, data }, queuedAt: now }; | ||
| + } | ||
| + return undefined; | ||
| +}; | ||
| + | ||
| +// A job without an incident time, or a handled run without one, is the same incident; otherwise an | ||
| +// incident that opened after the key was handled is a new one. | ||
| +const sameIncident = (incident: Option<number>, handled: HandledEntry): boolean => | ||
| + incident === undefined || handled.incident === null || incident <= handled.at; | ||
| + | ||
| +export const isDuplicate = (handled: Option<HandledEntry>, incident: Option<number>, now: number, dedupeHours: number): boolean => | ||
| + isSome(handled) && now - handled.at < dedupeHours * HOUR && sameIncident(incident, handled); | ||
Checks
No checks reported for this commit.
Runs
No runs for this commit.