Rebasely Beta: data may be purged

justintypescript-ultra-strict Public

docs: status, diagnostics table, macro lowering, limitations, learnings from the compiler foundation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9472e3cd2f1114656e08650df0936008fd809f63
Parent 9dcde16 Browse filesHistory

Changed files

2 files changed130 additions14 deletionsagainst 9dcde16

  1. ChangedLEARNINGS.md+62−5
  2. ChangedREADME.md+68−9
ChangedLEARNINGS.md+62−5
@@ -30,9 +30,66 @@
## Outcome
-With the MVP in place (`packages/runtime` + `packages/cli`), the harness reports 26/26 examples passing, including the five that pass by being rejected with the expected diagnostic code. Known MVP limitations, in honesty order:
+With the MVP in place (`packages/runtime` + `packages/cli`), the harness reported 26/26 examples passing, including the five that pass by being rejected with the expected diagnostic code. The MVP's known limitations, in honesty order, were:
-1. No semantic type-checking yet: `tsus build` strips types per-module (`ts.transpileModule`), so type errors surface at runtime. The next step is a `ts.createProgram` pass with a virtual file system that maps `.tsus` to preprocessed TS.
-2. No exhaustiveness checking: a non-exhaustive match throws `TSUS100` at runtime instead of failing at compile time.
-3. The `await`-in-arm detection is textual, so a string literal containing the word `await` would wrongly make that arm async.
-4. Regex literals are not tokenized; a regex containing a quote or brace character could confuse the scanner.
+1. No semantic type-checking: `tsus build` stripped types per-module (`ts.transpileModule`), so type errors surfaced at runtime.
+2. No exhaustiveness checking: a non-exhaustive match threw `TSUS100` at runtime instead of failing at compile time.
+3. The `await`-in-arm detection was textual, so a string literal containing the word `await` wrongly made that arm async.
+4. Regex literals were not tokenized; a regex containing a quote or brace character could confuse the scanner.
+
+All four are closed now. The next section is what closing them taught us.
+
+# Learnings from the compiler foundation
+
+## The scanner is free, the context is not
+
+`ts.createScanner` already knows strings, comments, numbers, and every operator, including the ones that matter for `?`: it emits `?.` and `??` as their own tokens, so telling them apart from a bare `?` costs nothing. What it cannot know is parser context. Two tokens need a rescan: `/` (division or regex) and `}` (end of a block or the resume of a template after `${`). Regexes are decided by a one-token lookbehind (after an identifier, literal, `)`, `]`, or `}` it is division), and templates by remembering the brace depth at each `TemplateHead`. Example 32 is the regex `/["'`]/g`, which the old hand-written scanner read as the start of a string and a template.
+
+## Typing match took three tries
+
+The obvious signature, `<V, R>(value: V, arms: { [K in TagOf<V>]: (p: Payload<V, K>) => R }) => R`, type-checks the arms but infers `R` as `unknown`: TypeScript cannot infer through a mapped type whose keys are a conditional type of another type parameter. Every match silently became `unknown`.
+
+Currying fixes inference. `__tsus_match(value)(arms)` fixes `V` in the first call, so in the second the mapped type has concrete keys. Then there was a choice about the result type. A single `R` is Rust-like (arms must agree, errors land on the offending arm), but it rejects the most common tagged-union code: `Idle => tag("Running", ...)`, `Running => tag("Done", ...)` returns two different variant types, and TypeScript will not unify them into the declared union. So the arms object is inferred as a type parameter `A extends MatchArms<V>`, and the match's type is the union of the arms' return types. Exhaustiveness comes from the constraint, unknown arms from an extra mapped type that maps foreign keys to `never`.
+
+One more wrinkle: when the constraint fails (the real error), TypeScript falls back to it, and if the constraint's arms return `unknown`, every use of the match reports a second, misleading error. The constraint returns `any`, which only ever surfaces after a match is already reported broken.
+
+`_` cannot be a key in the same object without turning the mapped type into a union, which breaks contextual typing for the arm parameters, so a wildcard match lowers to a separate `__tsus_match_or(value)(arms, otherwise)`.
+
+## Literal patterns are a different shape
+
+String patterns (the worker router) cannot share the keyed-object design: number and string literals would collide as keys, and the literal `"_"` would collide with the wildcard. They lower to `__tsus_match_lit(value)([[literal, handler], ...], otherwise?)`. A `const` type parameter keeps each literal's type, each literal is checked against the subject's type, and the `otherwise` parameter is a rest tuple that is required exactly when `Exclude<V, literals>` is not `never`. So a `"GET" | "POST" | "DELETE"` subject needs no `_` once all three are listed, and a plain `string` subject always needs one.
+
+## `?` is a parse error with a useful shape
+
+`const x = load()?;` is not TypeScript, but the parser's error recovery turns it into a conditional expression whose true branch is a zero-width missing identifier. That node gives the operand (the condition) and its precise extent, and walking its parents gives the enclosing function and the statement to hoist an early return in front of. A candidate is only accepted when the token after `?` is `;`, `)`, `]`, `}`, `,`, or the end of input, and when the true branch is missing, so valid ternaries, optional parameters (`(x?) => x`), optional tuple elements, and `?:` in types never match.
+
+Two consequences of riding the conditional operator's precedence: `?` covers everything to its left, which is exactly right for `await load()?` (it unwraps the awaited value), and `a()? + 1` is a syntax error that wants parentheses. Rust's postfix `?` binds tighter, and we chose not to fight the parser over it.
+
+Hoisting is only sound where the operand always runs, so `?` in a `?:` branch, on the right of `&&`/`||`/`??`, or in a loop condition is rejected (TSUS303) rather than silently evaluated early. `?` inside a match arm is rejected too (TSUS300): arms are closures and cannot return from the enclosing function.
+
+The type plumbing is three identity functions. `__tsus_try` constrains the operand to `Result | Option` (TSUS301 when it is not), a type guard narrows the bail case, and `__tsus_bail` re-types an `Err<T, E>` as `Err<never, E>` so it fits any `Result<U, E>`. When it does not fit, TypeScript's error lands on the generated `return`, which is how TSUS302 is detected.
+
+## Diagnostics are classified by where they land
+
+Every lowering pass writes through an emitter that records each output run as either a verbatim copy (maps offset-for-offset) or synthesized glue (maps to one anchor). Passes compose, so any offset in the final TypeScript maps back to the `.tsus` file. The same table classifies TypeScript's errors into stable codes: a TS2345 that lands on a match's copied `{` is a missing arm (TSUS201, with the missing variants read off the subject's type through the checker), an error on a copied arm name is TSUS202, a TS2345 at the start of a `__tsus_try(` argument is TSUS301, anything on the generated `return` is TSUS302. Everything else is TSUS200 with the TS code attached.
+
+The runtime import is prepended on line 1 with no newline, so line numbers in the lowered code, the emitted JS, and stack traces without source maps still match the `.tsus` file.
+
+## Semantic checking found a real bug immediately
+
+Example 16 had passed since the MVP, but `wrapAsync(readFile)` infers from `readFile`'s last overload, so the `Ok` payload was `string | Buffer`, and `text.trim()` was a type error that only worked at runtime because the call happened to pass `"utf8"`. The first `ts.createProgram` run flagged it. The fix pins the overload: `wrapAsync((file: string) => readFile(file, "utf8"))`.
+
+The virtual compiler host serves each `.tsus` file as `<file>.tsus.ts` and resolves `./x.tsus` specifiers itself; everything else goes through TypeScript's normal bundler resolution, so `.ts` files importing `.tsus` (example 14) and `.tsus` importing `.ts` (example 13) both type-check as one program.
+
+## Caches remember old compilers
+
+After the runtime's match signature changed, `next build` in example 27 produced an empty page: Next's persistent webpack cache in `.next/cache` kept serving the old loader output, which called the new runtime with the old arguments. Webpack cannot see through the loader's dynamic `import()` to the compiler files, so the loader now declares them with `addBuildDependency`.
+
+## Known limitations now
+
+1. The bundler plugins transform one file at a time and only run the syntactic rules; type errors surface in `tsus check`, not in the framework build.
+2. `tsus check` uses fixed compiler options and ignores `tsconfig.json`.
+3. `?` binds like `?:`, so it needs parentheses inside larger expressions; its return-type check is syntactic and does not see through type aliases.
+4. Match arms are typed as a union, and raw object literals in arms are not contextually typed by the declared return type (`tag` widens to `string`); use `tag(...)`.
+5. The regex lookbehind treats `/` after `)` or `}` as division, commas in type arguments inside an arm body rely on a merge heuristic, and an arm body cannot be a statement block.
+6. If any arm awaits, every arm of that match becomes async.
View file
ChangedREADME.md+68−9
@@ -2,9 +2,9 @@
This is my proposal for TypeScript-Ultra-Strict: a stricter TypeScript for a world where AI writes most of the code.
-> **This repo is a thought experiment, not a production tool.** The compiler works and every example passes, but there is no semantic type-checking, no exhaustiveness checking, no published packages, and no stability promise. It exists to make the proposal concrete enough to argue with. Do not build anything you care about on it yet.
+> **This repo is a thought experiment, not a production tool.** The compiler works, type-checks, and every example passes, but the macro syntax is lowered by a preprocessor with known edge cases, the bundler plugins skip semantic checking, there are no published packages, and there is no stability promise. It exists to make the proposal concrete enough to argue with. Do not build anything you care about on it yet.
-**Status:** working MVP. `packages/cli` implements `tsus check` and `tsus build`, `packages/runtime` implements `Option`/`Result`/`match` and the boundary layer, `packages/plugin` provides webpack/Vite/esbuild/Bun adapters, and all 29 projects under `examples/` pass (`npm install && npm run examples`). `LEARNINGS.md` documents what broke before the toolchain existed, how that shaped the design, and the known limitations.
+**Status:** working prototype. `packages/cli` implements `tsus check` and `tsus build`: a tokenizer built on `ts.createScanner`, a preprocessor that lowers `match` and the `?` operator with a source map back to the `.tsus` file, and a full `ts.createProgram` type-check over the project. `match` is exhaustiveness-checked at compile time, `tsus build` emits `.js.map` files that point at the `.tsus` source, `packages/runtime` implements `Option`/`Result`, the typed match helpers and the boundary layer, `packages/plugin` provides webpack/Vite/esbuild/Bun adapters, and all 36 projects under `examples/` pass (`npm install && npm run examples`). `LEARNINGS.md` documents what broke along the way, how that shaped the design, and the known limitations.
TypeScript's early design decisions assumed humans would write and maintain it, so a lot of features exist to make a human's life easier. We valued speed over quality. With AI writing most of our code, we can flip that tradeoff and force the language to be declarative and verifiable, like Rust.
@@ -44,17 +44,57 @@
type User = { name: string; email: Option<string> };
const loadUser = async (path: string): Promise<Result<User, IoError>> => {
- const raw = await readFile(path, "utf8");
- return raw.map(parseUser);
+ const raw = await readFile(path, "utf8")?; // ? returns the Err early
+ return parseUser(raw);
};
-// macro from the SDK
-const greeting = match(user.email) {
+// macro from the SDK; leaving out an arm is a compile error
+const greeting = match (user.email) {
Some(email) => `Reach me at ${email}`,
None => "No email on file",
};
```
+### How the macros compile
+
+`match` lowers to a call whose arms are an object keyed by variant tag:
+
+```ts
+match (opt) { Some(x) => a, None => b } // -> __tsus_match(opt)({ Some: (x) => (a), None: () => (b) })
+match (opt) { Some(x) => a, _ => b } // -> __tsus_match_or(opt)({ Some: (x) => (a) }, () => (b))
+match (path) { "/" => a, _ => b } // -> __tsus_match_lit(path)([["/", () => (a)]], () => (b))
+```
+
+`@tsus/runtime` types that object with a mapped type over the subject union's `tag` field, so TypeScript itself requires every variant (unless there is a `_` arm), rejects arms the union cannot hold, and narrows each arm's binding to its variant's payload. This works for `Option`, `Result`, and any user union with a string `tag`. Literal patterns lower to a list of `[literal, handler]` pairs; each literal must be a value of the subject's type, and `_` is required unless the literals cover a finite union like `"GET" | "POST"`. The match's type is the union of its arms' types. If any arm awaits, every arm becomes async and the match is a Promise; `await` inside a string or comment does not count.
+
+`expr?` works like Rust's: on an `Err` or `None` it returns that value from the enclosing function, otherwise it unwraps.
+
+```ts
+const port = parsePort(raw)?;
+// -> const __tsus_t0 = __tsus_try(parsePort(raw)); if (__tsus_isBail(__tsus_t0)) return __tsus_bail(__tsus_t0); const port = __tsus_t0.value;
+```
+
+The enclosing function must declare a `Result`, `Option`, or `Promise` of one as its return type, and the propagated error must fit it. In async code `?` applies to the awaited value (`await load()?`). `?` binds like the conditional operator, so it covers everything to its left: write `(a()?) + (b()?)`, not `a()? + b()?`. Optional chaining (`?.`), `??`, ternaries, and optional properties are untouched.
+
+### Diagnostics
+
+| Code | Meaning |
+|---|---|
+| `TSUS000` | Syntax error after macro expansion |
+| `TSUS001`-`TSUS005` | Banned `class`, `null`, `undefined`, `var`, `throw` |
+| `TSUS006` | Malformed `match` (bad pattern, mixed variant and literal patterns, duplicate arm) |
+| `TSUS100` | Runtime only: a match with no arm for its value (only reachable if the checker was bypassed) |
+| `TSUS200` | Any TypeScript error, mapped to the `.tsus` position; `tsCode` holds the TS code |
+| `TSUS201` | Non-exhaustive match: missing variants, or literal arms with no `_` |
+| `TSUS202` | A match arm for a variant or literal the subject cannot hold |
+| `TSUS203` | A match subject that is not a tagged union or a literal type |
+| `TSUS300` | `?` outside a function declaring a `Result`/`Option` return type (or at top level, or in a match arm) |
+| `TSUS301` | `?` applied to something that is not a `Result` or `Option` |
+| `TSUS302` | `?` propagating an `Err`/`None` the function's return type cannot hold |
+| `TSUS303` | `?` in a position that may not run (`?:` branch, right of `&&`/`\|\|`/`??`, loop condition) |
+
+Every TypeScript-derived code (`TSUS2xx`, `TSUS301`, `TSUS302`) carries the original TS code as `tsCode` in `tsus check --json`.
+
### Framework integrations
`packages/plugin` (@tsus/plugin) is one transform core with a thin adapter per host, all sharing the CLI's preprocessor and checker:
@@ -67,7 +107,7 @@
Tooling:
-- `tsus build` compiles to JS, `tsus check` type-checks without emitting. Both run as an esbuild/swc plugin so existing bundler setups pick up `.tsus` files without config changes.
+- `tsus build` compiles to JS with source maps, `tsus check` type-checks without emitting. Both run as an esbuild/swc plugin so existing bundler setups pick up `.tsus` files without config changes. (Today the plugins run the syntactic rules only; the semantic type-check lives in `tsus check`.)
- Editor support ships as a TypeScript language service plugin, so VS Code and every TS-aware editor get diagnostics, go-to-definition, and macro expansion previews for free.
- Diagnostics are machine-first: every error has a stable code, a JSON output mode, and a suggested fix, so an agent can consume `tsus check --json` in a loop without parsing prose. Human-readable output is a rendering of the same data.
- No new package manager and no new registry. npm packages work through the boundary layer, and published `.tsus` packages ship compiled JS plus `.d.ts` files, so consumers don't need to know the source language existed.
@@ -76,11 +116,21 @@
`evals/` holds a harness that tests the thesis directly: a model writes the same 30 small programs (config parsing, an LRU cache, retries with typed errors, a state machine, date ranges, an inventory ledger, and so on) in `.ts` and in `.tsus`, the compiler's errors go back to it for up to three repair rounds, and hidden tests the model never sees decide which version is correct. The tasks were chosen because null, undefined, exceptions and unhandled cases cause real bugs in them. Every task ships a reference solution in both languages and a plausible buggy solution that the hidden tests must catch (`node evals/validate.mjs`).
-Results are in [evals/RESULTS.md](evals/RESULTS.md). So far that file holds only the harness self-test, which runs the reference solutions through the pipeline and says nothing about the thesis. No model run has been recorded yet; `node evals/run.mjs --pilot` runs the 10-task pilot once the Claude Code CLI is logged in or `ANTHROPIC_API_KEY` is set. Even then, one model, one sample per task and 10 to 30 tasks is a pilot, not proof. [evals/README.md](evals/README.md) lists the caveats, starting with this one: the tsus checker does not type-check yet, so `.ts` currently gets more compile-time help than `.tsus`.
+Results are in [evals/RESULTS.md](evals/RESULTS.md). So far that file holds only the harness self-test, which runs the reference solutions through the pipeline and says nothing about the thesis. No model run has been recorded yet; `node evals/run.mjs --pilot` runs the 10-task pilot once the Claude Code CLI is logged in or `ANTHROPIC_API_KEY` is set. Even then, one model, one sample per task and 10 to 30 tasks is a pilot, not proof. [evals/README.md](evals/README.md) lists the caveats, starting with this one: results recorded before the compiler foundation landed came from a tsus checker that did not type-check, so `.ts` got more compile-time help than `.tsus`; `tsus check` now runs a full TypeScript type-check too.
+## Known limitations
+
+The four limitations the MVP shipped with are closed: `tsus check` type-checks the whole project with `ts.createProgram`, a non-exhaustive match is a compile-time `TSUS201`, `await` in arms is detected on the parsed AST, and the tokenizer is TypeScript's own scanner with regex and template rescanning. What is still rough, roughly in order of how likely it is to bite:
+
+1. The bundler plugins (`@tsus/plugin/*`) transform one file at a time and only run the syntactic rules. Type errors surface in `tsus check`, not in `next build` or `vite build`.
+2. `tsus check` uses fixed compiler options (strict, ES2022, bundler resolution, DOM and Node types) and ignores any `tsconfig.json`.
+3. `?` binds like the conditional operator, so `a()? + 1` is a syntax error; write `(a()?) + 1`. It cannot appear in a match arm (arms are closures), a `?:` branch, the right side of `&&`/`||`/`??`, or a loop condition. Its return-type check is syntactic: the declared type must be spelled `Result<...>`, `Option<...>`, or `Promise<...>` of one, not an alias.
+4. Match arms are typed as a union, not required to agree. Raw object literals in arms are not contextually typed by the function's return type, so `{ tag: "Circle" }` widens to `tag: string`; build variants with `tag(...)` or annotate.
+5. Regex detection is a one-token lookbehind: after `)` or `}` a `/` is read as division, so `if (x) /re/.test(s)` misparses. Commas in an arm body's type arguments are handled by a heuristic, and an arm body cannot be a statement block.
+
## The examples
-`examples/` is the test suite: 29 small projects, each with an `example.json` declaring what must happen. `npm run examples` builds and executes all of them and fails on any drift.
+`examples/` is the test suite: 36 small projects, each with an `example.json` declaring what must happen. `npm run examples` builds and executes all of them and fails on any drift.
**Core language and runtime**
@@ -91,6 +141,10 @@
| `03-result-basics` | `Result<T, E>` with `map`/`mapErr` |
| `04-match-option`, `05-match-result` | The `match` macro over Some/None and Ok/Err |
| `06-tagged-union` | User-defined tagged unions with a `_` wildcard arm |
+| `32-regex-literals` | Regex literals containing braces, quotes, a backtick, and a fake match block |
+| `33-await-in-string` | `await` inside a string in an arm stays synchronous; a real `await` makes the match async |
+| `34-try-operator` | `?` on Result and Option, several per expression, in async functions, next to `?.`, `??`, and ternaries |
+| `36-literal-patterns` | Literal arms over string, number, and boolean subjects; finite unions need no `_` |
**Rejected on purpose** (these pass by failing the checker with the right code)
@@ -101,6 +155,11 @@
| `09-banned-var` | TSUS004 |
| `10-banned-undefined` | TSUS003 |
| `11-banned-throw` | TSUS005 |
+| `30-non-exhaustive-match` | TSUS201, for a missing tagged-union variant, a missing `Err`, and literal arms that miss a union member |
+| `31-wrong-arm-type` | TSUS200 (an arm of the wrong type, a payload used as the wrong type) and TSUS202 (a misspelled variant) |
+| `35-try-misuse` | TSUS300, TSUS301, TSUS302, TSUS303 |
+
+Rejection examples can also pin exact positions (`"diagnostics": [{ "code", "file", "line", "col", "tsCode" }]` in `example.json`), which tests the source map back to the `.tsus` file.
**Interop and stress**
View file

Checks

No checks reported for this commit.

Runs

No runs for this commit.