commit missing stuff from v18 blog post

Author: pdubroy

blog/2026-02-20-ohm-v18.md

@@ -0,0 +1,64 @@
+---
+title: Ohm v18 Beta
+slug: ohm-v18
+description: "Ohm v18 compiles grammars to WebAssembly, making parsing ~20x faster and using a fraction of the memory."
+image: ./ohm-v18.png
+---
+
+_aka "The One that Compiles to Wasm"._
+
+After nearly a year of development, we're excited to announce the first beta release of Ohm v18 — the biggest change to Ohm since its initial release. We've totally reworked the core parsing engine to be WebAssembly-based, making parsing around 20x faster on real-world grammars while using a fraction of the memory.
+
+## What's new
+
+Every version of Ohm up to v17 worked the same way under the hood: when you call `grammar.match()`, Ohm walks a tree of parsing expression objects (PExprs), calling `eval()` on each node. (It's a so-called [tree-walking interpreter](https://craftinginterpreters.com/a-tree-walk-interpreter.html).) In the process, it builds up a huge parse tree, with each node a separate object that must be managed by the GC.
+
+v18 takes a completely different approach. At build time, the new `@ohm-js/compiler` translates your grammar into a WebAssembly module, compiling each rule to its own function. The parse tree is allocated into Wasm linear memory, and nodes used a packed representation which is much, much more memory efficient.
+
+Also, the runtime (`ohm-js`) is now separate from the compiler (`@ohm-js/compiler`):
+
+```bash
+npx ohm2wasm my-grammar.ohm   # compile at build time
+```
+
+```js
+import {Grammar} from 'ohm-js';
+
+// load and run at runtime
+const g = await Grammar.instantiate(fs.readFileSync('my-grammar.wasm'));
+```
+
+## How much faster?
+
+We've been benchmarking with two real-world workloads:
+
+1. Our [official ES5 grammar](https://github.com/ohmjs/ohm/blob/main/examples/ecmascript/src/es5.ohm) compiling a large (742K) JavaScript file.
+2. Shopify's [LiquidHTML grammar](https://github.com/Shopify/theme-tools/blob/main/packages/liquid-html-parser/grammar/liquid-html.ohm), parsing all the Liquid templates from their [Dawn theme](https://github.com/Shopify/dawn).
+
+One these two benchmarks, v18 parses about **22x faster** than v17, and requires less than 20% of the memory. 🔥
+
+## Breaking changes
+
+Along with the new runtime, v18 has a significantly reworked API. Check out the [migration guide](../docs/releases/ohm-js-18.0) for all the details on the new API, what's changed, and what's not in v18 yet.
+
+And please note: **the new API is still in flux**, so expect some changes before the stable release.
+
+## Try it out
+
+```bash
+npm install ohm-js@beta                      # Runtime (production dependency)
+npm install --save-dev @ohm-js/compiler@beta # Compiler (dev dependency)
+```
+
+If you want to kick the tires without changing your build setup, there's a compat helper that parses, compiles, and instantiates in one step — just like the old `ohm.grammar()`:
+
+```js
+import {grammar} from '@ohm-js/compiler/compat';
+
+const g = grammar('MyGrammar { start = "hello" }');
+using result = g.match('hello');
+```
+
+(This compiles on every call, so it's great for prototyping but probably not what you want for production.)
+
+We'd love to hear your feedback on v18. Give it a spin, and let us know what you think on [Discord](https://discord.gg/KwxY5gegRQ) or [GitHub Discussions](https://github.com/ohmjs/ohm/discussions).

blog/ohm-v18.png

@@ -0,0 +1,132 @@
+# CLI Migration Plan
+
+Unify around a single `ohm` CLI shipped by `@ohm-js/compiler`, with `compile`
+(wasm-first batch compilation), `match` (wasm-first), and type generation as the
+stable surface.
+
+## End-State CLI Surface
+
+Binary name: `ohm` (from `@ohm-js/compiler`)
+
+### `ohm compile <patterns...>`
+
+Compiles `.ohm` sources to wasm artifacts in batch (glob-aware), plus optional
+type generation. This is the v18 successor to `generateBundles`.
+
+Flags:
+
+- `--cwd <dir>` — base directory for glob expansion
+- `-o, --outDir <dir>` — where to write outputs (default: alongside source)
+- `-t, --withTypes` — generate corresponding `.d.ts`
+- `-g, --grammarName <name>` — compile only one grammar from a multi-grammar file
+  (default: compile all)
+- `-n, --dryRun` — print/plan without writing
+- `--quiet` / `--verbose`
+
+Output naming:
+
+- `path/to/foo.ohm` → `path/to/foo.ohm.wasm`
+- With `--withTypes`: `path/to/foo.ohm.d.ts`
+
+### `ohm match <inputPath>`
+
+Wasm-first matching. Accepts either a `.wasm` artifact or a source `.ohm` file
+(compile-then-match).
+
+Flags:
+
+- `-g, --grammar <path>` — `.ohm` (compile in-memory, then match) or `.wasm`
+  (load directly)
+- `--grammarName <name>` — select grammar from multi-grammar file
+- `--startRule <rule>` — optional start rule selection
+
+Behavior: exit 0 on success, non-zero with failure message on mismatch.
+
+### `ohm types <patterns...>` (optional)
+
+Standalone type generation, useful when you want `.d.ts` without emitting wasm.
+
+Flags: `--cwd`, `--outDir`, `--grammarName`, `--dryRun`
+
+### Naming
+
+- Primary verb: `ohm compile` (not `build`), since "compile to wasm" is the core
+  v18 story.
+- No default action — bare `ohm` shows help.
+
+## Migration Phases
+
+### Phase 0: Today
+
+- `@ohm-js/compiler` ships `ohm2wasm` (single-file, no globs/batch)
+- `@ohm-js/cli` ships `ohm` with `generateBundles` (v17 recipes) and `match`
+  (v17 runtime)
+
+### Phase 1: Introduce unified `ohm` in the compiler (v18 alpha → early beta)
+
+Goal: make `@ohm-js/compiler` fully usable as the official CLI.
+
+1. Add `ohm` bin entry to `@ohm-js/compiler`, alongside `ohm2wasm`.
+2. Refactor `ohm2wasm` implementation into `ohm compile`:
+   - Add glob support via `fast-glob`
+   - Add `--outDir`, `--cwd`, `--withTypes`, batch compilation
+   - If invoked as `ohm2wasm`, behave like `ohm compile <file>` with a
+     deprecation warning.
+3. Implement `ohm match` wasm-first:
+   - `--grammar` accepts `.wasm` (load directly) or `.ohm` (compile-then-match)
+4. Move `generateTypes` into `@ohm-js/compiler` (the compiler already bundles
+   `ohm-js-legacy` internally via esbuild — use that for type generation without
+   exposing it publicly).
+5. Update docs to recommend `pnpm add -D @ohm-js/compiler` and `ohm compile`.
+
+### Phase 2: Deprecate `@ohm-js/cli` (v18 beta)
+
+Goal: keep the old install path working while steering users to the compiler.
+
+1. Release a new major of `@ohm-js/cli` that:
+   - Depends on `@ohm-js/compiler`
+   - Forwards all args to the compiler's `ohm` CLI
+   - Prints deprecation warning
+2. Compatibility mapping:
+   - `ohm generateBundles …` → forwards to `ohm compile …`
+   - `ohm match …` → forwards to compiler's `ohm match …`
+3. Deprecate the package on npm.
+
+### Phase 3: Compiler CLI is the only real CLI (v18 stable)
+
+1. `@ohm-js/compiler`'s `ohm` is the official CLI in all docs.
+2. `@ohm-js/cli` remains as deprecated wrapper only — no new features.
+3. `generateBundles` kept as deprecated alias for one stable cycle.
+
+### Phase 4: Clean up (v19 or later)
+
+- Stop publishing `@ohm-js/cli`, or keep as permanent thin wrapper.
+- Optionally remove `ohm2wasm` alias, or keep it (low maintenance cost).
+
+## Type Generation in v18
+
+The compiler already bundles `ohm-js-legacy` via esbuild for internal use. Type
+generation can reuse this internal representation to produce `.d.ts` files
+without exposing legacy APIs publicly.
+
+- Primary: `ohm compile --withTypes`
+- Optional: `ohm types <patterns...>` for CI workflows that want types without
+  wasm
+
+## Risks and Guardrails
+
+- **Artifact format churn**: lock down output naming (`.ohm.wasm`, `.ohm.d.ts`)
+  early.
+- **Multi-grammar files**: default is compile all; `--grammarName` selects one.
+- **Dependencies**: anything needed at runtime by the CLI (`commander`,
+  `fast-glob`) must be in `dependencies` of `@ohm-js/compiler`, not
+  `devDependencies`.
+- **Node version**: `ohm match` with `.ohm` input requires Node 24 (same as the
+  compiler). Document clearly.
+
+## Future Considerations (not now)
+
+- `--emit wasm+js` for generating JS loader modules (depends on v18 runtime
+  loading API stabilizing)
+- `--watch` mode
+- Incremental compilation / caching via `--cacheDir`

docs/cli-migration-plan.md

@@ -0,0 +1,106 @@
+# Incremental semantics
+
+Ohm supports incremental parsing, meaning that once an input is parsed, it can be quickly reparsed after each edit operation. Incremental _parsing_ is straightforward to use: you just need to use instantiate a [Matcher object](https://ohmjs.org/docs/api-reference#matcher-objects) rather than directly using your Grammar's `match` method.
+
+It's also possible to build fully incremental processing pipelines on top of Ohm's incremental parsing, but this is less straightforward. In this document we talk about some of the strategies for doing so.
+
+## Understanding the _overlap rule_
+
+TODO
+
+## Defining an attribute
+
+The first step for building an incremental processing pipeline is to define an attribute. In Ohm, an _attribute_ is like an operation, but (a) it takes no arguments, and (b) it is memoized. The attribute's value for a given node will be recalculated whenever the edit may have affected that node. For nodes that are not affected by an edit, the attribute value is cached.
+
+So, the simplest kind of incremental processing pipeline you can build consists of a single attribute. For example, for an arithmetic grammar, you might define a `value` attribute for evaluating arithmetic expressions:
+
+```js
+const semantics = grammar.createSemantics().addAttribute('value', {
+  Exp: (addExp) => addExp.value,
+  AddExp_plus: (left, _op, right) => left.value + right.value,
+  AddExp_minus: (left, _op, right) => left.value - right.value,
+  AddExp: (priExp) => priExp.value,
+  PriExp_paren: (_open, exp, _close) => exp.value,
+  PriExp: (number) => number.value,
+  number(digits) {
+    return parseInt(this.sourceString, 10);
+  }
+});
+```
+
+And suppose you used the grammar and a `Matcher` object to evaluate an expression, then make an edit, then re-parse and re-evaluate:
+
+```js
+const m = grammar.matcher();
+
+m.setInput('(1 + 2) + (3 - 4)');
+assert.equal(semantics(m.match()).value, 2);
+
+m.replaceInputRange(1, 2, '0'); // Replace 1 with 0
+assert.equal(semantics(m.match()).value, 1);
+```
+
+Note that the `AddExp_minus` action, which calculates the value of "3 - 4", will only run once. The edit does not affect that part of the parse result, so the attribute value is cached.
+
+Ohm's caching of attribute values is naive; notably, it does not track dependencies between attribute values or do any kind of [autotracking](https://www.pzuraq.com/blog/what-is-reactivity) as seen in signals frameworks.
+
+If the naive caching is not sufficient, there is an **internal** operation named `_forgetMemoizedResultFor` TODO
+
+## Building more complex pipelines
+
+You can leverage the caching of attributes to build more complex transformations. For example, if you define an `ast` attribute, you can rely on object identity to determine which AST nodes were affected by an edit:
+
+```js
+  const m = grammar.matcher();
+  m.setInput("(1 + 2) + (3 - 4)");
+
+  const seen = new Set(); // Could also use WeakSet here.
+  let root = semantics(m.match()).ast;
+  seen.add(root);
+  seen.add(root.left);
+  seen.add(root.right);
+
+  m.replaceInputRange(1, 2, "0"); // Replace 1 with 0
+  root = semantics(m.match()).ast;
+
+  // The root and left child are recreated; the right child is reused.
+  assert(!seen.has(root));
+  assert(!seen.has(root.left));
+  assert(seen.has(root.right));
+```
+
+You can then build additional transformations on top of the AST, using recursive tree-walking functions and memoization to avoid reprocessing unchanged subtrees. For example:
+
+```js
+  // Returns an array containing all the nubmers in the given subtree.
+  function getNumbers(node, cache = new WeakMap()) {
+    if (cache.has(node)) return cache.get(node);
+
+    let result;
+    if (node.type === "Number") {
+      result = [node.value];
+    }  else if (node.type === "BinaryOp") {
+      result = [
+        ...getNumbers(node.left, cache),
+        ...getNumbers(node.right, cache),
+      ];
+    } else {
+      throw new Error(`Unknown node type: ${node.type}`);
+    }
+    cache.set(node, result);
+    return result;
+  }
+```
+
+Rather than implementing the memoization yourself, you can also use an existing library like [memoizee](https://www.npmjs.com/package/memoizee), [micro-memoize](https://www.npmjs.com/package/micro-memoize), etc.
+
+## Dependencies on siblings
+
+When an attribute value depends on the value of a sibling, you can:
+
+- Define an operation to compute the value, potentially taking one or more arguments with context information.
+- Define an attribute which caches the value of invoking the operation on all the node's children.
+
+This is an extremely useful pattern that can apply to many problems in text processing. For example, Ohm internally uses a version of this when calculating the absolute offset of nodes in the parse tree. Each node caches the relative offsets of all its children, so after an edit that affects _k_ nodes, the offsets can be updated in O(k) time.
+
+See [Zed Decoded: Rope & SumTree](https://zed.dev/blog/zed-decoded-rope-sumtree) for a discussion of how this same pattern is used in text editors.