progress

Author: Eric Hartford

docs/fix_more_annoyances.md

@@ -16,9 +16,9 @@ is equivalent to:
 items |> filter(|n| n % 2 == 0)
 ```
 
----
+## Design
 
-## Syntax
+### Syntax
 
 Any expression passed in a closure position that references `it`
 and has no explicit parameter list is treated as an implicit
@@ -36,22 +36,16 @@ single-parameter closure.
 //   3. No explicit parameter list is present
 ```
 
----
-
-## Semantics
-
 ### Type Inference
 
 `it` receives its type from the expected parameter type at the call
 site, exactly as explicit closure parameters do today.
 
 ```
-// filter expects fn(i32) -> bool
-// therefore it: i32
+// filter expects fn(i32) -> bool → it: i32
 items |> filter(it % 2 == 0)
 
-// map expects fn(User) -> str
-// therefore it: User
+// map expects fn(User) -> str → it: User
 users |> map(it.name)
 ```
 
@@ -74,54 +68,6 @@ items |> map(it.children |> filter(it.active))
 //                                    use explicit parameter: |c| c.active
 ```
 
-### Where `it` Is Available
-
-`it` is available anywhere a single-parameter closure is expected:
-
-```
-// Pipeline functions
-items |> filter(it > 0)
-items |> map(it.name)
-items |> any(it.active)
-items |> all(it > threshold)
-items |> find(it.id == target_id)
-items |> count(it.is_valid())
-items |> reduce(0, |acc, x| acc + x)   // multi-param: explicit
-
-// Method calls
-items.filter(it > 0)
-items.map(it.name)
-items.sort_by(it.age)
-
-// Standalone closures assigned to variables
-let is_even = (it % 2 == 0)           // type inferred from usage
-let double = (it * 2)
-
-// Function arguments
-let result = retry(3, it + 1)          // ERROR if fn expects 2+ params
-```
-
-### Where `it` Is NOT Available
-
-- **Multi-parameter closures.** If the expected function type has more
-  than one parameter, `it` is not available. Use explicit parameters.
-
-  ```
-  // reduce expects fn(Acc, T) -> Acc — two params, must be explicit
-  items |> reduce(0, |acc, x| acc + x)
-
-  // zip_with expects fn(A, B) -> C — two params, must be explicit
-  zip_with(xs, ys, |a, b| a + b)
-  ```
-
-- **Ambiguous closure position.** If the compiler cannot determine
-  that the expression is in a closure position (e.g., it's a bare
-  expression not passed to a function), `it` is a normal identifier
-  lookup and follows standard resolution.
-
-- **Nested implicit closures.** As described above, the inner closure
-  must use explicit parameters.
-
 ### `it` Is A Reserved Keyword
 
 `it` is a keyword, the same as `if`, `for`, `match`, `fn`. It cannot
@@ -137,33 +83,32 @@ fn it():               // ERROR: `it` is a reserved keyword
 items |> filter(it > 0)    // implicit closure parameter
 ```
 
-This eliminates all shadowing ambiguity. `it` always means exactly
-one thing: the implicit single-parameter closure reference.
-
-### Interaction With `_`
+### `_` Is Not A Closure Placeholder
 
-`_` remains the discard/wildcard symbol. `it` is the implicit parameter.
-They do not overlap.
+The `_` placeholder syntax (e.g., `items |> filter(_.age > 21)`) was
+previously specified as a closure shorthand. This has been removed from
+the language. `it` is the sole implicit closure parameter. `_` is
+exclusively a discard/wildcard:
 
-```
-// _ discards a value
-let _ = expensive_call()
-
-// it is the implicit parameter
-items |> filter(it > 0)
+- `_` in patterns: wildcard/discard (unchanged)
+- `_` in partial application: placeholder for curried argument (unchanged)
+- `_.field` as closure shorthand: **REMOVED** — use `it.field` instead
+- `it` in expressions: implicit single closure parameter (the one way)
 
-// In match arms, _ is wildcard, it is not special
-match value
-    Some(x) -> x
-    _ -> 0
-```
+### Where `it` Is NOT Available
 
----
+- **Multi-parameter closures.** If the expected function type has more
+  than one parameter, `it` is not available. Use explicit parameters.
+- **Ambiguous closure position.** If the compiler cannot determine
+  that the expression is in a closure position, `it` is a normal
+  identifier lookup and follows standard resolution.
+- **Nested implicit closures.** The inner closure must use explicit
+  parameters.
 
-## Desugaring
+### Desugaring
 
 The compiler desugars implicit closures early, during parsing or
-immediately after. The transformation is:
+immediately after:
 
 ```
 // Source
@@ -175,10 +120,7 @@ items |> filter(|__it| __it % 2 == 0)
 
 The desugared parameter name is internal. The user always writes `it`.
 
-### Detection Algorithm
-
-When the compiler encounters an expression in a closure-expected
-position:
+**Detection algorithm:**
 
 1. Walk the expression tree.
 2. If `it` appears anywhere in the expression:
@@ -193,110 +135,9 @@ Since `it` is a reserved keyword, it can never appear as a local
 variable or other binding. Any occurrence of `it` in an expression
 unambiguously signals an implicit closure.
 
----
-
-## Method Access Shorthand
-
-A common pattern is accessing a field or calling a method:
-
-```
-users |> map(it.name)
-users |> filter(it.is_active())
-users |> sort_by(it.age)
-```
+### Error Messages
 
-These desugar to:
-
-```
-users |> map(|__it| __it.name)
-users |> filter(|__it| __it.is_active())
-users |> sort_by(|__it| __it.age)
-```
-
----
-
-## Chaining
-
-`it` works naturally in chained expressions:
-
-```
-users
-    |> filter(it.age >= 18)
-    |> map(it.name |> uppercase)
-    |> filter(it |> starts_with("A"))
-```
-
-Each `it` in each pipeline step refers to that step's closure
-parameter. There is no ambiguity because each step is a separate
-closure position.
-
----
-
-## Comparison With Explicit Closures
-
-| Pattern | Explicit | Implicit |
-|---|---|---|
-| Simple predicate | `\|n\| n > 0` | `it > 0` |
-| Field access | `\|u\| u.name` | `it.name` |
-| Method call | `\|u\| u.is_active()` | `it.is_active()` |
-| Arithmetic | `\|n\| n * 2 + 1` | `it * 2 + 1` |
-| Two params | `\|a, b\| a + b` | not available |
-| Nested closure | `\|u\| u.items.filter(\|i\| i.ok)` | `it.items.filter(\|i\| i.ok)` |
-
----
-
-## Examples
-
-### Idiomatic With with `it`
-
-```
-// Filter and sum
-let total = items |> filter(it > 0) |> sum
-
-// Extract field
-let names = users |> map(it.name) |> collect[Vec]
-
-// Chain predicates
-let results = entries
-    |> filter(it.active)
-    |> filter(it.score > threshold)
-    |> map(it.name)
-
-// Sort
-let sorted = users |> sort_by(it.age)
-
-// Find
-let admin = users |> find(it.role == .Admin)
-
-// Any / All
-let has_errors = results |> any(it.is_err())
-let all_valid = inputs |> all(it.len() > 0)
-
-// String processing
-let cleaned = lines
-    |> map(it |> trim)
-    |> filter(it.len() > 0)
-    |> filter(it |> starts_with("#") |> not)
-
-// Nested — inner must be explicit
-let active_children = groups
-    |> map(it.members |> filter(|m| m.active) |> count)
-```
-
-### The Landing Page Example
-
-```
-fn main:
-    let sum = read_file("nums.txt")?
-        |> lines |> map(parse[i32]) |> filter(it % 2 == 0) |> sum
-    println("Sum of evens: {sum}")
-```
-
----
-
-## Error Messages
-
-### Nested implicit closure
+Three dedicated error codes:
 
 ```
 error[E0901]: nested implicit closure is ambiguous
@@ -309,8 +150,6 @@ error[E0901]: nested implicit closure is ambiguous
    = suggestion: groups |> map(it.items |> filter(|x| x.active))
 ```
 
-### Wrong arity
-
 ```
 error[E0902]: `it` used in context expecting 2 parameters
   --> src/main.w:8:30
@@ -321,8 +160,6 @@ error[E0902]: `it` used in context expecting 2 parameters
    = help: use explicit parameters: |acc, x| acc + x
 ```
 
-### `it` used as identifier
-
 ```
 error[E0903]: `it` is a reserved keyword
   --> src/main.w:3:9
@@ -332,4 +169,66 @@ error[E0903]: `it` is a reserved keyword
    |
    = note: `it` is the implicit closure parameter keyword
    = help: choose a different name
-```
+```
+
+---
+
+## Implementation Checklist
+
+### 1. Spec (`docs/with-specification.md`)
+- [x] Remove `_.field` closure shorthand from spec. `it.field` is the one way.
+      Done: `_.active` → `it.active` in §feature summary and intro.md;
+      "Placeholder syntax" → "Implicit `it` parameter (see §9.3.1)";
+      added full §9.3.1 subsection defining `it` keyword, scoping,
+      nested prohibition, and clarifying `_` is not a closure placeholder.
+- [ ] Add `it` to the keyword list in the lexical grammar section.
+- [ ] Add the implicit closure grammar production to §9.3 (closure expressions).
+- [ ] Document the desugaring algorithm in the spec.
+- [ ] Add error code definitions for E0901, E0902, E0903.
+
+### 2. Tokenizer / Parser (`src/Token.w`, `src/Parser.w`)
+- [ ] Add `it` as a reserved keyword token (e.g., `TK_IT`).
+- [ ] Reject `it` as an identifier in let bindings, fn declarations, parameter lists, and field names.
+- [ ] Detect implicit closure form: when parsing a call argument in a closure-expected position, check if the expression contains `it` with no explicit `|...|` parameter list.
+- [ ] Desugar to an explicit closure AST node with a synthetic parameter bound to `it`.
+- [ ] Handle nested detection: if an implicit closure body contains another `it` reference at a nested closure-expected position, emit E0901.
+
+### 3. Sema (`src/Sema.w`)
+- [ ] Type-check desugared implicit closures the same as explicit closures (no special path needed if desugaring is complete before sema).
+- [ ] Validate arity: if `it` appears in a context expecting `fn(A, B, ...) -> R` with arity != 1, emit E0902.
+- [ ] Validate keyword: if `it` appears as a binding name (let, fn, param, field), emit E0903.
+
+### 4. Codegen
+- [ ] No changes expected — implicit closures are desugared to standard closure AST nodes before codegen. Verify this holds.
+
+### 5. Docs
+- [ ] Update `docs/with-idiomatic-guide.md` with `it` usage examples and guidance (when to use `it` vs explicit params).
+- [ ] Update `docs/with-migration-guide.md` with mappings from Rust closure patterns to `it`.
+- [ ] Update `docs/intro.md` examples to use `it` where appropriate (partially done: `filter(it.active)` already there).
+
+### 6. Tests
+- [ ] Parser test: `it > 0` in closure position desugars to `|__it| __it > 0`.
+- [ ] Parser test: `it.name` field access desugars correctly.
+- [ ] Parser test: `it.is_active()` method call desugars correctly.
+- [ ] Parser test: nested `it` in nested closure position → E0901 error.
+- [ ] Parser test: `it` in multi-param context → E0902 error.
+- [ ] Parser test: `let it = 42` → E0903 error.
+- [ ] Parser test: `fn it():` → E0903 error.
+- [ ] Parser test: `|it| it + 1` → E0903 error (cannot use keyword as param name).
+- [ ] Sema test: `it` receives correct type from expected parameter type.
+- [ ] Sema test: chained pipelines — each `it` gets its step's type.
+- [ ] Integration test: `items |> filter(it > 0) |> map(it * 2)` produces correct output.
+- [ ] Integration test: explicit inner closure with implicit outer works (`it.children |> filter(|c| c.active)`).
+- [ ] Integration test: `it` works with method call syntax (`items.filter(it > 0)`).
+
+### 7. Bootstrap Compatibility
+- [ ] `it` implementation goes into the self-host compiler only. Bootstrap (Zig) is unchanged.
+- [ ] Self-host source (`src/`) must NOT use `it` syntax — bootstrap cannot compile it.
+- [ ] Self-host compiler must be able to compile user code that uses `it`.
+
+### 8. Validation Gates
+- [ ] All parser tests pass.
+- [ ] All sema tests pass.
+- [ ] All integration tests pass.
+- [ ] Stage3 chain passes with `it` support compiled in.
+- [ ] No regressions in existing closure tests.

docs/implicit_it.md

@@ -65,7 +65,7 @@ It appears in four forms:
 
 ```
 with lock.read() as data:
-    data.iter() |> filter(_.active) |> count()
+    data.iter() |> filter(it.active) |> count()
 ```
 
 The lock is held for exactly the block. The compiler knows the type implements `Scoped`, dispatches through the guard automatically. No keywords, no ceremony — the type tells the compiler everything.