|
1 | | -Rewrite Stale Tests |
2 | | -Goal: Convert 31 tests that import internal compiler modules |
3 | | -into end-to-end tests using the test runner's directive system. |
4 | | -Scope: Only test files. No compiler changes. This is mechanical |
5 | | -rewriting. |
6 | | -Pattern: Each old test calls internal APIs to invoke sema/codegen |
7 | | -and checks results programmatically. Each new test is a .w file |
8 | | -with //! directives that the test runner validates. |
9 | | -How to convert |
10 | | -Error message tests (16 tests: err_assign_immutable, |
11 | | -err_binop_types, err_borrow_conflict, etc.): |
12 | | -Write a .w file that triggers the error, use directive: |
13 | | -//! expect-check-fail |
14 | | -//! expect-error: assign to immutable variable |
15 | | -let x = 5 |
16 | | -x = 10 |
17 | | -Compiler internal tests (13 tests: ast_test, lexer_test, |
18 | | -parser_test, sema_test, etc.): |
19 | | -Convert to end-to-end tests that compile and run programs |
20 | | -exercising the relevant feature: |
21 | | -//! expect-stdout: 42 |
22 | | -fn main: |
23 | | - let x = 40 + 2 |
24 | | - println("{x}") |
25 | | -For tests that specifically test internal data structures (intern |
26 | | -pool, arena, etc.), either delete them (the fixpoint test proves |
27 | | -these work) or convert to programs that exercise the behavior |
28 | | -from the outside. |
29 | | -Integration tests (2 tests: integ_full_pipeline, |
30 | | -integ_multi_module): |
31 | | -Convert to multi-file build tests that compile and run programs |
32 | | -with multiple modules. |
33 | | -Checklist |
34 | | - |
35 | | - Convert all 16 err_* tests to //! expect-check-fail tests |
36 | | - Convert all 13 compiler internal tests to end-to-end tests |
37 | | - Convert 2 integration tests to multi-file build tests |
38 | | - Delete any tests that are now redundant (e.g., ast_test |
39 | | -testing pool internals that fixpoint already validates) |
40 | | - All 31 converted tests pass under scripts/run_tests.sh |
41 | | - |
42 | | -Exit gate |
43 | | -Zero stale-API test failures. All converted tests exercise real |
44 | | -compiler behavior through the CLI. |
| 1 | +# 02 — Rewrite Stale Tests |
| 2 | + |
| 3 | +Goal: Convert every test in `test/cases/` that imports internal |
| 4 | +compiler modules into an end-to-end test using CLI directives. |
| 5 | +No compiler changes. Only test files. |
| 6 | + |
| 7 | +Scope: 27 stale tests in `test/cases/`. Wave 1–3 tests are |
| 8 | +separate (they test the `compiler.foundation.*` rewrite and will |
| 9 | +be addressed when that work lands). |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +## How the Test Runner Works |
| 14 | + |
| 15 | +`scripts/run_tests.sh` scans `test/cases/*.w` and parses header |
| 16 | +directives: |
| 17 | + |
| 18 | +``` |
| 19 | +//! expect-stdout: <text> build+run, check stdout substring |
| 20 | +//! expect-check-fail: <msg> check must fail, stderr contains <msg> |
| 21 | +//! expect-error: <msg> alias for expect-check-fail |
| 22 | +//! expect-build-fail: <msg> build must fail, stderr contains <msg> |
| 23 | +//! check-only check only, no build/run |
| 24 | +//! args: <flags> extra compiler flags |
| 25 | +``` |
| 26 | + |
| 27 | +Every new test must use these directives. No test may import |
| 28 | +internal compiler modules (`use Ast`, `use Sema`, `use Types`, |
| 29 | +`use Mir`, `use Codegen`, `use Driver`, etc.). |
| 30 | + |
| 31 | +--- |
| 32 | + |
| 33 | +## Pass 1: Delete Redundant Tests |
| 34 | + |
| 35 | +These tests verify internal constants or data structures already |
| 36 | +validated by fixpoint (`stage2 == stage3`). Delete outright. |
| 37 | + |
| 38 | +- [ ] Delete `test/cases/ast_test.w` (tests AstPool node storage internals) |
| 39 | +- [ ] Delete `test/cases/intern_test.w` (tests InternPool symbol dedup internals) |
| 40 | +- [ ] Delete `test/cases/codegen_test.w` (tests LLVM type IDs, instruction opcodes) |
| 41 | +- [ ] Delete `test/cases/mir_test.w` (tests MirBody, basic blocks, terminators) |
| 42 | +- [ ] Delete `test/cases/safety_checks.w` (tests MIR terminator kind constants) |
| 43 | +- [ ] Delete `test/cases/driver_test.w` (tests SourceFile, CImport, render, Driver internals) |
| 44 | +- [ ] Delete `test/cases/err_diag_codes.w` (tests `E_UNDECLARED == 1` etc.) |
| 45 | +- [ ] Delete `test/cases/lexer_test.w` (redundant with hundreds of behavior tests) |
| 46 | +- [ ] Delete `test/cases/parser_test.w` (redundant — fixpoint is ultimate parser validation) |
| 47 | +- [ ] Run `./scripts/run_tests.sh` — confirm no regressions after deletions |
| 48 | + |
| 49 | +--- |
| 50 | + |
| 51 | +## Pass 2: Rewrite Error Tests |
| 52 | + |
| 53 | +Each test currently constructs Sema/TypeTable/AstPool objects |
| 54 | +directly. Rewrite as a `.w` program the user would write that |
| 55 | +triggers the same error via `//! expect-check-fail`. |
| 56 | + |
| 57 | +### 2.1 err_assign_immutable.w |
| 58 | + |
| 59 | +Current: Creates Sema, calls `define_var` with `is_mut=0`, checks `var_is_mut`. |
| 60 | + |
| 61 | +- [ ] Read current `test/cases/err_assign_immutable.w` |
| 62 | +- [ ] Rewrite as program that assigns to immutable variable: |
| 63 | + `//! expect-check-fail: immutable` + `let x = 5` then `x = 10` |
| 64 | +- [ ] Optionally create `behav_mutability.w` for the success case |
| 65 | + (`let mut x = 5; x = 10; println("{x}")` → `//! expect-stdout: 10`) |
| 66 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_assign_immutable.w` — passes |
| 67 | + |
| 68 | +### 2.2 err_binop_types.w |
| 69 | + |
| 70 | +Current: Constructs AST nodes for `i32 + i32`, `i32 == i32`, `bool && bool`, checks types. |
| 71 | + |
| 72 | +- [ ] Read current `test/cases/err_binop_types.w` |
| 73 | +- [ ] Rewrite as end-to-end program exercising binary operations: |
| 74 | + `//! expect-stdout: 3` + arithmetic, comparison, logical ops |
| 75 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_binop_types.w` — passes |
| 76 | + |
| 77 | +### 2.3 err_borrow_conflict.w |
| 78 | + |
| 79 | +Current: Constructs MirBody, adds borrows, runs BorrowChecker, checks error counts. |
| 80 | + |
| 81 | +- [ ] Read current `test/cases/err_borrow_conflict.w` |
| 82 | +- [ ] Rewrite as program with borrow conflict: |
| 83 | + `//! expect-check-fail: borrow` + shared ref then mutation |
| 84 | +- [ ] If compiler doesn't detect this yet, use `//! expect-stdout` test |
| 85 | + showing borrows work + add TODO comment for negative test |
| 86 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_borrow_conflict.w` — passes |
| 87 | + |
| 88 | +### 2.4 err_invalid_cast.w |
| 89 | + |
| 90 | +Current: Calls `cast_instruction(types, TYPE_I32, TYPE_I64)`, checks return values. |
| 91 | + |
| 92 | +- [ ] Read current `test/cases/err_invalid_cast.w` |
| 93 | +- [ ] Rewrite as program with invalid cast: |
| 94 | + `//! expect-check-fail: cast` + `let s = "hello"; let x = s as i32` |
| 95 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_invalid_cast.w` — passes |
| 96 | + |
| 97 | +### 2.5 err_match_str_exhaust.w |
| 98 | + |
| 99 | +Current: Checks `TypeTable.is_enum`, `TypeTable.is_bool`, pattern node kind constants. |
| 100 | + |
| 101 | +- [ ] Read current `test/cases/err_match_str_exhaust.w` |
| 102 | +- [ ] Rewrite as program with non-exhaustive match on string: |
| 103 | + `//! expect-check-fail: exhaustive` + match with no wildcard |
| 104 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_match_str_exhaust.w` — passes |
| 105 | + |
| 106 | +### 2.6 err_return_type.w |
| 107 | + |
| 108 | +Current: Sets `s.current_return_type = TYPE_I32`, checks `types_compatible`. |
| 109 | + |
| 110 | +- [ ] Read current `test/cases/err_return_type.w` |
| 111 | +- [ ] Rewrite as program with return type mismatch: |
| 112 | + `//! expect-check-fail: type` + `fn get_num() -> i32: "not a number"` |
| 113 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_return_type.w` — passes |
| 114 | + |
| 115 | +### 2.7 err_trait_bound.w |
| 116 | + |
| 117 | +Current: Creates TraitSolver, adds traits/impls, checks resolve and obligations. |
| 118 | + |
| 119 | +- [ ] Read current `test/cases/err_trait_bound.w` |
| 120 | +- [ ] Rewrite as program with trait bound violation: |
| 121 | + `//! expect-check-fail: trait` + call requiring trait not implemented |
| 122 | +- [ ] If compiler doesn't produce this error yet, use simpler trait test |
| 123 | + + add TODO for the negative case |
| 124 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_trait_bound.w` — passes |
| 125 | + |
| 126 | +### 2.8 err_type_mismatch.w |
| 127 | + |
| 128 | +Current: Checks `types_compatible(TYPE_I32, TYPE_STR)` etc. |
| 129 | + |
| 130 | +- [ ] Read current `test/cases/err_type_mismatch.w` |
| 131 | +- [ ] Rewrite as program with type mismatch: |
| 132 | + `//! expect-check-fail: type` + `let x: i32 = "hello"` |
| 133 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_type_mismatch.w` — passes |
| 134 | + |
| 135 | +### 2.9 err_undefined_fn.w |
| 136 | + |
| 137 | +Current: Checks `Sema.find_fn(s, "foo") == -1`. |
| 138 | + |
| 139 | +- [ ] Read current `test/cases/err_undefined_fn.w` |
| 140 | +- [ ] Rewrite as program calling undefined function: |
| 141 | + `//! expect-check-fail: undefined` + `fn main: foo()` |
| 142 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_undefined_fn.w` — passes |
| 143 | + |
| 144 | +### 2.10 err_undefined_type.w |
| 145 | + |
| 146 | +Current: Checks `TypeTable.lookup(types, "NoSuchType") == -1`. |
| 147 | + |
| 148 | +- [ ] Read current `test/cases/err_undefined_type.w` |
| 149 | +- [ ] Rewrite as program using undefined type: |
| 150 | + `//! expect-check-fail: undefined` + `let x: NoSuchType = 42` |
| 151 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_undefined_type.w` — passes |
| 152 | + |
| 153 | +### 2.11 err_undefined_var.w |
| 154 | + |
| 155 | +Current: Constructs AST ident for "unknown", checks `TYPE_ERROR`. |
| 156 | + |
| 157 | +- [ ] Read current `test/cases/err_undefined_var.w` |
| 158 | +- [ ] Rewrite as program using undefined variable: |
| 159 | + `//! expect-check-fail: undefined` + `fn main: println(x)` |
| 160 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_undefined_var.w` — passes |
| 161 | + |
| 162 | +### 2.12 err_wrong_arg_count.w |
| 163 | + |
| 164 | +Current: Registers function with 2 params, checks `fn_param_counts`. |
| 165 | + |
| 166 | +- [ ] Read current `test/cases/err_wrong_arg_count.w` |
| 167 | +- [ ] Rewrite as program calling function with wrong arg count: |
| 168 | + `//! expect-check-fail: argument` + `fn add(a: i32, b: i32) -> i32: a + b` |
| 169 | + then `add(1)` |
| 170 | +- [ ] Run `./scripts/run_tests.sh test/cases/err_wrong_arg_count.w` — passes |
| 171 | + |
| 172 | +### Pass 2 verification |
| 173 | + |
| 174 | +- [ ] Run `./scripts/run_tests.sh` — all error tests pass |
| 175 | + |
| 176 | +--- |
| 177 | + |
| 178 | +## Pass 3: Rewrite Internal & Integration Tests |
| 179 | + |
| 180 | +### 3.1 sema_test.w → behav_sema_basics.w |
| 181 | + |
| 182 | +Current: 10 test functions checking builtin types, scope chains, function |
| 183 | +registration, type compatibility, expression type checking, variant lookup. |
| 184 | + |
| 185 | +- [ ] Read current `test/cases/sema_test.w` |
| 186 | +- [ ] Write `test/cases/behav_sema_basics.w` exercising scope chains: |
| 187 | + `//! expect-stdout: 10` + nested scope with let bindings |
| 188 | +- [ ] Add variant lookup test: enum declaration, match, correct output |
| 189 | +- [ ] Delete `test/cases/sema_test.w` |
| 190 | +- [ ] Run `./scripts/run_tests.sh test/cases/behav_sema_basics.w` — passes |
| 191 | + |
| 192 | +### 3.2 type_test.w → behav_type_system.w |
| 193 | + |
| 194 | +Current: 18 test functions covering struct/enum/array/slice/tuple/fn/ptr/ref/ |
| 195 | +alias/option/result types, equality, generics, trait objects, scopes. |
| 196 | + |
| 197 | +- [ ] Read current `test/cases/type_test.w` |
| 198 | +- [ ] Write `test/cases/behav_type_system.w` with struct test: |
| 199 | + type declaration, field access, `//! expect-stdout` verification |
| 200 | +- [ ] Add enum test: variant construction, match, payload extraction |
| 201 | +- [ ] Add tuple test: tuple literal, field access by index |
| 202 | +- [ ] Add array test: array literal, `.len()` call |
| 203 | +- [ ] Delete `test/cases/type_test.w` |
| 204 | +- [ ] Run `./scripts/run_tests.sh test/cases/behav_type_system.w` — passes |
| 205 | + |
| 206 | +### 3.3 borrow_test.w → behav_borrow_basic.w |
| 207 | + |
| 208 | +Current: Tests BorrowInfo, NllRegion, BorrowChecker on manually-constructed MIR. |
| 209 | + |
| 210 | +- [ ] Read current `test/cases/borrow_test.w` |
| 211 | +- [ ] Write `test/cases/behav_borrow_basic.w` exercising borrows: |
| 212 | + `//! expect-stdout: ok` + shared borrow that compiles and runs |
| 213 | +- [ ] If borrow checker catches conflicts at check time, add negative test |
| 214 | + `err_borrow_mut.w` with `//! expect-check-fail: borrow` |
| 215 | +- [ ] Delete `test/cases/borrow_test.w` |
| 216 | +- [ ] Run `./scripts/run_tests.sh test/cases/behav_borrow_basic.w` — passes |
| 217 | + |
| 218 | +### 3.4 traits_test.w → behav_trait_basics.w |
| 219 | + |
| 220 | +Current: Tests TraitSolver (trait definition, impl registration, resolve, |
| 221 | +coherence, obligation lists). |
| 222 | + |
| 223 | +- [ ] Read current `test/cases/traits_test.w` |
| 224 | +- [ ] Write `test/cases/behav_trait_basics.w` with trait + impl + method call: |
| 225 | + `//! expect-stdout: hello` + trait definition, struct impl, method call |
| 226 | +- [ ] Delete `test/cases/traits_test.w` |
| 227 | +- [ ] Run `./scripts/run_tests.sh test/cases/behav_trait_basics.w` — passes |
| 228 | + |
| 229 | +### 3.5 integ_full_pipeline.w — rewrite in place |
| 230 | + |
| 231 | +Current: Manually constructs AST, runs each compiler phase, checks results. |
| 232 | + |
| 233 | +- [ ] Read current `test/cases/integ_full_pipeline.w` |
| 234 | +- [ ] Rewrite as real program exercising the full pipeline: |
| 235 | + `//! expect-stdout: 42` + nested function, if/else, arithmetic |
| 236 | +- [ ] Run `./scripts/run_tests.sh test/cases/integ_full_pipeline.w` — passes |
| 237 | + |
| 238 | +### 3.6 integ_multi_module.w — rewrite in place |
| 239 | + |
| 240 | +Current: Imports every compiler module, creates one instance of each. |
| 241 | + |
| 242 | +- [ ] Read current `test/cases/integ_multi_module.w` |
| 243 | +- [ ] Check how existing multi-file tests work (e.g., `shadow_helper.w`) |
| 244 | +- [ ] Create `test/cases/integ_helper_module.w` with `//! check-only` + |
| 245 | + `pub fn add(a: i32, b: i32) -> i32: a + b` |
| 246 | +- [ ] Rewrite `test/cases/integ_multi_module.w`: |
| 247 | + `//! expect-stdout: 3` + `use integ_helper_module` + `println("{add(1, 2)}")` |
| 248 | +- [ ] If `use` path resolution doesn't support relative test imports, |
| 249 | + make it a single-file test instead |
| 250 | +- [ ] Run `./scripts/run_tests.sh test/cases/integ_multi_module.w` — passes |
| 251 | + |
| 252 | +### Pass 3 verification |
| 253 | + |
| 254 | +- [ ] Run `./scripts/run_tests.sh` — all tests pass |
| 255 | + |
| 256 | +--- |
| 257 | + |
| 258 | +## Final Verification |
| 259 | + |
| 260 | +- [ ] Run: `grep -r "^use " test/cases/*.w | grep -E "(Ast|Sema|Types|Mir|Codegen|Driver|BorrowCfg|Lexer|Token|Parser|InternPool|Source|CImport|render|Span|Diag|MirLower)"` — prints nothing |
| 261 | +- [ ] Run `./scripts/run_tests.sh` — zero failures |
| 262 | +- [ ] Every rewritten error test triggers the expected diagnostic through normal compilation |
| 263 | +- [ ] Every rewritten behavior test builds, runs, and produces correct output |
0 commit comments