|
1 | | -# Coverage Scripts |
| 1 | +# Coverage Helpers |
2 | 2 |
|
3 | | -Coverage workflow orchestration is implemented in Python and is used by `.github/workflows/coverage.yml`. |
4 | | -GitHub matrix helpers live in `tools/coverage/ci_matrix.py`. |
5 | | -GitHub artifact aggregation helpers live in `tools/coverage/ci_reports.py`. |
| 3 | +Coverage CI is driven by `.github/workflows/coverage.yml`. |
6 | 4 |
|
7 | | -## Entrypoint |
8 | | -```bash |
9 | | -python3 tools/coverage/coverage.py <command> |
10 | | -``` |
11 | | - |
12 | | -## Commands |
13 | | -- `run-all`: run the full local flow (or a selected range). |
14 | | -- `step <name>`: run a single step. |
15 | | -- `list-tests --suite <cpp|python|js> --profile <...>`: show resolved tests for a profile. |
16 | | -- `validate-config`: validate YAML suite configs. |
17 | | - |
18 | | -Step modules are in `tools/coverage/steps/`. |
19 | | - |
20 | | -## Profiles |
21 | | -Supported profiles: |
22 | | -- `cpu` |
23 | | -- `gpu` (GPU-only; runs only tests explicitly marked for GPU execution) |
24 | | -- `npu` (NPU-only; runs only tests explicitly marked for NPU execution) |
25 | | - |
26 | | -Profile-specific test selection and args are defined in: |
27 | | -- `tools/coverage/config/tests_cpp.yml` |
28 | | -- `tools/coverage/config/tests_python.yml` |
29 | | -- `tools/coverage/config/tests_js.yml` |
30 | | - |
31 | | -## Local Prerequisites |
32 | | -- Python 3.10+ |
33 | | -- C/C++ build toolchain and system deps (installed by `install-deps`) |
34 | | -- Node.js + npm for JS coverage steps |
35 | | - |
36 | | -To install dependencies locally (including Node.js): |
37 | | -```bash |
38 | | -python3 tools/coverage/coverage.py step install-deps --install-nodejs --nodejs-version 22 |
39 | | -``` |
40 | | - |
41 | | -`--install-nodejs` is optional. If omitted and Node.js is missing, JS coverage steps will fail. |
| 5 | +`tools/coverage` now contains only the helper code still used by that workflow: |
| 6 | +- suite definitions in `config/` |
| 7 | +- suite test runners in `steps/` |
| 8 | +- native C/C++ coverage collection in `steps/collect_cpp_coverage.py` |
| 9 | +- summary and duration aggregation in `ci_reports.py` |
42 | 10 |
|
43 | | -## Typical Local Flows |
44 | | -Run full flow with dependency installation: |
| 11 | +Main entrypoint: |
45 | 12 | ```bash |
46 | | -python3 tools/coverage/coverage.py run-all --profile cpu --install-deps --install-nodejs --nodejs-version 22 |
| 13 | +python3 tools/coverage/coverage.py step <run-cpp-tests|run-python-tests|run-js-tests|collect-cpp-coverage> |
47 | 14 | ``` |
48 | 15 |
|
49 | | -Run full flow without reinstalling deps: |
50 | | -```bash |
51 | | -python3 tools/coverage/coverage.py run-all --profile cpu |
52 | | -``` |
53 | | - |
54 | | -Run only test + coverage collection phase: |
55 | | -```bash |
56 | | -python3 tools/coverage/coverage.py run-all --profile cpu --from run-cpp-tests --to package-artifacts |
57 | | -``` |
58 | | - |
59 | | -Validate configs: |
| 16 | +Additional helper commands: |
60 | 17 | ```bash |
| 18 | +python3 tools/coverage/coverage.py list-tests --suite <cpp|python|js> --profile <cpu|gpu|npu> |
61 | 19 | python3 tools/coverage/coverage.py validate-config |
62 | 20 | ``` |
63 | 21 |
|
64 | | -Inspect resolved tests: |
65 | | -```bash |
66 | | -python3 tools/coverage/coverage.py list-tests --suite python --profile gpu |
67 | | -``` |
68 | | - |
69 | | -## Outputs |
70 | | -Main artifacts in workspace root: |
71 | | -- `coverage.info` (native C/C++ lcov) |
72 | | -- `coverage-artifact-metadata.json` (CI shard-artifact identity file used for summary aggregation) |
73 | | -- `cpp-coverage-stats.env` (per-run C++ shard stats) |
74 | | -- `cpp-test-durations.csv` (per-run C++ test durations in seconds) |
75 | | -- `python-coverage-stats.env` (per-run Python shard stats) |
76 | | -- `python-test-durations.csv` (per-run Python test durations in seconds) |
77 | | -- `python-coverage.xml` (Python coverage XML) |
78 | | -- `js-coverage-stats.env` (per-run JS shard stats) |
79 | | -- `js-test-durations.csv` (per-run JS test durations in seconds) |
80 | | -- `js-lcov.info` (Node.js lcov) |
81 | | -- `coverage-report/index.html` (HTML report) |
82 | | -- `.tmp/coverage-local/step_summary.md` (local summary when not in GitHub Actions) |
83 | | - |
84 | | -## Key Runtime Options |
85 | | -- `--parallel-jobs <N>` |
86 | | -- `--cpp-test-concurrency <N>`: run configured C++ coverage test binaries in parallel. Values above `1` isolate gcov output per run and merge it during `collect-cpp-coverage`. |
87 | | -- `--pytest-workers <N>` |
88 | | -- `--js-test-concurrency <N>` |
89 | | -- `--profile <name>` |
90 | | -- `--install-deps` |
91 | | -- `--install-nodejs` |
92 | | -- `--nodejs-version <major>` |
93 | | - |
94 | | -## Step-specific Environment |
95 | | -- `CXX_TEST_NAMES=name1,name2,...`: limit `run-cpp-tests` to the named config entries. This is used by the GitHub Actions workflow to shard C++ coverage jobs. |
96 | | -- `PY_TEST_NAMES=name1,name2,...`: limit `run-python-tests` to the named config entries. This is used by the GitHub Actions workflow to shard Python coverage jobs. |
97 | | -- `JS_TEST_NAMES=name1,name2,...`: limit `run-js-tests` to the named config entries. This is used by the GitHub Actions workflow to shard JS coverage jobs. |
98 | | -- `COVERAGE_WRITE_STEP_SUMMARY=false`: suppress per-step summary output so only the final workflow summary is published in GitHub Actions. |
| 22 | +Important environment variables used by the workflow: |
| 23 | +- `TEST_PROFILE` |
| 24 | +- `BUILD_DIR` |
| 25 | +- `INSTALL_PKG_DIR` |
| 26 | +- `BIN_DIR` |
| 27 | +- `CXX_TEST_NAMES` |
| 28 | +- `PY_TEST_NAMES` |
| 29 | +- `JS_TEST_NAMES` |
| 30 | +- `COVERAGE_WRITE_STEP_SUMMARY=false` |
0 commit comments