Skip to content

Commit 268a3ee

Browse files
committed
refactor
1 parent 2266cc8 commit 268a3ee

File tree

16 files changed

+511
-1829
lines changed

16 files changed

+511
-1829
lines changed

.github/dockerfiles/ov_test/ubuntu_22_04_x64/Dockerfile

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ RUN apt-get update && \
2323
ca-certificates \
2424
gpg-agent \
2525
tzdata \
26+
# coverage
27+
lcov \
2628
# parallel gzip
2729
pigz \
2830
# Samples

.github/workflows/coverage.yml

Lines changed: 329 additions & 800 deletions
Large diffs are not rendered by default.

.github/workflows/job_build_linux.yml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ on:
6565
type: boolean
6666
required: false
6767
default: true
68+
upload-coverage-notes:
69+
description: 'Whether to upload gcno notes captured from the build directory'
70+
type: boolean
71+
required: false
72+
default: false
6873

6974
permissions: read-all
7075

@@ -109,6 +114,7 @@ jobs:
109114
ARTIFACTS_SHARE: "/mount/build-artifacts"
110115
MANIFEST_PATH: '${{ github.workspace }}/openvino/manifest.yml'
111116
PRODUCT_TYPE: public_linux_${{ inputs.os }}_${{ inputs.arch }}_release
117+
COVERAGE_NOTES_DIR: ${{ env.BUILD_DIR }}/coverage_build_notes
112118
steps:
113119
- name: Append the environment variable - load SCCACHE_AZURE_CONNECTION_STRING from file
114120
shell: bash
@@ -269,6 +275,15 @@ jobs:
269275
run: tar -cvf - * | pigz > ${BUILD_DIR}/openvino_tests.tar.gz
270276
working-directory: ${{ env.INSTALL_TEST_DIR }}
271277

278+
- name: Capture coverage notes for main build
279+
if: ${{ inputs.upload-coverage-notes }}
280+
run: |
281+
rm -rf "${COVERAGE_NOTES_DIR}"
282+
mkdir -p "${COVERAGE_NOTES_DIR}/main-build"
283+
rsync -a --prune-empty-dirs \
284+
--include='*/' --include='*.gcno' --exclude='*' \
285+
"${BUILD_DIR}/" "${COVERAGE_NOTES_DIR}/main-build/"
286+
272287
- name: Build Debian packages
273288
if: ${{ inputs.build-debian-packages }}
274289
env:
@@ -324,6 +339,14 @@ jobs:
324339
tar -czvf ${BUILD_DIR}/openvino_node_npm_package.tar.gz bin openvino-node-*
325340
working-directory: ${{ env.OPENVINO_REPO }}/src/bindings/js/node
326341

342+
- name: Capture coverage notes for JS build
343+
if: ${{ inputs.upload-coverage-notes && fromJSON(inputs.affected-components).JS_API && inputs.build-js }}
344+
run: |
345+
mkdir -p "${COVERAGE_NOTES_DIR}/js-build"
346+
rsync -a --prune-empty-dirs \
347+
--include='*/' --include='*.gcno' --exclude='*' \
348+
"${BUILD_DIR}/" "${COVERAGE_NOTES_DIR}/js-build/"
349+
327350
- name: Build RPM packages
328351
if: ${{ inputs.build-rpm-packages }}
329352
run: |
@@ -411,6 +434,14 @@ jobs:
411434
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
412435
if-no-files-found: 'error'
413436

437+
- name: Upload coverage build notes
438+
if: ${{ inputs.upload-coverage-notes }}
439+
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
440+
with:
441+
name: coverage_build_notes
442+
path: ${{ env.COVERAGE_NOTES_DIR }}
443+
if-no-files-found: 'error'
444+
414445
- name: Prepare debian packages for storage on share
415446
if: ${{ always() }}
416447
continue-on-error: true

tools/coverage/README.md

Lines changed: 20 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -1,98 +1,30 @@
1-
# Coverage Scripts
1+
# Coverage Helpers
22

3-
Coverage workflow orchestration is implemented in Python and is used by `.github/workflows/coverage.yml`.
4-
GitHub matrix helpers live in `tools/coverage/ci_matrix.py`.
5-
GitHub artifact aggregation helpers live in `tools/coverage/ci_reports.py`.
3+
Coverage CI is driven by `.github/workflows/coverage.yml`.
64

7-
## Entrypoint
8-
```bash
9-
python3 tools/coverage/coverage.py <command>
10-
```
11-
12-
## Commands
13-
- `run-all`: run the full local flow (or a selected range).
14-
- `step <name>`: run a single step.
15-
- `list-tests --suite <cpp|python|js> --profile <...>`: show resolved tests for a profile.
16-
- `validate-config`: validate YAML suite configs.
17-
18-
Step modules are in `tools/coverage/steps/`.
19-
20-
## Profiles
21-
Supported profiles:
22-
- `cpu`
23-
- `gpu` (GPU-only; runs only tests explicitly marked for GPU execution)
24-
- `npu` (NPU-only; runs only tests explicitly marked for NPU execution)
25-
26-
Profile-specific test selection and args are defined in:
27-
- `tools/coverage/config/tests_cpp.yml`
28-
- `tools/coverage/config/tests_python.yml`
29-
- `tools/coverage/config/tests_js.yml`
30-
31-
## Local Prerequisites
32-
- Python 3.10+
33-
- C/C++ build toolchain and system deps (installed by `install-deps`)
34-
- Node.js + npm for JS coverage steps
35-
36-
To install dependencies locally (including Node.js):
37-
```bash
38-
python3 tools/coverage/coverage.py step install-deps --install-nodejs --nodejs-version 22
39-
```
40-
41-
`--install-nodejs` is optional. If omitted and Node.js is missing, JS coverage steps will fail.
5+
`tools/coverage` now contains only the helper code still used by that workflow:
6+
- suite definitions in `config/`
7+
- suite test runners in `steps/`
8+
- native C/C++ coverage collection in `steps/collect_cpp_coverage.py`
9+
- summary and duration aggregation in `ci_reports.py`
4210

43-
## Typical Local Flows
44-
Run full flow with dependency installation:
11+
Main entrypoint:
4512
```bash
46-
python3 tools/coverage/coverage.py run-all --profile cpu --install-deps --install-nodejs --nodejs-version 22
13+
python3 tools/coverage/coverage.py step <run-cpp-tests|run-python-tests|run-js-tests|collect-cpp-coverage>
4714
```
4815

49-
Run full flow without reinstalling deps:
50-
```bash
51-
python3 tools/coverage/coverage.py run-all --profile cpu
52-
```
53-
54-
Run only test + coverage collection phase:
55-
```bash
56-
python3 tools/coverage/coverage.py run-all --profile cpu --from run-cpp-tests --to package-artifacts
57-
```
58-
59-
Validate configs:
16+
Additional helper commands:
6017
```bash
18+
python3 tools/coverage/coverage.py list-tests --suite <cpp|python|js> --profile <cpu|gpu|npu>
6119
python3 tools/coverage/coverage.py validate-config
6220
```
6321

64-
Inspect resolved tests:
65-
```bash
66-
python3 tools/coverage/coverage.py list-tests --suite python --profile gpu
67-
```
68-
69-
## Outputs
70-
Main artifacts in workspace root:
71-
- `coverage.info` (native C/C++ lcov)
72-
- `coverage-artifact-metadata.json` (CI shard-artifact identity file used for summary aggregation)
73-
- `cpp-coverage-stats.env` (per-run C++ shard stats)
74-
- `cpp-test-durations.csv` (per-run C++ test durations in seconds)
75-
- `python-coverage-stats.env` (per-run Python shard stats)
76-
- `python-test-durations.csv` (per-run Python test durations in seconds)
77-
- `python-coverage.xml` (Python coverage XML)
78-
- `js-coverage-stats.env` (per-run JS shard stats)
79-
- `js-test-durations.csv` (per-run JS test durations in seconds)
80-
- `js-lcov.info` (Node.js lcov)
81-
- `coverage-report/index.html` (HTML report)
82-
- `.tmp/coverage-local/step_summary.md` (local summary when not in GitHub Actions)
83-
84-
## Key Runtime Options
85-
- `--parallel-jobs <N>`
86-
- `--cpp-test-concurrency <N>`: run configured C++ coverage test binaries in parallel. Values above `1` isolate gcov output per run and merge it during `collect-cpp-coverage`.
87-
- `--pytest-workers <N>`
88-
- `--js-test-concurrency <N>`
89-
- `--profile <name>`
90-
- `--install-deps`
91-
- `--install-nodejs`
92-
- `--nodejs-version <major>`
93-
94-
## Step-specific Environment
95-
- `CXX_TEST_NAMES=name1,name2,...`: limit `run-cpp-tests` to the named config entries. This is used by the GitHub Actions workflow to shard C++ coverage jobs.
96-
- `PY_TEST_NAMES=name1,name2,...`: limit `run-python-tests` to the named config entries. This is used by the GitHub Actions workflow to shard Python coverage jobs.
97-
- `JS_TEST_NAMES=name1,name2,...`: limit `run-js-tests` to the named config entries. This is used by the GitHub Actions workflow to shard JS coverage jobs.
98-
- `COVERAGE_WRITE_STEP_SUMMARY=false`: suppress per-step summary output so only the final workflow summary is published in GitHub Actions.
22+
Important environment variables used by the workflow:
23+
- `TEST_PROFILE`
24+
- `BUILD_DIR`
25+
- `INSTALL_PKG_DIR`
26+
- `BIN_DIR`
27+
- `CXX_TEST_NAMES`
28+
- `PY_TEST_NAMES`
29+
- `JS_TEST_NAMES`
30+
- `COVERAGE_WRITE_STEP_SUMMARY=false`

tools/coverage/ci_matrix.py

Lines changed: 0 additions & 172 deletions
This file was deleted.

0 commit comments

Comments
 (0)