Add test coverage reporting with grcov and codecov integration#61
Merged
Add test coverage reporting with grcov and codecov integration#61
Conversation
- Add cargo-tarpaulin to development dependencies in flake.nix - Add 'cov' app for running tests with coverage using tarpaulin - Update CI workflow to run tests with coverage and upload to codecov.io - Modify integration tests to support tarpaulin compilation flags - Handle CARGO_TARGET_DIR environment variable for coverage builds - Add proper RUSTFLAGS extraction from tarpaulin for coverage compilation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive test coverage reporting capabilities to the project by integrating codecov.io with cargo-tarpaulin for Rust code coverage analysis. The changes enable automated coverage collection during CI runs and provide developers with a local coverage testing workflow.
Key changes:
- Integration of cargo-tarpaulin for Rust code coverage collection
- Addition of a 'cov' flake app for local coverage testing with proper environment setup
- CI workflow modifications to run tests with coverage and automatically upload results to codecov.io
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| flake.nix | Adds cargo-tarpaulin dependency and creates 'cov' app for coverage testing |
| .github/workflows/CI.yaml | Updates CI to run coverage tests and upload results to codecov.io |
| src/server/integration_tests.rs | Modifies integration tests to support tarpaulin compilation with proper flag extraction and target directory handling |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
- Configure tarpaulin to use LLVM engine for more accurate coverage - Enables consistent coverage reporting across development and CI
- Increase LSP analysis timeout from 5s to 15s for better test reliability - Increase LSP ready timeout from 5s to 15s for better test reliability - Extract target directory determination into reusable get_target_dir() function - Improve code organization in integration test setup
The test_lsp_declaration test fails when running with coverage enabled. Adding ignore attribute to prevent test failures during coverage runs.
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
The codecov action can auto-detect coverage files without specifying the file parameter, simplifying the configuration.
- Replace cargo-tarpaulin with grcov for more reliable coverage reporting - Add llvm-tools to Rust components for coverage instrumentation - Simplify integration test binary compilation by removing one-time compilation logic - Update coverage app script to use LLVM instrumentation and grcov output formats - Add test app script for basic test execution - Remove tarpaulin-specific code paths from integration tests
- Remove .tarpaulin.toml configuration file - Add scripts/run-test.sh and scripts/run-cov.sh for standardized testing - Update flake.nix to use grcov with LLVM-based coverage reporting - Support HTML, Cobertura, and Markdown coverage report formats - Add Nix apps for test and cov commands - Update CHANGELOG.md, CLAUDE.md, and README.md with new coverage workflow - Integrate test scripts into development environment packages
Update flake.nix to use correct package names: - test app now references run-test package - cov app now references run-cov package
- Simplify CI workflow to use run-cov script directly - Add console feedback message during coverage report generation
Enable test_lsp_declaration to run with coverage enabled after fixing coverage compatibility issues.
Remove unnecessary comment and simplify macro implementation in integration_tests.rs for better code clarity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces comprehensive test coverage reporting to the project using grcov (LLVM-based code coverage) and codecov.io integration. This is the first coverage implementation for the project.
Key Changes
New Coverage Infrastructure
grcovrun-cov.shandrun-test.shscripts for standardized testing workflowsNix Flake Enhancements
grcovand LLVM tools to development dependenciescovandtestapps for easy coverage analysis:nix run .#covandnix run .#testTest Infrastructure Improvements
CARGO_TARGET_DIRenvironment variable for coverage buildsCI/CD Pipeline Updates
cargo testwith coverage-enabled testing in GitHub ActionsTechnical Implementation
Coverage Workflow
Usage Examples
Files Added/Modified
New Files
scripts/run-cov.sh: Comprehensive coverage execution scriptscripts/run-test.sh: Standardized test execution scriptModified Files
.github/workflows/CI.yaml: Add coverage steps and codecov integrationflake.nix: Add grcov, LLVM tools, and new coverage/test appssrc/*/integration_tests.rs: Improve reliability and simplify binary managementCLAUDE.md,README.md,CHANGELOG.md: Update documentation with coverage instructionsBenefits
This establishes the foundation for maintaining high code quality and comprehensive test coverage as the project continues to grow.