Patch coverage is a specific type of code coverage metric that focuses exclusively on the lines of code that have been changed in a commit or pull request. Unlike overall code coverage which measures the entire codebase, patch coverage provides targeted insights into whether new or modified code is properly tested.
How Patch Coverage Works
Patch coverage works by isolating the specific lines that have been added or modified in a commit or pull request (the "patch") and then determining what percentage of those lines are executed by tests. The process typically involves:
- Diff Generation: Creating a diff between the base version and the new version
- Line Tracking: Identifying which specific lines have been added or modified
- Coverage Analysis: Determining which of those lines are executed during test runs
- Percentage Calculation: Dividing the number of covered lines by the total number of changed lines
Benefits of Patch Coverage
Patch coverage offers several advantages over traditional code coverage metrics:
- Focused Feedback: Provides information specifically about the code that's changing
- Incremental Improvement: Allows teams to enforce high test coverage for new code even in legacy codebases with lower overall coverage
- Immediate Relevance: Concentrates attention on the parts of the code currently being developed
- PR-Specific Insights: Helps reviewers assess the quality of tests for the specific changes being reviewed
- Incremental Adoption: Enables gradual improvement of test coverage without requiring immediate remediation of all existing gaps
Patch Coverage in CI/CD Workflows
In modern CI/CD pipelines, patch coverage is often used as a quality gate:
- Pull Request Checks: Enforcing minimum patch coverage before merging code
- Commit Validation: Ensuring tests cover new functionality before proceeding
- Developer Feedback: Providing immediate information about test coverage during development
- Code Review Support: Giving reviewers objective data about test completeness
Setting Patch Coverage Thresholds
Organizations typically set coverage thresholds for patch coverage that are higher than their overall coverage requirements. Common approaches include:
- High Threshold: Requiring 80-100% coverage for new or changed code
- Gradual Implementation: Starting with a lower threshold and increasing it over time
- Context-Sensitive Requirements: Setting different thresholds for different parts of the codebase
- Exemption Processes: Allowing documented exceptions for specific cases where tests are impractical
Patch Coverage in OtterWise
OtterWise provides robust support for patch coverage:
- Status Checks: GitHub status checks that pass or fail based on patch coverage
- PR Comments: Detailed information about patch coverage in pull request comments
- Visualization: Visual representation of which changed lines are covered or uncovered
- Configurability: Customizable patch coverage thresholds at the repository level
- Historical Tracking: Analysis of patch coverage trends over time
Patch coverage represents a targeted approach to ensuring code quality by focusing on the specific changes being made. By enforcing high test coverage for new and modified code, teams can gradually improve overall quality without being overwhelmed by the task of completely testing legacy codebases at once.
