Coverage Thresholds

Coverage thresholds are minimum acceptable levels of code coverage set by development teams to maintain code quality. These thresholds act as quality gates that help prevent code with insufficient test coverage from being merged or deployed.

Purpose of Coverage Thresholds

Coverage thresholds serve several important purposes in a development workflow:

• Quality Control: Establish a minimum standard for test coverage
• Preventing Regression: Stop changes that would significantly reduce coverage
• Encouraging Testing: Motivate developers to write tests for new code
• Visibility: Make code quality expectations clear to all team members
• Gradual Improvement: Allow teams to iteratively improve coverage in legacy codebases

Types of Coverage Thresholds

Teams commonly implement several types of thresholds:

• Global Threshold: Minimum coverage percentage across the entire codebase
• Patch Threshold: Required coverage for new or changed code in a specific PR or commit
• File/Directory Thresholds: Different coverage requirements for different parts of the codebase
• Metric-Specific Thresholds: Separate minimums for line, branch, function, or statement coverage
• Differential Thresholds: Requirements that coverage cannot decrease by more than a specified amount

Implementing Coverage Thresholds

Coverage thresholds can be implemented at various stages of development:

• Local Development: Configure testing tools to fail if thresholds aren't met
• CI/CD Pipelines: Make builds fail if coverage drops below thresholds
• Pull Request Checks: Use status checks to prevent merging code with insufficient coverage
• Pre-commit Hooks: Prevent commits that would reduce coverage below thresholds

Setting Appropriate Thresholds

Determining the right thresholds for your project involves several considerations:

• Project Maturity: New projects can start with higher thresholds than legacy codebases
• Criticality: Critical components may require higher coverage than utility modules
• Team Capability: Consider your team's experience with testing practices
• Incremental Improvement: Start with achievable thresholds and gradually increase them
• Context-Specific Requirements: Set different thresholds for different types of code (e.g., UI vs. business logic)

Thresholds in OtterWise

OtterWise provides several ways to implement and enforce coverage thresholds:

• Status Checks: Configurable checks that can block PRs with insufficient coverage
• Patch Coverage Requirements: Ensure new or modified code meets minimum thresholds
• PR Comments: Automatic feedback when coverage drops below defined thresholds
• Historical Trending: Track coverage over time to identify declining trends

While coverage thresholds are valuable tools for maintaining code quality, they should be viewed as minimum requirements rather than targets. The ultimate goal is to have meaningful tests that verify functionality, not just to achieve a specific coverage percentage. Balancing coverage metrics with other quality indicators like complexity and code smells provides a more complete picture of code health.