This is an extensive list of all metrics we support tracking and making reports with. It will be progressively updated as we add more options. Generally, we have no language or framework restrictions, however, some metrics might not be supported as output in all test frameworks.
Code Coverage
A cornerstone of OtterWise and code health measurement is Code Coverage (often referred to as Type Coverage). This metric tells you from 0 to 100, how much of your code is covered / executed by tests, generally on a per-line level.
So if you have a Code Coverage of 50%, that means every other functional line of code, is being executed by at least one test. Functional lines are statements, conditionals, and such. Code comments and method declarations are generally not considered functional lines, we also support calculating method declarations into your Code Coverage metrics.
Code Coverage: Patch
A useful metric that comes with Code Coverage is Patch Coverage, which is the Code Coverage of any changed lines of code in a commit, meaning not for the entire code base, but rather the git diff. A good rule of thumb, if you are pushing for healthier code, is to require collaborators to have a minimum of X % patch coverage.
Code Coverage: Branches / Paths
We also support branch coverage, which is a way of calculating the Code Coverage of your conditionals. A method with an if statement needs to be run through in more than one round to be considered fully covered when tracking branch coverage. With branch coverage, you can see if all possible combinations of conditionals are covered. For example, if both a true and false option was evaluated. If only a true option was passed, branch coverage would be 50%, while Code Coverage might still be 100% as all the code was executed.
Please see the documentation for your test framework if and how it supports branch coverage. We automatically collect it, if outputted.
Type Coverage
If you are using a language with weak typing, and want to embrace types, you can take advantage of type coverage tracking. This number, between 0 and 100, will tell you how much of your code has types defined.
We can also tell you what part of each line is missing a type, such as the method return type, parameter type and more.
Not all test frameworks support type coverage output, so please consult with the documentation for your test framework and read our documentation about type coverage.
Type Coverage: Patch
Type Coverage also supports Patch Coverage, which tells you how much of your changed code is covered by types. This is most useful for PRs and any case where you have git diffs.
Complexity
Cyclomatic Complexity gives you an indication of how complicated your codebase is. It is collected automatically when code coverage is uploaded, for supported test frameworks such as PHPUnit, Pest, and many more. You can learn more about Complexity in our blog: Understanding CRAP and Complexity Metrics.
Change Risk Anti-Patterns (CRAP)
CRAP combines complexity and code coverage, to help you identify if your code has a high risk of breaking or causing unintended side effects when modified, due to low code coverage and high complexity. A low CRAP score signals easy to understand code (low complexity) and high code coverage, a high score indicates the opposite. You can learn more about CRAP in our blog: Understanding CRAP and Complexity Metrics.
