This is a full list of the metrics we support for tracking and reporting. We'll update it as we add more options. Generally, we have no language or framework restrictions; however, some metrics might not be supported as output in all test frameworks.
Code Coverage
A cornerstone of OtterWise and code health measurement is Code Coverage. This metric tells you from 0 to 100, how much of your code is covered / executed by tests, generally on a per-line level.
So if you have a Code Coverage of 50%, that means every other functional line of code is executed by at least one test. Functional lines are statements, conditionals, and such. Code comments and method declarations are generally not considered functional lines. However, we also support counting method declarations toward your Code Coverage metrics.
Code Coverage: Patch
A useful metric that comes with Code Coverage is Patch Coverage, which is the Code Coverage of any changed lines of code in a commit, meaning the git diff rather than the entire code base. A good rule of thumb, if you are pushing for healthier code, is to require collaborators to have a minimum of X % patch coverage.
Code Coverage: Branches / Paths
We also support branch coverage, which is a way of calculating the Code Coverage of your conditionals. A method with an if statement needs to run more than once to be considered fully covered when tracking branch coverage. With branch coverage, you can see if all possible combinations of conditionals are covered. For example, if both a true and a false option were evaluated. If only a true option was passed, branch coverage would be 50%, while Code Coverage might still be 100% since all the code ran.
Please see the documentation for your test framework if and how it supports branch coverage. We automatically collect it, if outputted.
Type Coverage
If you are using a language with weak typing, and want to embrace types, you can take advantage of type coverage tracking. This number, between 0 and 100, will tell you how much of your code has types defined.
We can also tell you what part of each line is missing a type, such as the method return type, parameter type and more.
Not all test frameworks support type coverage output, so please consult your test framework's documentation, and read our documentation about type coverage.
Type Coverage: Patch
Type Coverage also supports Patch Coverage, which tells you how much of your changed code is covered by types. This is most useful for PRs and any case where you have git diffs.
Complexity
Cyclomatic Complexity gives you an indication of how complicated your codebase is. It is collected automatically when code coverage is uploaded, for supported test frameworks such as PHPUnit, Pest, and many more. You can learn more about Complexity in our blog: Understanding CRAP and Complexity Metrics.
Change Risk Anti-Patterns (CRAP)
CRAP combines complexity and code coverage, to help you identify if your code has a high risk of breaking or causing unintended side effects when modified, due to low code coverage and high complexity. A low CRAP score signals easy-to-understand code (low complexity) and high code coverage. A high score indicates the opposite. You can learn more about CRAP in our blog: Understanding CRAP and Complexity Metrics.
