Introduction
This guide will help you integrate OtterWise with your Python project to track code coverage. OtterWise supports all major Python testing frameworks and works seamlessly with popular frameworks like Django, Flask, and FastAPI.
Whether you're building web applications, APIs, data science projects, or command-line tools, this guide covers the setup for your specific use case.
Prerequisites
Before setting up OtterWise for your Python project, you'll need:
- A GitHub repository connected to OtterWise
- Python installed (version 3.9 or higher recommended)
- A testing framework configured in your project
- Your OtterWise repository token (found in your repository settings after enabling it in OtterWise)
Supported Coverage Formats
OtterWise supports the following coverage formats for Python projects:
- Cobertura XML - Standard XML format, widely supported
- Clover XML - Alternative XML format (less common for Python but supported)
We recommend using Cobertura XML format as it's the standard for Python projects and fully supported by coverage.py.
Setup with pytest
pytest is the most popular testing framework for Python. Combined with pytest-cov, it provides comprehensive coverage reporting.
Installation
Install pytest and pytest-cov:
pip install pytest pytest-cov
Or add to your requirements.txt or requirements-dev.txt:
pytest>=8.0.0 pytest-cov>=4.1.0
Configuration
Create or update your pytest.ini or pyproject.toml to configure coverage:
[pytest]
addopts =
--cov=src
--cov-report=xml:coverage.xml
--cov-report=term-missing
testpaths = tests
Using pyproject.toml:
[tool.pytest.ini_options]
addopts = [
"--cov=src",
"--cov-report=xml:coverage.xml",
"--cov-report=term-missing",
]
testpaths = ["tests"]
[tool.coverage.run]
source = ["src"]
omit = [
"*/tests/*",
"*/__pycache__/*",
"*/venv/*",
"*/virtualenv/*",
]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"def __repr__",
"raise AssertionError",
"raise NotImplementedError",
"if __name__ == .__main__.:",
]
Running Tests with Coverage
Run pytest with coverage:
pytest --cov=src --cov-report=xml
This will generate a coverage report at coverage.xml.
Uploading to OtterWise
After generating the coverage report, upload it to OtterWise using the bash uploader:
# Using environment variable export OTTERWISE_TOKEN=your-repo-token bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) # Or pass token directly bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) --repo-token your-repo-token
Setup with unittest
unittest is Python's built-in testing framework. For coverage, you'll use coverage.py.
Installation
pip install coverage
Configuration
Create a .coveragerc file in your project root:
[run]
source = src
omit =
*/tests/*
*/test_*.py
*/__pycache__/*
*/venv/*
*/virtualenv/*
[report]
exclude_lines =
pragma: no cover
def __repr__
raise AssertionError
raise NotImplementedError
if __name__ == .__main__.:
[xml]
output = coverage.xml
Running Tests with Coverage
Run unittest with coverage:
# Run coverage coverage run -m unittest discover # Generate XML report coverage xml
Or combine into a single command:
coverage run -m unittest discover && coverage xml
Setup with coverage.py (Any Test Runner)
coverage.py works with any Python test runner including nose2, ward, or custom test runners.
Installation
pip install coverage
Basic Usage
# Run your test command with coverage coverage run -m your_test_command # Generate XML report coverage xml # Or generate both HTML and XML coverage html coverage xml
Framework-Specific Setup
Django
Django projects typically use Django's built-in test runner with coverage.py or pytest-django.
Option 1: Using Django's test command with coverage.pyInstall coverage:
pip install coverage
Create .coveragerc:
[run]
source = .
omit =
*/migrations/*
*/tests/*
*/test_*.py
*/__pycache__/*
*/venv/*
manage.py
*/settings/*
[report]
exclude_lines =
pragma: no cover
def __repr__
raise AssertionError
raise NotImplementedError
if __name__ == .__main__.:
class .*\bProtocol\):
@(abc\.)?abstractmethod
[xml]
output = coverage.xml
Run tests with coverage:
coverage run --source='.' manage.py test coverage xmlOption 2: Using pytest-django
Install pytest-django:
pip install pytest pytest-cov pytest-django
Create pytest.ini:
[pytest]
DJANGO_SETTINGS_MODULE = myproject.settings
python_files = tests.py test_*.py *_tests.py
addopts =
--cov=.
--cov-report=xml:coverage.xml
--cov-report=term-missing
--reuse-db
Run tests:
pytest
Flask
Flask projects typically use pytest with pytest-flask for testing.
Install dependencies:
pip install pytest pytest-cov pytest-flask
Create pytest.ini:
[pytest]
addopts =
--cov=app
--cov-report=xml:coverage.xml
--cov-report=term-missing
testpaths = tests
[tool.coverage.run]
source = ["app"]
omit = [
"*/tests/*",
"*/__pycache__/*",
"*/venv/*",
]
Example test structure:
# tests/conftest.py
import pytest
from app import create_app
@pytest.fixture
def app():
app = create_app('testing')
return app
@pytest.fixture
def client(app):
return app.test_client()
Run tests:
pytest
FastAPI
FastAPI projects use pytest with httpx for testing.
Install dependencies:
pip install pytest pytest-cov httpx
Create pyproject.toml:
[tool.pytest.ini_options]
addopts = [
"--cov=app",
"--cov-report=xml:coverage.xml",
"--cov-report=term-missing",
]
testpaths = ["tests"]
[tool.coverage.run]
source = ["app"]
omit = [
"*/tests/*",
"*/__pycache__/*",
"*/venv/*",
"*/__main__.py",
]
Example test structure:
# tests/conftest.py
import pytest
from fastapi.testclient import TestClient
from app.main import app
@pytest.fixture
def client():
return TestClient(app)
Run tests:
pytest
CI Integration Examples
GitHub Actions
Here's a complete example for a Python project using GitHub Actions with the official OtterWise action:
name: Tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.9', '3.10', '3.11', '3.12']
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov
- name: Run tests with coverage
run: pytest --cov=src --cov-report=xml
- name: Upload Coverage to OtterWise
if: matrix.python-version == '3.12'
uses: getOtterWise/github-action@v1
with:
token: ${{ secrets.OTTERWISE_TOKEN }}
Important
Remember to add your OTTERWISE_TOKEN to GitHub Secrets (Settings > Secrets and variables > Actions).
Note: We only upload coverage from one Python version (3.12 in this example) to avoid duplicate reports.
GitHub Actions with Django
name: Django Tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 2
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'
cache: 'pip'
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install coverage
- name: Run tests with coverage
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_db
run: |
coverage run --source='.' manage.py test
coverage xml
- name: Upload Coverage to OtterWise
uses: getOtterWise/github-action@v1
with:
token: ${{ secrets.OTTERWISE_TOKEN }}
GitLab CI
test:
image: python:3.12
stage: test
before_script:
- pip install --upgrade pip
- pip install -r requirements.txt
- pip install pytest pytest-cov
script:
- pytest --cov=src --cov-report=xml --cov-report=term
- bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) --repo-token $OTTERWISE_TOKEN
coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
CircleCI
version: 2.1
jobs:
test:
docker:
- image: cimg/python:3.12
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "requirements.txt" }}
- run:
name: Install Dependencies
command: |
python -m venv venv
. venv/bin/activate
pip install -r requirements.txt
pip install pytest pytest-cov
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "requirements.txt" }}
- run:
name: Run Tests with Coverage
command: |
. venv/bin/activate
pytest --cov=src --cov-report=xml
- run:
name: Upload Coverage to OtterWise
command: bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) --repo-token $OTTERWISE_TOKEN
workflows:
version: 2
test:
jobs:
- test
Using Tox for Multi-Environment Testing
Tox allows you to test your package against multiple Python versions and environments.
Install tox:
pip install tox
Create tox.ini:
[tox]
envlist = py39,py310,py311,py312
[testenv]
deps =
pytest
pytest-cov
commands =
pytest --cov=src --cov-report=xml --cov-report=term-missing
[testenv:coverage]
basepython = python3.12
commands =
pytest --cov=src --cov-report=xml
bash -c "curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh | bash"
Run tests across all environments:
tox
Upload coverage from specific environment:
tox -e coverage
Quick Reference: Coverage Commands
| Framework | Command | Output Location |
|---|---|---|
| pytest | pytest --cov=src --cov-report=xml |
coverage.xml |
| unittest + coverage.py | coverage run -m unittest && coverage xml |
coverage.xml |
| Django (manage.py) | coverage run manage.py test && coverage xml |
coverage.xml |
| Django (pytest) | pytest --cov=. --cov-report=xml |
coverage.xml |
| Flask | pytest --cov=app --cov-report=xml |
coverage.xml |
| FastAPI | pytest --cov=app --cov-report=xml |
coverage.xml |
Using Poetry for Dependency Management
If you're using Poetry for dependency management:
Add test dependencies:
poetry add --group dev pytest pytest-cov
Configure in pyproject.toml:
[tool.pytest.ini_options]
addopts = [
"--cov=src",
"--cov-report=xml:coverage.xml",
"--cov-report=term-missing",
]
testpaths = ["tests"]
[tool.coverage.run]
source = ["src"]
omit = [
"*/tests/*",
"*/__pycache__/*",
]
Run tests:
poetry run pytest
Troubleshooting
Common Issues and Solutions
| Issue | Solution |
|---|---|
| Coverage file not found |
|
| Low or zero coverage |
|
| Import errors in tests |
|
| Django database errors |
|
| Coverage not matching locally vs CI |
|
Debugging Coverage Generation
To verify your coverage is being generated correctly:
# Run coverage locally pytest --cov=src --cov-report=xml --cov-report=term # Check if the file exists ls -la coverage.xml # View the first few lines to verify format head -20 coverage.xml # Check coverage data store coverage report coverage html # Generate HTML report for detailed view
A valid Cobertura XML file should start like this:
<?xml version="1.0" ?>
<coverage version="..." timestamp="..." lines-valid="..." lines-covered="..." line-rate="..." ...>
<sources>
<source>/path/to/your/project</source>
</sources>
<packages>
...
Advanced Configuration
Parallel Testing with Coverage
For large test suites, you can run tests in parallel and combine coverage:
# Install pytest-xdist for parallel testing pip install pytest-xdist # Run tests in parallel pytest -n auto --cov=src --cov-report=xml # For coverage.py with parallel mode coverage run --parallel-mode -m pytest coverage combine coverage xml
Branch Coverage
Enable branch coverage for more detailed coverage metrics:
[tool.coverage.run] branch = true source = ["src"] [tool.coverage.report] show_missing = true skip_covered = false
Run with branch coverage:
pytest --cov=src --cov-branch --cov-report=xml
Additional Upload Options
The OtterWise bash uploader supports several options for advanced use cases. See the Bash Uploader documentation for complete details.
Common options for Python projects:
# Specify custom coverage file location bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \ --repo-token $OTTERWISE_TOKEN \ --file ./coverage.xml # Add a flag for test suite identification bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \ --repo-token $OTTERWISE_TOKEN \ --flag unit-tests # For monorepos - track different packages separately bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \ --repo-token $OTTERWISE_TOKEN \ --file ./packages/backend/coverage.xml \ --flag backend
Next Steps
Once you have coverage data uploading successfully, you can:
- Configure Pull Request comments to see coverage changes in your PRs
- Set up status checks to enforce coverage thresholds
- Enable line annotations to highlight uncovered code in PRs
- Use badges to display your coverage status in your README
For additional help, check our example repository or contact OtterWise support.