Coverage Parts (Split Uploads)

Introduction

When your test suite runs across several parallel CI runners (say, sharded PHPUnit groups or a matrix job), each runner only produces coverage for the slice of code it executed. Uploading those slices as separate flags or components would fragment the results and throw off the totals.

Coverage parts tell OtterWise that an upload is part N of M for a given flag and component. OtterWise waits until every part has arrived before posting status checks, PR comments, or totals, and then merges them into one report.

When to use parts vs. flags vs. components

  • Parts: the same test suite executed across multiple runners (parallel sharding).
  • Flags: different test suites for the same code, like unit vs. integration.
  • Components: different sub-projects in a monorepo, like a frontend and a backend.

Uploading Parts

Each runner invokes the bash uploader with two extra flags:

Option Description
--part The index of this upload (1-based, must be a positive integer).
--part-total The total number of parts that will be uploaded for this flag and component. Must match across all parts.

Both flags default to 1, so if you only upload once you never need to set them.

Example: 3-way Parallel PHPUnit

# Runner 1
bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./coverage/shard-1.xml \
  --part 1 \
  --part-total 3

# Runner 2
bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./coverage/shard-2.xml \
  --part 2 \
  --part-total 3

# Runner 3
bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./coverage/shard-3.xml \
  --part 3 \
  --part-total 3

Example: Parts Combined with Flags and Components

Parts are scoped to a flag and component pair, so you can shard one suite while still uploading other suites as single uploads: 

# Unit tests (2 parallel shards) on the backend component
bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./backend/coverage/unit-shard-1.xml \
  --component backend --flag unit-tests \
  --part 1 --part-total 2

bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./backend/coverage/unit-shard-2.xml \
  --component backend --flag unit-tests \
  --part 2 --part-total 2

# Integration tests (single upload) on the same component
bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
  --file ./backend/coverage/integration.xml \
  --component backend --flag integration-tests

GitHub Actions Example

On GitHub Actions the shard index and total come from a matrix strategy: 

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - run: ./vendor/bin/pest --parallel --processes=4 --shard=${{ matrix.shard }}/4
      - name: Upload coverage to OtterWise
        env:
          OTTERWISE_TOKEN: ${{ secrets.OTTERWISE_TOKEN }}
        run: |
          bash <(curl -s https://raw.githubusercontent.com/getOtterWise/bash-uploader/main/uploader.sh) \
            --file ./coverage/clover.xml \
            --part ${{ matrix.shard }} \
            --part-total 4

How OtterWise Processes Parts

Each upload is stored as its own CommitCoverage row, keyed by (commit, flag, component, part_index). Commit-level totals are recomputed after every part arrives, but the commit is only treated as fully covered once the received part count matches part_total.

Until the final part arrives:

  • Status checks, PR comments, and line annotations are deferred, so reviewers see one merged result instead of a stream of partial ones.
  • The commit's has_coverage flag stays false, so the dashboard won't surface partial numbers.
  • Carry-forward from the default branch is not triggered. The incomplete parts are still valid inputs.

Uploads are idempotent. Re-sending the same --part value (for example, after a retried CI step) overwrites the previous row for that part rather than double-counting.

Rules

  • --part-total must match across all parts. If runner 1 sends --part-total 3 but runner 2 sends --part-total 4, OtterWise uses the highest value seen and keeps waiting for the missing parts.
  • Each --part index should be unique per flag+component. Two uploads with --part 1 will overwrite each other.
  • Parts are scoped per flag and component. An upload with --flag unit-tests --part 1/2 has nothing to do with one using --flag integration-tests --part 1/2; each pair must complete on its own.
  • If one runner fails to upload, OtterWise will not post a PR comment or status check for that commit until the missing part is uploaded (or until a later commit arrives with complete coverage).
  • Values must be positive integers. --part 0, negative values, and values greater than --part-total are rejected by the uploader.

Troubleshooting

Symptom Cause & Fix
PR comment / status check never appears At least one part didn't upload. Check each runner's CI logs for the OtterWise uploader output. Re-run the failing shard with the same --part value to complete the set.
Coverage numbers look too low Likely a --part-total mismatch or a missing part. Verify every runner uses the same --part-total value and that indices 1 through N are each uploaded exactly once.
Uploader exits with "--part must be a positive integer" The matrix variable was empty or non-numeric. Echo the value before invoking the uploader to confirm it's set.
Uploader exits with "--part cannot be greater than --part-total" Your shard indexes exceed the declared total. Update --part-total to match the number of shards.