Skip to content

Automation

Continuous integration practices

We adhere to industry recommendations from the OSSF Scorecard project, among others.

Since all code changes require a pull request (PR) along with one or more reviewers, we automate quality and security checks before, during, and after a PR is merged to trunk (develop).

We use a combination of tools coupled with peer review to increase its compound effect in detecting issues early.

This is a snapshot of our automated checks at a glance.

Continuous Integration practices

Pre-commit checks

Pre-commit configuration.

Pre-commit checks are crucial for a fast feedback loop while ensuring security practices at the individual change level.

To prevent scenarios where these checks are intentionally omitted at the client side, we run at CI level too.

These run locally only for changed files

Pre-Pull Request checks

For an improved contributing experience, most of our checks can run locally. For maintainers, this also means increased focus on reviewing actual value instead of standards and security malpractices that can be caught earlier.

These are in addition to pre-commit checks.

  • Static typing analysis. mypy checks for static typing annotations to prevent common bugs in Python that may or may not lead to abuse.
  • Tests. We run unit, functional, and performance tests (see our definition). Besides breaking changes, we are investing in mutation testing to find additional sources of bugs and potential abuse.
  • Security baseline. bandit detects common security issues defined by Python Code Quality Authority (PyCQA).
  • Complexity baseline. We run a series of maintainability and cyclomatic checks to reduce code and logic complexity. This aids reviewers' cognitive overhead and long-term maintainers revisiting legacy code at a later date.

Pull Request checks

While we trust contributors and maintainers do go through pre-commit and pre-pull request due diligence, we verify them at CI level.

Checks described earlier are omitted to improve reading experience.

  • Semantic PR title. We enforce PR titles follow semantic naming, for example chore(category): change. This benefits contributors with a lower entry bar, no need for semantic commits. It also benefits everyone looking for an useful changelog message on what changed and where.
  • Related issue check. Every change require an issue describing its needs. This enforces a PR has a related issue by blocking merge operations if missing.
  • Acknowledgment check. Ensures PR template is used and every contributor is aware of code redistribution.
  • Code coverage diff. Educates contributors and maintainers about code coverage differences for a given change.
  • Contribution size check. Suggests contributors and maintainers to break up large changes (100-499 LOC) in smaller PRs. It helps reduce overlooking security and other practices due to increased cognitive overhead.
  • Dependency vulnerability check. Verifies any dependency changes for common vulnerability exposures (CVEs), in addition to our daily check on any dependencies used (e.g., Python, Docker, Go, etc.)
  • GitHub Actions security check. Enforces use of immutable 3rd-party GitHub Actions (_e.g., actions/checkout@<git-SHA>_) to prevent abuse. Upgrades are handled by a separate automated process that includes a maintainer review to also prevent unexpected behavior changes.

After merge checks

Checks described earlier are omitted to improve reading experience.

We strike a balance in security and contribution experience. These automated checks take several minutes to complete. Failures are reviewed by a maintainer on-call and before a release.

Continuous deployment practices

We adhere to industry recommendations from the OSSF Scorecard project, among others.

Releases are triggered by maintainers along with a reviewer - detailed info here. In addition to checks that run for every code change, our pipeline requires a manual approval before releasing.

We use a combination of provenance and signed attestation for our builds, source code sealing, SAST scanners, Python specific static code analysis, ephemeral credentials that last a given job step, and more.

This is a snapshot of our automated checks at a glance.

Continuous Deployment practices

More details to come


Last update: 2023-07-28