Purpose: This guide provides rules of thumb, best practices, and must-haves for designing robust, maintainable, and efficient pipelines. Use it as Confluence documentation or share with customers to standardize CI/CD across teams.
Key Models:
Strategy | Description | When to Use |
GitHub Flow |
| Fast-moving teams, continuous deploy to prod |
GitLab Flow | Combines feature branches, environment branches, and tags | Multi-environment deployments (dev, staging, prod) |
Gitflow |
| Large teams with formal release cadence |
Trunk-Based | Single | Continuous integration, high release frequency |
Best Practices:
Align branch types with environments: e.g.
feature/*
→ dev,release/*
→ staging,main
→ production.Enforce naming conventions (
feature/
,bugfix/
,hotfix/
,release/
) for pipeline automation.Keep branches short-lived (<1 week) to reduce merge conflicts.
Why It Matters: Tags provide immutable references to releases, enable reproducible builds, and drive semantic versioning.
Best Practices:
Use semantic versioning (e.g.
v{MAJOR}.{MINOR}.{PATCH}
) for release tags.Automate tag creation on merge to
release/*
ormain
with GitLab/GitHub CI jobs.Inject
CI_COMMIT_TAG
or equivalent into artifacts and Docker images.Maintain a CHANGELOG.md generated from commit messages or PR descriptions.
Group related steps into stages to optimize cache reuse, parallelism, and clarity.
Stage | Steps Included | Purpose |
Prep | Checkout, install dependencies, cache restore | Establish reproducible workspace |
Static Checks | Lint, code style, license scans | Fail fast on simple errors |
Build | Compile, transpile, package artifact | Produce versioned binary or library |
Unit Tests | Execute unit tests in parallel | Validate core logic quickly |
Integration Tests | Spin up test infra, run integration suites | Verify component interactions |
Security Scans | SAST, SCA, license compliance | Catch vulnerabilities and policy violations |
Containerize | Build Docker image | Standardize runtime environment |
Publish | Push to registry (npm, Docker, Maven) | Share artifacts for downstream pipelines |
Deploy (Staging) | Terraform plan/apply, deploy to staging | Test in production-like environment |
Smoke Tests | Quick health checks | Verify basic functionality |
Promote/Deploy (Prod) | Canary/blue-green deploy, post-deploy tests | Controlled release to production |
Cleanup | Teardown temporary resources | Free cost and prevent resource leaks |
Grouping Rules:
Fail Fast: Run lightweight checks (lint, license) before expensive stages.
Cache Smart: Install dependencies once in Prep to reuse caches across stages.
Parallelize: Run unit tests and static checks concurrently to shorten pipeline time.
Stage Isolation: Keep security scans separate to run on final artifacts.
Why Modular? Break monolithic pipelines into focused child pipelines for reuse, clarity, and cost control.
Patterns:
Parent Pipeline: Detects branch/type and orchestrates child pipelines.
Child Pipelines: Dedicated to specific concerns:
lint.yml
,build.yml
,test.yml
,security.yml
,deploy.yml
,cleanup.yml
.Promotion Pipelines: Separate pipeline triggered on tags to reuse validated artifacts without rebuild.
Benefits:
Faster Feedback: Feature branches trigger only
lint
+unit-test
child pipelines.Cost Efficiency: Skip heavy integration/security for quick PR validations.
Traceability: Parent-child links log exactly which pipeline ran when.
Trigger Events:
Push to feature branches: run PR validations (lint + unit tests).
Merge to
develop
ormain
: run full pipelines including integration and security.Tag push (
v*
): trigger promotion pipeline for production deploy.Scheduled Pipelines: nightly regression tests or periodic security scans.
Rules of Thumb:
Use branch glob patterns (
feature/*
,release/*
) in CI rules.Avoid running full pipelines on every commit—use rules to limit when heavy stages execute.
Leverage
when: manual
ormanual
approvals for production-critical steps.
Secret Management: Use vaults or built-in CI secrets to avoid leaking credentials.
RBAC & Audit Logs: Enforce least-privilege permissions for pipeline roles and runners.
SAST & SCA: Integrate static code analysis (SonarQube/CodeQL) and dependency scanning (Trivy) early.
Policy-as-Code: Use tools like Open Policy Agent or GitLab’s security policies to block non-compliant changes.
Metrics & Logs:
Record pipeline duration, success/failure rates, queue times.
Emit logs to centralized logging (ELK, Datadog).
Alerts & Dashboards:
Notify on pipeline failures via Slack, email, or PagerDuty.
Build dashboards showing pipeline health, average lead time, and MTTR for failures.
Regular Reviews:
Audit pipeline performance quarterly.
Remove stale stages and unused steps.
Cost Control:
Clean up old artifacts and images automatically.
Right-size runner resources and configure auto-scaling.
Documentation:
Keep pipeline-as-code docs in repo (README.md).
Document common pipeline patterns in Confluence.
Fail Fast: Catch simple errors before heavy stages.
Keep It DRY: Use templates/includes for common logic.
Version Pipelines: Tag pipeline configs same as code releases.
Test Pipeline Changes: Use a
ci-test
branch to validate changes.Secure by Default: Default to deny, explicitly whitelist what’s allowed.
Use Environment Parity: Mirror configs across dev, staging, and prod.
A perfectly designed pipeline balances speed, security, and maintainability. By standardizing your branching strategy, tagging scheme, stage grouping, and adopting modular pipelines, you empower teams to innovate rapidly while maintaining high quality and reliability.
Share this guide with your teams and customers to establish a consistent CI/CD foundation that scales with your growth.