CI/Release Workflow with python-semantic-release and Woodpecker CI

Designing a CI/release workflow with python-semantic-release and Woodpecker CI that supports RC builds while maintaining a grouped changelog requires careful configuration of both tools. The continuous integration pipeline must be set up to automatically generate release candidates during development cycles and then group all changes into a unified changelog when final releases are created. This approach ensures smooth versioning and comprehensive documentation of changes across your software lifecycle.

Contents

• Understanding Continuous Integration with python-semantic-release and Woodpecker CI
• Configuring RC Builds with python-semantic-release
• Setting Up Woodpecker CI for Your Workflow
• Implementing Changelog Grouping for Final Releases
• Creating a Complete CI/CD Workflow
• Best Practices and Troubleshooting

Understanding Continuous Integration with python-semantic-release and Woodpecker CI

Continuous integration forms the backbone of modern software development, enabling teams to automate the process of building, testing, and deploying code changes. When you combine python-semantic-release with Woodpecker CI, you create a powerful automated release system that follows semantic versioning principles while maintaining clear documentation of changes. This integration allows you to automate your entire release process, from initial commit to final deployment, while maintaining control over release candidates and final releases.

python-semantic-release is a Python tool designed specifically for automating software releases using semantic versioning and conventional commit message formatting. It analyzes your commit history to determine the next version number and can automatically create release notes, publish releases to package repositories, and even update version numbers in your project files. Unlike manual release processes, this tool ensures consistency in versioning and reduces human error when managing releases across multiple projects.

Woodpecker CI, on the other hand, serves as your continuous integration platform. Originally forked from Drone 0.8 in 2019, Woodpecker provides a lightweight yet powerful CI/CD engine built on Docker containers. It supports multiple workflows and can be easily configured to run your python-semantic-release automation as part of your build pipeline. The platform’s community-focused approach ensures it remains free and open source while offering extensive customization options for your CI/CD needs.

The synergy between these tools creates a seamless workflow where code changes trigger automated builds, release candidates are generated for testing, and final releases are prepared with comprehensive changelogs. This approach not only saves time but also provides transparency into your release process, making it easier for teams to track progress and understand the impact of changes.

Key Benefits of the Combined Approach

When you integrate python-semantic-release with Woodpecker CI, you gain several advantages:

1. Automated version management - No more manual version bumping or guessing about what comes next
2. Release candidate automation - Generate RC builds for testing without manual intervention
3. Comprehensive changelogs - Maintain clear documentation of all changes between releases
4. Consistent deployment process - Standardize how releases are created and deployed across projects
5. Reduced human error - Automate repetitive tasks that often lead to mistakes

Configuring RC Builds with python-semantic-release

Release candidate (RC) builds are essential for testing new features and bug fixes before they reach production. With python-semantic-release, you can configure your project to automatically generate RC builds from specific branches, ensuring that your testing environment always has access to the latest changes while maintaining stable release cycles.

The configuration for RC builds is handled in your pyproject.toml file. To enable RC builds, you’ll need to define pre-release branches with their own matching patterns and pre-release tokens. Here’s a practical example of how to configure RC builds:

[tool.semantic_release]
branch = "main"
build_command = "python -m build --sdist --wheel ."
version_toml = ["pyproject.toml:tool.poetry.version"]
major_on_zero = true

[tool.semantic_release.branches.alpha]
match = "^(feat|fix|perf)/.+"
prerelease = true
prerelease_token = "alpha"

[tool.semantic_release.branches.beta]
match = "^(feat|fix|perf)/.+"
prerelease = true
prerelease_token = "beta"

In this configuration, we’ve defined two pre-release branches: alpha and beta. The match pattern determines which commit messages trigger releases on these branches. Any commit message starting with feat, fix, or perf followed by a slash will trigger a new RC build on the respective branch.

The prerelease_token field specifies how these pre-releases should be named. For example, a release on the alpha branch might be versioned as 1.0.0a1, while a beta release might be 1.0.0b1. This naming convention follows semantic versioning standards and makes it clear which stage of development each release represents.

Creating RC Builds in Practice

When working with RC builds, you’ll typically follow this workflow:

1. Create feature or fix branches - Developers create branches for their changes using conventional commit message prefixes
2. Push to pre-release branches - Changes are pushed to alpha or beta branches depending on the maturity level
3. Automatic RC generation - python-semantic-release detects the new commits and creates RC builds automatically
4. Testing and validation - Your Woodpecker CI pipeline builds and tests these RC versions
5. Promotion or rollback - Based on testing results, RC builds are either promoted to final releases or rolled back

You can also configure multiple RC channels to support different testing environments. For example, you might have alpha builds for internal testing and beta builds for public testing, each with their own release cadence and stability requirements.

Authentication for RC Builds

When publishing RC builds to package repositories or version control systems, you’ll need to provide authentication tokens. In your Woodpecker CI configuration, you can set environment variables like GH_TOKEN for GitHub or GITLAB_TOKEN for GitLab. These tokens allow python-semantic-release to create releases, manage release notes, and update version numbers in your repository.

The tokens should be configured as secrets in your Woodpecker CI setup and referenced in your pipeline configuration. This ensures secure authentication without exposing sensitive credentials in your codebase.

Setting Up Woodpecker CI for Your Workflow

Woodpecker CI provides a lightweight yet powerful platform for running your continuous integration pipeline. Setting up Woodpecker CI to work with python-semantic-release requires configuring a pipeline that triggers on the right events and executes the necessary commands for building, testing, and releasing your software.

The first step in setting up Woodpecker CI is to install it on your infrastructure. Woodpecker can be deployed on various platforms including Kubernetes, Docker, or as a standalone binary. Once installed, you’ll need to configure it to connect to your Git repository (GitHub, GitLab, Gitea, etc.) and authenticate with appropriate permissions to read repository contents, create releases, and manage issues.

Creating the Woodpecker Pipeline Configuration

Your Woodpecker CI pipeline is defined in a .woodpecker.yml file at the root of your repository. This YAML file specifies the steps that should be executed for each build. Here’s a comprehensive example of how to configure Woodpecker CI for python-semantic-release:

pipeline:
 # Build and test for all branches
 build:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m build --sdist --wheel .
 - pip install dist/*.whl
 when:
 event: [push, pull_request]

 # Run semantic release for main branch (final releases)
 release:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m python_semantic_release publish
 secrets:
 - GH_TOKEN
 when:
 event: push
 branch: main
 status: success

 # Generate RC builds for pre-release branches
 rc-build:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m python_semantic_release publish
 secrets:
 - GH_TOKEN
 when:
 event: push
 branch: [alpha, beta]
 status: success

This configuration defines three main pipeline steps:

1. Build - Runs on all pushes and pull requests to ensure code compiles and basic tests pass
2. Release - Executes semantic release on the main branch when builds succeed, creating final releases
3. RC-build - Runs semantic release on alpha and beta branches, generating release candidates

The secrets section references environment variables that should be configured in your Woodpecker CI settings. These typically include authentication tokens for your Git provider and package repositories.

Environment Variables and Secrets

To ensure secure authentication, you should configure the following environment variables in your Woodpecker CI settings:

• GH_TOKEN - GitHub personal access token with appropriate permissions
• GITLAB_TOKEN - GitLab access token if using GitLab
• PYPI_TOKEN - PyPI token if publishing to PyPI
• GITHUB_TOKEN - GitHub token for creating releases and issues

These tokens should have minimal permissions required for your workflow. For example, a GitHub token for python-semantic-release typically needs:

• repo scope to create releases
• read:packages scope if publishing to GitHub Packages
• issues scope if creating GitHub issues for changelogs

Branch Protection and Workflow Triggers

Woodpecker CI allows you to fine-tune when your pipeline runs using the when condition. In our example, we’ve configured different triggers for different types of releases:

• Final releases only run on the main branch after successful builds
• RC builds run on alpha and beta branches
• Regular builds run on all branches and pull requests

You can further customize these triggers based on your workflow needs. For example, you might want to skip certain branches or only run releases after specific conditions are met.

Multi-Environment Support

If you need to support multiple environments (development, staging, production), you can extend your Woodpecker CI configuration to include environment-specific steps. This might involve:

1. Deploying to staging servers after RC builds
2. Running additional integration tests before final releases
3. Promoting releases from staging to production after manual approval

Each environment can have its own pipeline steps and conditions, allowing you to create a comprehensive deployment workflow that matches your organization’s processes and requirements.

Implementing Changelog Grouping for Final Releases

Changelog grouping is essential for providing clear documentation of what changed between releases. With python-semantic-release, you can configure how changelogs are generated and formatted, ensuring they present information in a way that’s useful for both developers and end users.

The key to proper changelog grouping lies in the configuration of templates and output formats. By default, python-semantic-release generates changelogs based on conventional commit messages, but you can customize how these are grouped and presented in your final releases.

Configuring Changelog Templates

In your pyproject.toml file, you can specify custom templates for changelog generation. Here’s how to configure changelog grouping for final releases:

[tool.semantic_release.changelog.default_templates]
changelog_file = "docs/source/CHANGELOG.rst"
output_format = "rst"

This configuration tells python-semantic-release to write the changelog to a specific file using reStructuredText format. You can customize the template further by creating your own template files and referencing them in the configuration:

[tool.semantic_release.changelog]
changelog_file = "CHANGELOG.md"
template_dir = "templates"

With this setup, you can create multiple template files in the templates directory for different types of changes (features, bug fixes, breaking changes, etc.), allowing for more granular changelog grouping.

Advanced Changelog Customization

For more sophisticated changelog grouping, you can leverage the power of Jinja2 templates. Create a custom template file like templates/changelog.j2:

# Changelog

{% for version in versions %}
## {{ version.version }} - {{ version.date }}

### Features
{% for commit in version.commits %}
{% if commit.type == 'feat' %}
- {{ commit.message }}
{% endif %}
{% endfor %}

### Bug Fixes
{% for commit in version.commits %}
{% if commit.type == 'fix' %}
- {{ commit.message }}
{% endif %}
{% endfor %}

### Breaking Changes
{% for commit in version.commits %}
{% if commit.type == 'feat' and commit.scope == 'breaking' %}
- **BREAKING**: {{ commit.message }}
{% endif %}
{% endfor %}

{% endfor %}

This template groups changes by category (features, bug fixes, breaking changes) for each release, making the changelog more organized and easier to read. You can further customize it to include additional metadata like commit hashes, author information, or links to pull requests.

Integrating Changelog with Documentation

To ensure your changelog is accessible to users, integrate it directly into your project documentation. The changelog should be prominently displayed in your project’s README or documentation site, allowing users to quickly understand what has changed between versions.

Consider using a documentation tool like Sphinx (for Python projects) with a theme that supports changelog integration. You can also include links to the full changelog in your release announcements and update notifications.

Automating Changelog Updates

One of the benefits of using python-semantic-release is that it automatically updates your changelog with each release. This ensures that your documentation stays current without manual intervention. However, you may want to add additional information to releases, such as:

• Performance improvements
• Deprecated features
• Migration guides
• Special thanks or acknowledgments

You can extend the changelog template to include these elements while maintaining the automated grouping of commits. This hybrid approach combines the efficiency of automated changelog generation with the human touch that makes releases more comprehensive and user-friendly.

Creating a Complete CI/CD Workflow

Now that we’ve covered the individual components, let’s create a complete CI/CD workflow that integrates python-semantic-release with Woodpecker CI, supporting both RC builds and final releases with proper changelog grouping. This comprehensive workflow will automate your entire release process from commit to deployment.

Workflow Overview

The complete workflow consists of the following stages:

1. Development - Developers create feature branches and commit changes using conventional commit messages
2. RC Build - Changes are merged to pre-release branches, triggering RC builds via Woodpecker CI
3. Testing - RC versions are tested in staging environments
4. Final Release - Approved changes are merged to main, triggering final releases
5. Deployment - Final releases are deployed to production
6. Documentation - Changelogs are automatically updated and published

Complete Configuration Files

Here are the complete configuration files needed for this workflow:

1. .woodpecker.yml - Woodpecker CI Pipeline:

# Pipeline configuration for python-semantic-release with Woodpecker CI
pipeline:
 # Build and test for all branches
 build:
 image: python:3.11
 commands:
 - pip install build python-semantic-release pytest
 - python -m build --sdist --wheel .
 - pip install dist/*.whl
 - pytest tests/
 when:
 event: [push, pull_request]

 # Test RC builds in staging environment
 test-staging:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m build --sdist --wheel .
 - pip install dist/*.whl
 - pytest tests/integration/
 when:
 event: push
 branch: [alpha, beta]
 status: success

 # Generate RC builds
 rc-build:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m python_semantic_release publish
 secrets:
 - GH_TOKEN
 - PYPI_TOKEN
 when:
 event: push
 branch: [alpha, beta]
 status: success

 # Final release workflow
 release:
 image: python:3.11
 commands:
 - pip install build python-semantic-release
 - python -m python_semantic_release publish
 secrets:
 - GH_TOKEN
 - PYPI_TOKEN
 when:
 event: push
 branch: main
 status: success

# Notifications
notify:
 email:
 on_success: change
 on_failure: always

2. pyproject.toml - python-semantic-release Configuration:

[build-system]
requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"]
build-backend = "setuptools.build_meta"

[tool.semantic_release]
branch = "main"
build_command = "python -m build --sdist --wheel ."
version_toml = ["pyproject.toml:tool.poetry.version"]
major_on_zero = true
commit_author = "semantic-release <semantic-release>"
commit_parser = "angular"
changelog_file = "CHANGELOG.md"
changelog_start_re = "^## "
logging_use_named_logger = true

[tool.semantic_release.branches.alpha]
match = "^(feat|fix|perf|docs)/.+"
prerelease = true
prerelease_token = "alpha"

[tool.semantic_release.branches.beta]
match = "^(feat|fix|perf|docs)/.+"
prerelease = true
prerelease_token = "beta"

[tool.semantic_release.changelog]
changelog_file = "CHANGELOG.md"
template_dir = "templates"

3. templates/changelog.j2 - Custom Changelog Template:

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [{{ version.version }}] - {{ version.date }}

### Added
{% for commit in version.commits %}
{% if commit.type == 'feat' %}
- {{ commit.message }}
{% endif %}
{% endfor %}

### Changed
{% for commit in version.commits %}
{% if commit.type == 'feat' and commit.scope == 'breaking' %}
- **BREAKING**: {{ commit.message }}
{% endif %}
{% endfor %}

### Fixed
{% for commit in version.commits %}
{% if commit.type == 'fix' %}
- {{ commit.message }}
{% endif %}
{% endfor %}

### Documentation
{% for commit in version.commits %}
{% if commit.type == 'docs' %}
- {{ commit.message }}
{% endif %}
{% endfor %}

Workflow Execution Flow

When you implement this complete workflow, here’s how it will execute:

1. A developer creates a feature branch with a conventional commit like feat: Add user authentication
2. After pushing the branch, Woodpecker CI runs the build step to ensure the code compiles and tests pass
3. When the feature branch is merged to the alpha branch, the rc-build step triggers:

• python-semantic-release analyzes the commits
• Creates a new RC version (e.g., 1.0.0a1)
• Publishes to GitHub Packages and PyPI
• Updates the changelog with the new RC build

4. The RC version is tested in the staging environment
5. Once testing is complete and the feature is approved, it’s merged to main
6. The final release workflow triggers:

• python-semantic-release analyzes all changes since the last release
• Creates a final version (e.g., 1.0.0)
• Groups all changes into a comprehensive changelog
• Publishes to all configured repositories
• Creates a GitHub release with the changelog

Handling Complex Release Scenarios

This workflow can be extended to handle more complex scenarios:

1. Multiple Release Channels: You can create additional pre-release branches (like hotfix) for urgent fixes
2. Manual Approval: Add a manual step in Woodpecker CI before final releases to require human approval
3. Environment-Specific Deployments: Add deployment steps for specific environments after releases
4. Rollback Mechanisms: Implement steps to rollback releases if issues are discovered post-deployment

Monitoring and Feedback

To ensure your workflow runs smoothly, implement monitoring and feedback mechanisms:

1. Webhook Notifications: Configure notifications for successful and failed builds
2. Dashboard Monitoring: Use Woodpecker’s dashboard to track build status and performance
3. Release Analytics: Monitor release frequency, success rates, and common failure points
4. Team Communication: Integrate with Slack, Discord, or other communication tools for real-time notifications

Best Practices and Troubleshooting

Implementing a CI/release workflow with python-semantic-release and Woodpecker CI requires attention to detail and adherence to best practices. This section covers essential recommendations for maintaining a smooth release process and troubleshooting common issues that may arise.

Best Practices for Continuous Integration

Effective continuous integration relies on following established practices that ensure quality, consistency, and reliability in your releases:

1. Conventional Commits: Enforce conventional commit message formatting across your team. This ensures python-semantic-release can correctly categorize changes and generate accurate changelogs. The format should follow type(scope): description where type is one of: feat, fix, docs, style, refactor, test, chore.

2. Branch Strategy: Implement a clear branching strategy. Consider using:

• main for production-ready code
• develop for integration testing
• alpha and beta branches for pre-release builds
• Feature branches for individual development work

3. Automated Testing: Maintain comprehensive test coverage that runs on every build. This includes unit tests, integration tests, and potentially end-to-end tests for critical functionality.

4. Code Quality Checks: Integrate code quality tools like flake8, black, and mypy into your CI pipeline to catch issues before they reach production.

5. Security Scanning: Add security scanning steps to detect vulnerabilities in dependencies before they’re included in releases.

Release Management Best Practices

When managing releases with python-semantic-release, consider these best practices:

1. Version Control Integration: Ensure your version control system is properly configured to work with python-semantic-release. This includes setting up appropriate authentication tokens and repository permissions.

2. Release Channels: Create multiple release channels for different stages of your software lifecycle. This allows you to test features in RC builds before promoting them to final releases.

3. Changelog Maintenance: Regularly review and maintain your changelog templates. Ensure they provide clear, useful information for both technical users and end-users.

4. Release Coordination: Coordinate releases across teams to avoid conflicts. Consider using release calendars and communication channels to keep everyone informed about upcoming releases.

5. Rollback Strategy: Always have a rollback plan in place. This might include keeping previous releases available for quick redeployment or implementing automatic rollback mechanisms if critical issues are detected.

Common Issues and Troubleshooting

Even with careful configuration, you may encounter issues with your CI/release workflow. Here are some common problems and their solutions:

1. python-semantic-release not detecting version changes

• Cause: Commit messages don’t follow conventional format
• Solution: Check your commit messages match the expected patterns. Use tools like commitlint to validate commits before they’re merged.

2. RC builds not being created

• Cause: Branch configuration doesn’t match your actual branch names
• Solution: Verify your branch names in .woodpecker.yml match the actual branch names in your repository.

3. Changelog grouping not working as expected

• Cause: Template configuration issues or incorrect template file paths
• Solution: Verify your template files exist and paths are correct. Test with a simple template first to isolate issues.

4. Authentication failures during publishing

• Cause: Insufficient permissions in authentication tokens
• Solution: Ensure your tokens have the required scopes. For GitHub, this typically includes repo for creating releases and read:packages for package publishing.

5. Woodpecker CI pipeline failing unexpectedly

• Cause: Missing dependencies or environment variables
• Solution: Check that all required dependencies are installed in your Docker images and that all necessary environment variables are configured.

Performance Optimization

To ensure your CI/release workflow runs efficiently:

1. Use Caching: Configure Docker layer caching in Woodpecker CI to speed up builds that reuse dependencies
2. Parallelize Steps: Run independent steps in parallel to reduce total build time
3. Optimize Docker Images: Use smaller, more efficient base images and only install necessary dependencies
4. Schedule Builds: Consider scheduling less critical builds during off-hours to balance resource usage

Security Considerations

Security should be a priority in your CI/release workflow:

1. Secret Management: Use secure secret management systems and regularly rotate authentication tokens
2. Dependency Scanning: Scan dependencies for vulnerabilities before including them in releases
3. Access Control: Implement proper access controls for your CI system and repositories
4. Audit Logging: Maintain comprehensive audit logs of all release activities for compliance and troubleshooting

Monitoring and Maintenance

Regular monitoring and maintenance ensure your workflow continues to function effectively:

1. Performance Monitoring: Track build times, success rates, and resource usage
2. Error Tracking: Implement error tracking to quickly identify and resolve issues
3. Regular Updates: Keep python-semantic-release and Woodpecker CI updated to benefit from improvements and security patches
4. Team Training: Ensure team members understand the workflow and can troubleshoot basic issues

By following these best practices and maintaining vigilance for potential issues, you can create a robust CI/release workflow with python-semantic-release and Woodpecker CI that reliably supports RC builds and final releases with proper changelog grouping.

Sources

1. python-semantic-release Documentation — Comprehensive guide for automating Python releases with semantic versioning: https://python-semantic-release.readthedocs.io/en/latest/
2. Woodpecker CI About Page — Information about Woodpecker CI, a community-driven CI/CD platform forked from Drone: https://woodpecker-ci.org/about
3. Woodpecker CI GitHub Repository — Source code and documentation for the Woodpecker CI platform: https://github.com/woodpecker-ci/woodpecker

Conclusion

Designing a CI/release workflow with python-semantic-release and Woodpecker CI that supports RC builds while maintaining grouped changelogs creates a powerful automated release system. By configuring pre-release branches in python-semantic-release and setting up appropriate Woodpecker CI pipeline steps, you can automatically generate release candidates for testing and then group all changes into comprehensive final releases.

The continuous integration process becomes seamless, with code changes triggering automated builds, RC versions being tested in staging environments, and final releases being prepared with complete documentation. This approach not only saves time but also reduces human error, provides transparency into your release process, and ensures consistent versioning across your software lifecycle.

When implementing this workflow, remember to follow best practices like conventional commits, comprehensive testing, and proper security measures. Regular monitoring and maintenance will ensure your CI/release workflow continues to function effectively as your project evolves. The result is a robust, automated release system that supports both rapid development cycles and stable production releases.