Automated Versioning With CI/CD: A QoL Improvement For Amml-utils

Hey team, let's talk about making our lives a little easier when it comes to versioning the amml-utils project. Right now, we're manually tagging new versions, which, let's be honest, can sometimes slip our minds, especially when we're deep in the weeds implementing a new dataset. This can create some friction in our workflow, particularly when integrating these updates with projects like amml-python-base where specific versions are referenced in slim_requirements.txt.

The Current Manual Tagging Process and Its Downsides

Currently, the process requires a conscious effort to remember to create a new tag for each release. This is where we need to be very careful. It's easy to get caught up in the details of dataset implementation and forget this crucial step. Forgetting to tag means that the changes aren't immediately available for use in other projects and can also cause confusion when trying to reproduce results or track down the source of a bug. The manual nature of this process introduces room for human error and can slow down the overall development cycle. Every time we introduce a change, no matter how small, we have to remember to do it manually. This can lead to delays and inefficiencies. This manual step isn't just about the act of creating the tag; it's about the subsequent updates, testing, and deployment processes that rely on the tag being present. The more complex a project becomes, the more important it is to automate routine tasks like versioning. The benefits of automation extend beyond mere convenience; they provide a more reliable and consistent workflow. This consistency is essential for maintaining a high level of code quality and ensuring that new features and bug fixes are deployed quickly and efficiently.

The Proposed Solution: CI/CD for Automatic Tagging

My suggestion is to automate the tagging process using a CI/CD (Continuous Integration/Continuous Deployment) action. The idea is simple: the CI/CD system would automatically create a tag with an incremented patch version (e.g., 1.1.3 becomes 1.1.4) whenever changes are merged into the main branch. This means that for small, inconsequential changes – the kind that don't warrant a major or minor version bump – the tagging would happen automatically. We, as developers, would still be responsible for incrementing the major or minor versions for more significant changes. The main idea here is to automate the repetitive tasks to give us the ability to focus on the more important and challenging aspects of our projects. Imagine being able to merge a small fix and know that a new version tag is automatically created without any additional work on your part. This kind of automation is a Quality of Life (QoL) improvement, as it reduces the cognitive load on developers and minimizes the chances of errors.

The benefits extend to everyone involved in the project. The reduction of manual steps leads to fewer mistakes and a faster overall process. The system becomes more reliable and easier to maintain. When you automate repetitive steps, you free yourself to work on more valuable tasks like writing new code and improving project architecture.

Benefits of Automated Tagging

• Reduced Human Error: Automating the process minimizes the risk of forgetting to tag a release, which can lead to inconsistencies and problems downstream.
• Increased Efficiency: Automated tagging saves time and effort, allowing us to focus on other tasks.
• Improved Consistency: Ensures that all releases are tagged consistently, making it easier to track changes and manage versions.
• Faster Release Cycles: Automated tagging can speed up the release process, as it removes a manual step.

Implementation Details and Considerations

Implementing this would involve setting up a CI/CD pipeline (e.g., using GitHub Actions, GitLab CI, or Jenkins) that triggers on pushes to the main branch. The action would then: Firstly, it gets the latest tag, then parses the version number. Increment the patch version. Finally, create a new tag with the incremented version number. We also need to consider how to handle cases where a new major or minor version is needed. In this case, we would manually update the version number in a configuration file (e.g., pyproject.toml, setup.py, or similar) and the CI/CD action would then pick up the new version from there. We'll need to carefully document the process and provide clear guidelines for when and how to increment the major/minor versions versus relying on automatic patch versioning. The configuration file becomes a single source of truth for your project’s version. The CI/CD pipeline should be designed in a way that it minimizes the chance of errors. The process must be thoroughly tested before it is implemented to ensure that all the edge cases are accounted for. This includes handling the first tag of the project.

Potential Challenges and Solutions

• Version Conflicts: If multiple developers merge changes simultaneously, there could be conflicts when the CI/CD action tries to create a tag. This could be addressed by adding some logic to handle conflicts or by queuing the tagging process.
• Versioning Standards: We need to clearly define and document our versioning standards (e.g., Semantic Versioning) to ensure consistency. To make sure that the versioning remains consistent, we will need to standardize the versioning style throughout the team.
• Testing: Thorough testing is necessary to ensure that the CI/CD action works correctly and doesn't introduce any issues. We must ensure that the automatic tagging process works smoothly.

Conclusion: A Step Towards a Smoother Workflow

Automating the tagging process with a CI/CD action presents a practical Quality of Life (QoL) improvement for the amml-utils project. By reducing manual effort and potential errors, we can streamline our workflow and ensure consistent, reliable releases. This improvement can lead to increased efficiency and, most importantly, provide a better development experience for everyone involved. What do you think, @HenKlei and @HoeflerMatthias? Your feedback and thoughts are greatly appreciated!

This simple addition to our workflow can yield great improvements in the long run. The implementation should be straightforward, and the benefits are clear: less manual effort, fewer errors, and a more streamlined development process. Automating tasks frees up valuable time and cognitive resources for developers, allowing them to focus on more complex and challenging aspects of the project. Automating routine tasks also reduces the risk of human error, which can lead to more stable and reliable releases.

To learn more about CI/CD and automated versioning, check out these resources:

• GitHub Actions Documentation: Official documentation for GitHub Actions, a popular CI/CD platform.
• Semantic Versioning: A widely used standard for versioning software.