Merge pull request #790 from vespa-engine/thomasht86/pyvespa-release-…

…pipeline

CI: pyvespa release pipeline

.github/workflows/pyvespa.yml

@@ -0,0 +1,52 @@
+name: pyvespa - Release and upload PyPI
+
+on:
+  workflow_dispatch:
+  release:
+    types: [published]
+jobs:
+
+  update_version:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .[build]
+
+      - name: Set version from ref tag that triggered the workflow (strip prefix 'v' from tag)
+        id: set_version
+        run: echo "version=${{ format('v{0}', github.ref_name) }}" >> $GITHUB_OUTPUT
+
+      - name: Update version
+        run: |
+          echo "Updating pyvespa version to ${{ steps.set_version.outputs.version }}"
+          python vespa/utils/update_version.py --version ${{ steps.set_version.outputs.version }}
+
+      - name: Build
+        run: |
+          python -m build
+      
+      - name: Upload to PyPI
+        if: github.event_name == 'release'
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN_PYVESPA }} #TODO: this must be added to secrets
+        run: python -m twine upload dist/* 
+
+      - name: Create Pull Request
+        if: github.event_name == 'release'
+        uses: peter-evans/create-pull-request@v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          commit-message: "Update pyvespa version to ${{ steps.set_version.outputs.version }}"
+          title: "Update pyvespa version to ${{ steps.set_version.outputs.version }}"
+          body: "This PR updates the pyvespa version to ${{ steps.set_version.outputs.version }}"
+          branch: "update-pyvespa-version-${{ steps.set_version.outputs.version }}"
+          base: "master"
+          labels: "pyvespa"

README.md

@@ -22,13 +22,14 @@ The main goal of the library is to allow for faster prototyping and get familiar
 ## vespacli
 
 This repo also contains the python wrapper for the [Vespa CLI](https://docs.vespa.ai/en/vespa-cli).
-See [README](https://github.com/vespa-engine/pyvespa/tree/master/vespacli) and [Veso]
+See [README](https://github.com/vespa-engine/pyvespa/tree/master/vespacli).
 
 ## License
-Code licensed under the Apache 2.0 license. See [LICENSE](LICENSE) for terms.
 
+Code licensed under the Apache 2.0 license. See [LICENSE](LICENSE) for terms.
 
 ## Development environment
+
 To install editable version of the library with dev dependencies, run the following command from the root directory of the repository:
 
 ```python
@@ -40,29 +41,16 @@ Note that this will enforce linting and formatting with [Ruff](https://github.co
 This means that you may get an error message when trying to commit changes if the code does not pass the linting and formatting checks. The errors are detailed in the output, and you can optionally run manually with `ruff` CLI-tool.
 
 ## Releases
-Find releases and release notes on [GitHub](https://github.com/vespa-engine/pyvespa/releases).
-
 
-### Release instructions
-* Check out master branch
-* Temporarily change library version number in `get_target_version()` in [setup.py](setup.py) to the new version,
-  e.g. "0.16.0".
-* Run from the pyvespa root directory to create the library files:
-
-```
-python3 -m pip install --upgrade pip
-python3 -m pip install twine wheel
+Find releases and release notes on [GitHub](https://github.com/vespa-engine/pyvespa/releases).
 
-python3 setup.py sdist bdist_wheel
-``` 
+### Release details
 
-With write access to [pypi.org/project/pyvespa/](https://pypi.org/project/pyvespa/),
-upload, this requires username "__token__" and the token value as password, including the pypi- prefix:
+The release flow is semi-automated, but involves a few manual steps.
 
-```
-python3 -m twine upload dist/*
-```
+1. Create a new release from [github.com/vespa-engine/pyvespa/releases/new](https://github.com/vespa-engine/pyvespa/releases/new).
+2. Make sure to tag the release with the version number, e.g., `v0.41.0`.
+3. This tag will trigger a github action that will publish the package to [PyPI](https://pypi.org/project/pyvespa/).
+4. A PR will also be automatically created to update the affected files with the new version. This PR should be merged to keep the version updated in the repository.
 
-At this point, the package has been released.
-Create a new release tag at [github.com/vespa-engine/pyvespa/releases/new](https://github.com/vespa-engine/pyvespa/releases/new)
-with a summary of the code changes.
+This workflow can also be dispatched manually, but note that steps 3 and 4 will ONLY be triggered by a release.

pyproject.toml

@@ -1,3 +1,5 @@
+# Copyright Vespa.ai. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
+
 [build-system]
 requires = ["setuptools>=69", "wheel", "build", "twine", "versioneer[toml]"]
 build-backend = "setuptools.build_meta"
@@ -8,9 +10,7 @@ version = "0.40.0"
 description = "Python API for vespa.ai"
 readme = "README.md"
 keywords = ["vespa", "search engine", "data science"]
-classifiers = [
-    "License :: OSI Approved :: Apache Software License"
-]
+classifiers = ["License :: OSI Approved :: Apache Software License"]
 dependencies = [
     "requests",
     "requests_toolbelt",
@@ -19,13 +19,10 @@ dependencies = [
     "cryptography",
     "aiohttp",
     "tenacity",
-    "typing_extensions"
+    "typing_extensions",
 ]
-requires-python = ">=3.8"
 
-[project.urls]
-homepage = "https://pypi.org/project/pyvespa"
-repository = "https://github.com/vespa-engine/pyvespa"
+requires-python = ">=3.8"
 
 [[project.authors]]
 name = "Thiago G. Martins"
@@ -34,7 +31,10 @@ name = "Thiago G. Martins"
 name = "Thomas Thoresen"
 email = "[email protected]"
 
-# License file 
+[project.urls]
+homepage = "https://pypi.org/project/pyvespa"
+repository = "https://github.com/vespa-engine/pyvespa"
+
 [project.license]
 file = "LICENSE"
 
@@ -47,42 +47,66 @@ dev = [
     "nbconvert",
     "runnb",
     "ruff",
-    "pre-commit"
+    "pre-commit",
 ]
 build = [
-"setuptools==69.0.3",
-"build==1.0.3",
-"twine==4.0.1",
-"toml==0.10.2",
-"requests~=2.26.0",
-"bump-my-version==0.21.0",
+    "setuptools==69.0.3",
+    "build==1.0.3",
+    "twine==4.0.1",
+    "toml==0.10.2",
+    "requests~=2.26.0",
+    "ruff",
 ]
 
 [tool.setuptools]
 packages = ["vespa"]
 
+[tool.ruff]
+exclude = [
+    ".bzr",
+    ".direnv",
+    ".eggs",
+    ".git",
+    ".git-rewrite",
+    ".hg",
+    ".ipynb_checkpoints",
+    ".mypy_cache",
+    ".nox",
+    ".pants.d",
+    ".pyenv",
+    ".pytest_cache",
+    ".pytype",
+    ".ruff_cache",
+    ".svn",
+    ".tox",
+    ".venv",
+    ".vscode",
+    "__pypackages__",
+    "_build",
+    "buck-out",
+    "build",
+    "dist",
+    "node_modules",
+    "site-packages",
+    "venv",
+]
+line-length = 88
+indent-width = 4
+
 [tool.setuptools.package-data]
-vespa = ["py.typed","templates/*"]
+vespa = ["py.typed", "templates/*"]
+
+[tool.ruff.lint]
+select = ["E4", "E7", "E9", "F"]
+ignore = []
+fixable = ["ALL"]
+unfixable = []
+dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
 
-[tool.bumpversion]
-current_version = "0.40.0"
-excluded_paths = ["docs", "vespacli", "setup.py"]
-parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
-serialize = ["{major}.{minor}.{patch}"]
-search = "{current_version}"
-replace = "{new_version}"
-regex = false
-ignore_missing_version = false
-ignore_missing_files = false
-tag = true
-sign_tags = true
-tag_name = "v{new_version}"
-tag_message = "Bump version: {current_version} → {new_version}"
-allow_dirty = false
-commit = true
-message = "Bump version: {current_version} → {new_version}"
-commit_args = ""
-[[tool.bumpversion.files]]
-filename = "pyproject.toml"
-[[tool.bumpversion.files]]
-filename = "vespa/__init__.py"
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+skip-magic-trailing-comma = false
+line-ending = "auto"
+docstring-code-format = false
+docstring-code-line-length = "dynamic"

vespa/utils/update_version.py

@@ -0,0 +1,41 @@
+# Copyright Vespa.ai. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
+
+# This script should modify top level pyproject.toml to update the version of pyvespa
+# And modify vespa/__init__.py to update the version of pyvespa
+
+import sys
+import re
+from pathlib import Path
+import argparse
+
+PYPROJECT_TOML_PATH = Path(__file__).parent.parent.parent / "pyproject.toml"
+VERSION_FILE_PATH = Path(__file__).parent.parent / "__init__.py"
+
+
+def update_version(new_version: str):
+    # Update version in pyproject.toml
+    with open(PYPROJECT_TOML_PATH, "r") as f:
+        content = f.read()
+    new_content = re.sub(r'version = ".*"', f'version = "{new_version}"', content)
+    with open(PYPROJECT_TOML_PATH, "w") as f:
+        f.write(new_content)
+
+    # Update version in vespacli/_version_generated.py
+    with open(VERSION_FILE_PATH, "r") as f:
+        content = f.read()
+    new_content = re.sub(
+        r'__version__ = ".*"', f'__version__ = "{new_version}"', content
+    )
+    with open(VERSION_FILE_PATH, "w") as f:
+        f.write(new_content)
+    print(f"Updated version to {new_version}")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Update pyvespa version")
+    parser.add_argument(
+        "-v", "--version", type=str, help="New version to set", required=True
+    )
+    args = parser.parse_args()
+    update_version(args.version)
+    sys.exit(0)