Explorar el Código

chore(ci): Add production PyPI publishing workflow

Introduces production PyPI publishing triggered by v* tag pushes, while
Test PyPI now requires manual dispatch. Both indexes never receive the
same run, ensuring proper separation between rehearsal and production.

Fixes #22786
Martin Hauser hace 3 semanas
padre
commit
642b4e5c2a

+ 54 - 32
.github/workflows/release.yml

@@ -44,7 +44,7 @@ jobs:
           cache: pip
 
       - name: Install build tooling
-        run: python -m pip install --upgrade build twine
+        run: python -m pip install --upgrade build twine packaging
 
       - name: Install documentation toolchain
         run: python -m pip install -r requirements.txt
@@ -60,6 +60,20 @@ jobs:
       - name: Check package metadata
         run: twine check dist/*
 
+      - name: Verify the release tag
+        # Both checks run here, in the unprivileged build job, against the wheel that becomes this
+        # run's artifact, so neither publish job has to check out the repository or execute its
+        # scripts while holding id-token: write. A failure here skips every downstream job.
+        if: startsWith(github.ref, 'refs/tags/v')
+        env:
+          TAG: ${{ github.ref_name }}
+        run: |
+          [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?$ ]] || {
+            echo "Ref '$TAG' is not a release tag of the form vX.Y.Z[-designation]"
+            exit 1
+          }
+          python scripts/verify_release_tag.py "$TAG" dist/*.whl
+
       - name: Upload package artifacts
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
@@ -276,15 +290,14 @@ jobs:
     name: Publish package to Test PyPI
     runs-on: ubuntu-latest
     needs: [smoke-test, cli-smoke-test, verify-dependencies, verify-sdist]
-    # Publishing always requires a v* tag ref: a tag push publishes to Test PyPI
-    # automatically, and a manual dispatch does the same when the chosen ref is a v* tag.
-    # Branch dispatches still run the build, verify, and smoke-test jobs (a useful dry run)
-    # but the publish job is skipped. Production PyPI publishing is intentionally absent
-    # during the v4.6.x preview; it arrives with the v4.7.0 feature branch.
-    # startsWith() only routes to this job (workflow `if:` expressions cannot regex-match);
-    # the exact tag format (v<release.yaml version>) is enforced below by the "Enforce
-    # release tag format" step and scripts/verify_release_tag.py before any upload.
-    if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
+    # Test PyPI remains an opt-in rehearsal channel: only a manual dispatch from a v* tag publishes
+    # here, so the publish path can be exercised against a real index without touching production.
+    # A branch dispatch still runs the build, verify, and smoke-test jobs as a dry run, with both
+    # publish jobs skipped.
+    # startsWith() is only a coarse route to this job; workflow if: expressions cannot regex-match.
+    # The tag format and the tag-to-wheel version match are enforced in the build job, which fails
+    # the whole run before anything is uploaded.
+    if: github.event_name == 'workflow_dispatch' && startsWith(github.ref, 'refs/tags/v')
     environment:
       name: testpypi
       url: https://test.pypi.org/p/netbox
@@ -293,36 +306,45 @@ jobs:
       id-token: write
 
     steps:
-      - name: Enforce release tag format
-        env:
-          TAG: ${{ github.ref_name }}
-        run: |
-          [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?$ ]] || {
-            echo "Ref '$TAG' is not a release tag of the form vX.Y.Z[-designation]"
-            exit 1
-          }
-
-      - name: Check out repository
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - name: Download package artifacts
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
         with:
-          persist-credentials: false
-      - name: Set up Python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+          name: python-package-distributions
+          path: dist/
+
+      - name: Publish package distributions to Test PyPI
+        uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
         with:
-          python-version: '3.12'
-      - name: Install tooling
-        run: python -m pip install --upgrade pip packaging
+          repository-url: https://test.pypi.org/legacy/
+          print-hash: true
 
+  publish-pypi:
+    name: Publish package to PyPI
+    runs-on: ubuntu-latest
+    needs: [smoke-test, cli-smoke-test, verify-dependencies, verify-sdist]
+    # A v* tag push is the production path. Test PyPI is an opt-in rehearsal rather than a promotion
+    # stage, so it is deliberately absent from this job's needs: an outage, a duplicate filename, or
+    # a misconfiguration on a test service must not block a verified production release. The four
+    # verification jobs above already ran against these exact artifacts. The protected pypi
+    # environment supplies the deliberate approval step, and because accepted PyPI filenames cannot
+    # be replaced or reused, a filename the index already holds fails the job instead of being
+    # skipped.
+    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+    environment:
+      name: pypi
+      url: https://pypi.org/p/netbox
+    permissions:
+      contents: read
+      id-token: write
+
+    steps:
       - name: Download package artifacts
         uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
         with:
           name: python-package-distributions
           path: dist/
 
-      - name: Verify the git tag matches the built version
-        run: python scripts/verify_release_tag.py "${{ github.ref_name }}" dist/*.whl
-
-      - name: Publish package distributions to Test PyPI
+      - name: Publish package distributions to PyPI
         uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
         with:
-          repository-url: https://test.pypi.org/legacy/
+          print-hash: true

+ 9 - 7
docs/development/building-the-package.md

@@ -1,8 +1,10 @@
 # Building the Package
 
-NetBox package artifacts (a wheel and a source distribution) can be built and verified locally. During the v4.6.x preview period, published artifacts are for maintainer validation only. Installing NetBox via pip is not a supported installation path yet. Experimental support for installing from production PyPI is planned for NetBox v4.7.0. This page is intended for maintainers and contributors working on the packaging itself; routine development does not require building a package.
+NetBox package artifacts (a wheel and a source distribution) can be built and verified locally. Installing NetBox from the Python package is experimental in NetBox v4.7 and is not recommended for production use. This page is intended for maintainers and contributors working on the packaging itself. Routine development does not require building a package.
 
-The artifacts are always built by CI from a clean checkout (see `.github/workflows/release.yml`). A local build is useful for testing packaging changes before they are merged.
+Published artifacts are always built by CI from a clean checkout (see `.github/workflows/release.yml`). A local build is useful for testing packaging changes before they are merged.
+
+Release tags trigger the production PyPI publishing workflow. Before a release tag is pushed, confirm that the `pypi` GitHub Actions environment has required reviewers configured so the upload waits for approval after the package checks complete. Referencing the environment in the workflow does not create an approval gate by itself. See [Confirm Package Publishing Prerequisites](./release-checklist.md#confirm-package-publishing-prerequisites) and [Publish to PyPI](./release-checklist.md#publish-to-pypi) for the required repository checks and release procedure.
 
 ## Prerequisites
 
@@ -12,7 +14,7 @@ Install the minimum local build tooling (all three are also included in the `dev
 python -m pip install --upgrade build packaging twine
 ```
 
-Building also requires a freshly rendered copy of the documentation site (see [Building](#building) below). The documentation toolchain (`zensical`, `mkdocs`, `mkdocs-material`, `mkdocstrings`) is pinned in `requirements.txt` rather than the `dev` group because it is also needed outside packaging, such as documentation previews and CI's `docs` job.
+Building also requires a freshly rendered copy of the documentation site (see [Building](#building) below). The documentation toolchain, including `zensical`, `mkdocs`, `mkdocs-material`, `mkdocstrings`, and `mkdocstrings-python`, is pinned in `requirements.txt` rather than the `dev` group because it is also needed outside packaging, such as documentation previews and CI's `docs` job.
 
 ## Building
 
@@ -41,7 +43,7 @@ The package version and the wheel's runtime dependency metadata are both compute
 
 ## Clean-tree caveat
 
-Always build release artifacts from a clean checkout. The build configuration keeps deployment-local files out of the artifacts: the Hatch excludes drop every `configuration*.py` and `ldap_config*.py` except the two tracked configuration templates (`configuration_example.py` and `configuration_testing.py`, which are force-included explicitly), and CI verifies the contents of both the wheel and the sdist before anything is published.
+Always build release artifacts from a clean checkout. The Hatch configuration excludes `netbox/netbox/configuration*.py` and `netbox/netbox/ldap_config*.py`, then force-includes only the two tracked configuration templates, `configuration_example.py` and `configuration_testing.py`. The sdist additionally excludes the checkout-level `netbox/configuration.py` and `netbox/ldap_config.py` symlinks. CI verifies the complete contents of both distributions before anything is published.
 
 These checks are defense in depth, not a license to build from a dirty tree: other untracked files under `netbox/` can still be picked up by a local build. CI builds from a clean checkout, so the published artifacts are unaffected. For a comparable local build, use a fresh `git clone` or a separate clean worktree rather than your day-to-day development tree.
 
@@ -87,11 +89,11 @@ NETBOX_SMOKETEST_BASE=/tmp/netbox-build-test-root \
 /tmp/netbox-build-test/bin/netbox check
 ```
 
-Without configuration, a wheel-installed NetBox looks for `$NETBOX_ROOT/conf/configuration.py` (default `/opt/netbox/conf/configuration.py`), which normally does not exist on a development workstation. The environment variables above point `netbox check` at the same minimal configuration module used by the release workflow's smoke-test job (`scripts/smoketest_configuration.py`); run the command from the repository root so `PYTHONPATH` can find it. `NETBOX_SMOKETEST_BASE` sets the writable scratch directory under which the module creates its media, reports, and scripts roots; `NETBOX_ROOT` points the fixed collected-static root at the same directory. Any other importable configuration module works the same way via `NETBOX_CONFIGURATION` (and `PYTHONPATH`, if the configuration lives outside the package). To exercise the full post-install task sequence from the wheel, run `netbox upgrade --no-input` with the same environment against a throwaway database (the collected static files land under `$NETBOX_ROOT/static`); this is what the release workflow's smoke-test job does. The documentation ships pre-rendered in the wheel, so there is nothing to build on the instance; `--build-docs` remains a checkout-only convenience for rendering the documentation from its sources.
+Without configuration, a wheel-installed NetBox looks for `$NETBOX_ROOT/conf/configuration.py` (default `/opt/netbox/conf/configuration.py`), which normally does not exist on a development workstation. The environment variables above point `netbox check` at the same minimal configuration module used by the release workflow's smoke-test job (`scripts/smoketest_configuration.py`); run the command from the repository root so `PYTHONPATH` can find it. `NETBOX_SMOKETEST_BASE` sets the writable scratch directory under which the module creates its media, reports, and scripts roots. `NETBOX_ROOT` sets the instance root, from which the fixed collected-static path `$NETBOX_ROOT/static` is derived. Any other importable configuration module works the same way via `NETBOX_CONFIGURATION` (and `PYTHONPATH`, if the configuration lives outside the package). To exercise the full post-install task sequence from the wheel, run `netbox upgrade --no-input` with the same environment against a throwaway database (the collected static files land under `$NETBOX_ROOT/static`); this is what the release workflow's smoke-test job does. The documentation ships pre-rendered in the wheel, so there is nothing to build on the instance; `--build-docs` remains a checkout-only convenience for rendering the documentation from its sources.
 
 ## Packaging architecture
 
-This section is a developer-facing overview of how the package is assembled and how a pip-installed NetBox behaves at runtime. User-facing installation documentation for the pip install path will be added alongside experimental PyPI support (planned for NetBox v4.7.0); this page does not cover end-user installation steps.
+This section is a developer-facing overview of how the package is assembled and how a pip-installed NetBox behaves at runtime. End-user installation steps live in [Install NetBox from the Python Package](../installation/3b-python-package.md).
 
 ### Dynamic metadata
 
@@ -99,7 +101,7 @@ This section is a developer-facing overview of how the package is assembled and
 
 ### sdist and the sdist-to-wheel guard
 
-`python -m build` produces both an sdist and a wheel, with the wheel built from the sdist. The release workflow's `verify-sdist` job rebuilds a wheel from the candidate sdist and runs `scripts/verify_wheel_metadata.py` and `scripts/verify_wheel_contents.py` against it, so a missing build input (for example the metadata hook or `base_requirements.txt`) cannot regress unnoticed. The rendered documentation site is one such build input: it reaches the sdist through its own force-include (`[tool.hatch.build.targets.sdist.force-include]`), so this guard also fails if that force-include is removed or broken.
+`python -m build` produces both an sdist and a wheel, with the wheel built from the sdist. The release workflow's `verify-sdist` job rebuilds a wheel from the candidate sdist and runs `scripts/verify_wheel_metadata.py` and `scripts/verify_wheel_contents.py` against it, so a missing build input, for example the metadata hook, `netbox/release.yaml`, or `requirements.txt`, cannot regress unnoticed. The rendered documentation site is one such build input: it reaches the sdist through its own force-include (`[tool.hatch.build.targets.sdist.force-include]`), so this guard also fails if that force-include is removed or broken.
 
 ### Wheel data layout
 

+ 51 - 10
docs/development/release-checklist.md

@@ -196,6 +196,16 @@ Once CI has completed and a colleague has reviewed the PR, merge it. This effect
 !!! warning
     To ensure a streamlined review process, the pull request for a release **must** be limited to the changes outlined in this document. A release PR must never include functional changes to the application: Any unrelated "cleanup" needs to be captured in a separate PR prior to the release being shipped.
 
+### Confirm Package Publishing Prerequisites
+
+Complete these checks before creating the release tag.
+
+Confirm that the existing PyPI trusted publisher still matches this repository, `.github/workflows/release.yml`, and the `pypi` environment name. If a Test PyPI rehearsal is planned, confirm the corresponding Test PyPI trusted publisher and `testpypi` environment as well. The trusted publisher's environment name must match the publish job's `environment.name`, otherwise the index rejects the upload before any file is transferred.
+
+Confirm that the `pypi` GitHub Actions environment has required reviewers configured so the production upload waits for approval after the package checks complete. Enable **Prevent self-review**, restrict deployments to `v*` tags, and leave administrator bypass disabled unless the maintainers deliberately require it. Referencing an environment from the workflow does not configure these protection rules; if the environment does not exist, GitHub creates it without an approval gate. The `testpypi` environment does not need an approval gate because a rehearsal run is dispatched deliberately.
+
+The published package version is derived from `netbox/release.yaml` (the `version` field plus any `designation`, e.g. `beta1` becomes `4.7.0b1`), not from the git tag. Confirm that the intended tag and `netbox/release.yaml` agree before creating the release. The publishing workflow verifies the match again against the built wheel.
+
 ### Create a New Release
 
 Create a [new release](https://github.com/netbox-community/netbox/releases/new) on GitHub with the following parameters.
@@ -207,23 +217,54 @@ Create a [new release](https://github.com/netbox-community/netbox/releases/new)
 
 Once created, the release will become available for users to install from GitHub.
 
-### Publish to Test PyPI
+### Publish to PyPI
+
+Creating the GitHub release pushes the new tag and starts the Python package publishing workflow. With the prerequisites above in place, the workflow builds and verifies the wheel and source distribution, then holds the production upload until the `pypi` deployment is approved. Approving the deployment publishes the verified artifacts to **PyPI**. Installing NetBox from the Python package is experimental in NetBox v4.7 and is not recommended for production use.
+
+A manual `workflow_dispatch` run from a `v*` release tag publishes to **Test PyPI** instead. This remains available as an optional rehearsal after packaging or publishing changes, but it is not required for every production release. Dispatching from a branch runs the build and verification jobs as a dry run without publishing anywhere.
+
+Dispatch a rehearsal from the release tag with GitHub CLI:
+
+```no-highlight
+gh workflow run release.yml --ref vX.Y.Z
+```
 
-Pushing a release tag triggers the Python package publishing workflow, which publishes the tagged release automatically to **Test PyPI** for maintainer validation. Installing NetBox via pip is not a supported installation path during the v4.6.x preview period; production PyPI publishing is planned for the v4.7.0 feature branch. A manual `workflow_dispatch` run publishes to Test PyPI only when the selected ref is a `v*` release tag; dispatching from a branch runs the build and verification jobs as a dry run without publishing.
+When a Test PyPI rehearsal is useful for a release, keep the production deployment awaiting approval while you dispatch the workflow from the same tag and validate the rehearsal. The rehearsal is a separate workflow run and rebuilds the distributions, so it validates the packaging and publishing path rather than the exact files waiting for production. Approve the production deployment after the rehearsal completes.
+
+Test PyPI enforces the same filename immutability. Once it has accepted either distribution generated for a release tag, dispatching that tag again is expected to fail because the workflow rebuilds the same wheel and source distribution filenames. A further rehearsal requires a new package version and matching tag.
+
+Official pre-release tags, including beta and release-candidate versions, are published to PyPI as well. This is intentional. Pip does not select pre-release versions by default unless the user explicitly requests one or no compatible stable release is available.
 
 After a publish run completes:
 
 * Verify that the build, CLI smoke-test (`cli-smoke-test`), smoke-test, dependency-verification (`verify-dependencies`), and sdist-verification (`verify-sdist`) jobs succeeded. The dependency-verification job fails the release if `requirements.txt` has drifted from `base_requirements.txt` or if the built wheel's `Requires-Dist` does not match `requirements.txt`; the sdist-verification job fails it if the sdist ships unexpected configuration files or cannot rebuild a valid wheel.
-* Verify that the publish job used the expected trusted-publishing environment (`testpypi`).
-* Confirm that the new version is visible on Test PyPI.
-* Install the published wheel into a fresh virtual environment and run `netbox check` against a minimal configuration module. The preview artifact is published to Test PyPI while NetBox's pinned runtime dependencies are expected to resolve from PyPI; to avoid mixed-index dependency resolution during validation, install the pinned dependencies from PyPI first, then install the Test PyPI artifact without resolving dependencies again:
+* Verify that the publish job used the expected trusted-publishing environment: `pypi` for a production release or `testpypi` for a rehearsal.
+* Confirm that the new version is visible on the corresponding package index.
+* Test the published wheel using the [wheel smoke-test procedure](./building-the-package.md#test-installing-the-wheel). For a production release, replace the local wheel installation command in that procedure with:
+
+    ```no-highlight
+    /tmp/netbox-build-test/bin/python -m pip install "netbox==<version>"
+    ```
+
+    For a Test PyPI rehearsal, install NetBox's pinned runtime dependencies from PyPI first and then install the candidate without resolving dependencies from the test index:
 
     ```no-highlight
-    pip install -r requirements.txt
-    pip install --no-deps --index-url https://test.pypi.org/simple/ netbox==<version>
+    /tmp/netbox-build-test/bin/python -m pip install -r requirements.txt
+    /tmp/netbox-build-test/bin/python -m pip install \
+        --no-deps \
+        --index-url https://test.pypi.org/simple/ \
+        "netbox==<version>"
     ```
 
-!!! note "Trusted publishing prerequisites"
-    Publishing requires a one-time setup by the project owners: a `netbox` project and a configured GitHub trusted publisher on Test PyPI, plus the corresponding `testpypi` GitHub Actions environment.
+    Run `netbox check` with the configuration and environment variables shown in the linked procedure.
+
+!!! warning "Production PyPI uploads are final"
+    Distribution files uploaded to PyPI cannot be replaced. A release may be yanked, and a release or an individual file may be deleted, but an uploaded filename can never be reused. Correcting an accepted distribution file requires publishing a new NetBox version.
+
+    If the publish job fails, check PyPI and the job log to determine whether any distribution file was accepted before deciding how to recover.
+
+    If no file was accepted and the cause can be corrected without changing the built distributions, correct it and re-run only the failed `publish-pypi` job. That reuses the package artifacts already built and verified in the original workflow run. Do not use **Re-run all jobs**, because it rebuilds the distributions.
+
+    If correcting the failure requires changing package contents or metadata, prepare a new NetBox version and release tag instead.
 
-The published package version is derived from `netbox/release.yaml` (the `version` field plus any `designation`, e.g. `beta1` becomes `4.7.0b1`), not from the git tag. Ensure the tag and `release.yaml` agree before tagging a pre-release.
+    If PyPI accepted either distribution file, do not retry the publish job. Production publishing fails on duplicate filenames by design, so the retry fails when it reaches the already accepted file. Yank the incomplete release, record the accepted filenames and hashes, and publish a new NetBox version rather than combining files from separate builds.