Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate building wheels for common build dependencies #130709

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

cdce8p
Copy link
Member

@cdce8p cdce8p commented Nov 15, 2024

Proposed change

Followup to

By building wheels for common build dependencies, we can speed up the normal wheel build significantly. Without it, pip would create those wheel when installing the isolated build environments for dependencies, however those aren't cached / uploaded to wheels.home-assistant.io. Thus the wheel build would be run for each new dependency.

For full details, see description on home-assistant/wheels#807.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@cdce8p cdce8p requested a review from a team as a code owner November 15, 2024 16:54
@home-assistant home-assistant bot added cla-signed code-quality small-pr PRs with less than 30 lines. labels Nov 15, 2024
@cdce8p cdce8p force-pushed the build-common-wheels branch 2 times, most recently from 3cba1ef to a19a2ab Compare November 15, 2024 17:00
@cdce8p cdce8p requested a review from frenck November 15, 2024 17:01
Copy link
Member

@frenck frenck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means we start building wheels for packages we do not use?!?

That makes no sense to me. We will end up wasting more time as we skip releases, besides, we will put up wheels we do not use...

../Frenck

@home-assistant home-assistant bot marked this pull request as draft November 15, 2024 20:02
@home-assistant
Copy link

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@cdce8p
Copy link
Member Author

cdce8p commented Nov 15, 2024

This means we start building wheels for packages we do not use?!?

They are used:

  • When building wheels and any one of them is a requirement (or part of the build system)
  • For any PR on the wheel builder repo
  • When building new docker images

We will end up wasting more time as we skip releases

The workflow itself takes roughly ~10min (for all jobs) when no new wheels need to be build. It's run once per day to account for PRs to the requirements file (i.e. if new once are added there). This can be remove though if we trigger a workflow_dispatch event afterwards.
Aside from that, it's explicitly only run on dependabot PRs when a new version is released. Thus we don't waste resources there.

@cdce8p
Copy link
Member Author

cdce8p commented Nov 27, 2024

This change would have also helped identify an issue with a recent ninja release even earlier. See https://github.com/home-assistant/core/actions/runs/12006896617/attempts/1

If it wasn't fixed upstream in time, it would have come up when we were trying to update numpy, pandas and possibly others as well.

--
Ref scikit-build/ninja-python-distributions#274

@cdce8p cdce8p marked this pull request as ready for review November 27, 2024 18:46
@home-assistant home-assistant bot requested a review from frenck November 27, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed code-quality small-pr PRs with less than 30 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants