Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch data for all installers, not only pip #39

Merged
merged 1 commit into from
Nov 10, 2024
Merged

Fetch data for all installers, not only pip #39

merged 1 commit into from
Nov 10, 2024

Conversation

hugovk
Copy link
Owner

@hugovk hugovk commented Nov 10, 2024

Helps #36.

pypinfo --help
...
Options:
...
  --all                   Show downloads by all installers, not only pip.

pypinfo --all tells it to fetch data for all installers, and not to filter to only pip (the default).

(--all was added to pypinfo in ofek/pypinfo#51.)

This results in a cheaper BigQuery call: the pip-only query costs an extra 25% in bytes processed, in bytes billed, and in dollars; see #36 (comment).

installer_name download_count
pip 1,121,198,711
uv 117,194,833
requests 29,828,272
poetry 23,009,454
None 8,916,745
bandersnatch 6,171,555
setuptools 1,362,797
Bazel 1,280,271
Browser 1,096,328
Nexus 593,230
Homebrew 510,247
Artifactory 69,063
pdm 62,904
OS 13,108
devpi 9,530
conda 2,272
pex 194
Total 1,311,319,514

pip is still by far the most popular installer, and unsurprising uv is up there too, with about 10% of pip's downloads.

The others are 10% of uv or less. A lot of them are mirroring services, that we wanted to exclude before.

I think given uv's importance, and my expectation that it will continue to take a bigger share of the pie, plus especially the extra cost for filtering by just pip, means that we should switch to fetching data for all downloaders. Plus the others don't account for that much of the pie.

@hugovk hugovk merged commit 1727e06 into main Nov 10, 2024
3 checks passed
@hugovk hugovk deleted the all-installers branch November 10, 2024 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant