Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scripts/ci: When building a package, add corresponding staging repo #1197

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

scripts/ci: When building a package, add corresponding staging repo #1197

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions scripts/.pop-os.asc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----

mQINBFlL+3MBEADdNM9Xy2t3EtKU1i3R1o1OCgJqLiDm8OZZq47InYID8oAPKRjd
0UDVJTrvfsB4oJH97VRi2hGv2xmc19OaFE/NsQBZW/7/3ypLr8eyaNgvscsmG/WN
cM1cbMZtwd1b0JOr3bNTzp6WKRI3jo9uRw7duM8FwPjKm76LboDQbAR+4Szm3O8x
/om8Gs1MRPUkY2dVz5KzednFLHwy7qnUXR3WRB5K1L9EBZkFDDNqnyViUIrE4bTm
BC9mTg/Xfw/QXUFYz3t/YTYduAU0o1q2yei+8tVAJKh7H9t3PrQ95l3RUUcaAvba
A9zlCrI8fonpxu7eSpkqzT4uCkfxdLVwittl1DumKTEkSXDQ5txY21igbSZZQwBA
Zf9MnFhJfPsEIq2YHRc1FBcQxiAIpnGizv7FgYY5FxmZQ7592dMQOZ00h+lDSQug
NMxloHCogaXR038uIKGTQnQEVcT46FtTRkLMSvbigy+RVSchdu9MEBBPgD3vSv53
NEobXsLiZ9hF6Hk7XI2WxP5j1zWTPmzxvf9NDOWz2Sw9Z+ilf252LXoxZQaMngp8
XL32uvw7q+mjB6F1W/qpe3b32uu7eGNrDWJ5veE808hpXXj803TllmRUfMGUrtY9
k7uUTQQWtrJ5uZ0QmsTk1oJHCPIUjjuiNtQfq28+bfg8FEJ/F1N1mB0IvwARAQAB
tCxQb3AgT1MgKElTTyBTaWduaW5nIEtleSkgPGluZm9Ac3lzdGVtNzYuY29tPokC
NwQTAQIAIgUCWUv7cwIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQIE3Y
rsM6ev8kXw/4p/8fOH8wM59ZoU0t1+fv3O8dYaDdTVTVIEno9snrsx5A5tbMu59r
HoBaxGenv/PB0l8yANhRX+HVmU/l0Sj0lTlEkYzgH/IT2Ne60s1ETgI7DlgSuYyP
H8wq6185+2DyE2+R/XcXGq0I++QUq1Y6rS+B4KIyYcgpJotcVNFaZiwuJZE31uLg
kVMZrm1oObHear7P2JQTbgsENMZDJEhQBCGKVdnAfVdKUeUrd07syr0cDe3kwY9o
cNc00bhIh23cLTJE2omok9yCsXoeFJlPMyZw8WvEa5oaYWzP4Yw7nF8/27JTzZ70
DjK2D2xoTkr0cP87LtZulS6FC3lxLuZ6hSaxsqoBH8Dd1uyYVEzLDsIRMtSHsXk+
3kLrr1p7/7/vjGShlYkbLtP4jWnlHc6vSxIzm/MQmQMCfjeo3QH7GGw88mYtXngQ
/Zna6wz0oL6pGM/4t90SCxTxRqCnoxMxzkcpt9n42bj79grESOMH4wm3ExfuPk7I
DtY+SqzIq0QvoPbC3XJLusWVgwUsRF2FpTTRTHEiWEMjWDKDVEyT4K1k1k3f/gi2
6LdtXwqDwzUvJJU5HYwVFywt+0jt5F0ZlTlPizz3iHw4gMLOielRShl+gZrU2U0O
aj1Hyts9LymEKMUvRQGwMqCZcXo6sGjs59tTsfeGX16PTOyBri8eoLkCDQRZS/tz
ARAApD9IWm4zS1AuBcOTuvh1E/ciKHGIUtW3JftD9ah8loEeckakgG5Xn9he1X6J
yxPULpsptcCCcKXlw853ZQK9PLQJX6moWLH+qf2Zo3UAn/YEsWk+KsHoxPXHNUds
u/j6UKkqEk8c7H92hUo8aWghO3p4HDVJ9KmGtueQ3jOv8Qun7Eh9cIo0A59cKmMv
jKUiYHLIJw8bkveQ8rVPul1ZHn56ORiBi58vm3tzjI4UWHQMjiKxXT6H5eG/f5K6
uaK8lljh6n6jhdnQCpBcdtSIbhE/6YRv2+IgL+BRssvprBtx4/sBwKjNNqzWPeGy
UDHMiF88ETYqZ8DfukQ/e5XuaxjU41g/F8cw8BeVTBMveb1YTyOoWcWvTL+hoBfS
qYc/lvDHmmJ7/IgeMvUE6KoByP4ub5wX52mJTqgMC4GMhA04BC60B+NfVAXLh2pa
TRJAHoWTDswOxbR6q9zPEFGZzV04B9Y96EavwMpT5IzG2fOPBwvdT0EDnt+vQ/iB
c9O7CvkRTROAV+RoNCLY2XU8yNc/XxuI66PCE4Q96jW4uDzHvi6sPW/glsfRi2NT
RWCO15KMVf0aypXeBpSbHIXIYGdXRQRpw980IW6PrElPpqZ5/DGbkXei5CuruF2R
mltuu3MqYQjcUvP9T7s0e5GAFgQFrR/8q29nVULq8IF4vzUAEQEAAYkCHwQYAQIA
CQUCWUv7cwIbDAAKCRAgTdiuwzp6/wTGD/9Co4gEmTTOW++FneMMJo5K4WqeWVRg
g1q5+yoVqgWq3k6lLsEC5kxR305BAAcvXo9XPKdo62ySYmhIFOpIz/TkeTUxDZaw
sLtcBxXUME2L5j/1od1V9lxecUvLAgA11o5Kb8TMKn5ZcmGhadtTLslWQcYsKqhw
LaYQRlcxLDHYT8DXFkHgDhUMMbpt07dU5v5lIjgtGNHRhdS7/lCmSWOBtYapwpAH
GYSmahN0zO36VHzOB5uwFue0tSoQiBEvLrCV/8ZtT2S5NkXaSmisz6B5Vr6DRtWI
OamW5pMbSL8WQNQ99Kik05ctERjv2NgxI4JQo/a4KKthRrT4JlixXmrfJDuPyDPp
RuTu1Elo6snoqWKQNf1sEPKvcv7EviNxBOhbTKivWrJXMnbOme7+UlNLcq7VAFp3
x5hxk/ap0WqH/hs7+8jMBC8nS402MoM7EyLS0++kbOuEL/Prf3+JxFRqIu5Df77J
+bUmTtKICV43ikiVWmnP5OuJj2JPSOTR+rLxAQYpyHmo7HKXE63FbH1FVLgsT88+
EW6VtI01I7EYmKQXEqQo52yfeHKDrQjGNVBWMKcXj0SVU+QQ1Ue/4yLwA+74VD2d
fOyJI22NfTI+3SMAsMQ8L+WVQI+58bu7+iEqoEfHCXikE8BtTbJAN4Oob1lrjfOe
5utH/lMP9suRWw==
=NL3f
-----END PGP PUBLIC KEY BLOCK-----
15 changes: 13 additions & 2 deletions scripts/ci
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import shutil
from subprocess import check_call, check_output
import sys
from threading import Lock
from urllib.error import HTTPError
from urllib.request import urlopen

from lib import foreach_repo_parallel, github_post
from git import git_ids_and_branches, git_timestamp_id, git_datetime_id, git_archive_id
Expand Down Expand Up @@ -270,7 +272,7 @@ def dpkg_source(name, git, series):

return dsc_path, tar_path

def dpkg_binary(dsc_path, name, git, series, build_arch, build_all):
def dpkg_binary(dsc_path, name, git, series, build_arch, build_all, pockets):
with open(dsc_path, "r") as fp:
dsc = Dsc(fp)

Expand Down Expand Up @@ -354,9 +356,18 @@ def dpkg_binary(dsc_path, name, git, series, build_arch, build_all):
"--extra-repository=deb http://ppa.launchpad.net/" + ppa_proposed + "/ubuntu " + series.codename + " main",
"--extra-repository=deb-src http://ppa.launchpad.net/" + ppa_proposed + "/ubuntu " + series.codename + " main",
"--extra-repository-key=" + path.join(POP_DIR, "scripts", ppa_key),
"--extra-repository-key=" + path.join(POP_DIR, "scripts", ".pop-os.asc"),
"--no-apt-distupgrade",
]

for (repo, _) in pockets:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern here is what might be in pockets, and the fact that this potentially adds multiple additional repositories. If I understand, if the same commit is pushed to multiple branches, the CI will use the same build. I'm not sure if the dependencies it adds here might then cause possible problems...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All you really need is to add the branch the package is being built from, so if building for focal on branch new-feature, you'd add http://apt.pop-os.org/staging/new-release focal main.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that in the (unlikely) case that the same commit is pushed to two branches between CI runs, pockets will contain both branches, and the same build will be used for both. If another branch is later updated to a commit that already has a build for another branch (for instance, a fast-forward merge), the package will be copied without rebuilding.

That's why this requires a loop and potentially could add multiple branches. But I'm happy to be corrected.

try:
urlopen("http://apt.pop-os.org/staging/{}/dists/{}/Release".format(repo, series.codename))
except HTTPError:
pass
else:
sbuild.append("--extra-repository=deb http://apt.pop-os.org/staging/{} {} main".format(repo, series.codename))

if build_all:
sbuild.append("--arch-all")

Expand Down Expand Up @@ -404,7 +415,7 @@ def build_packages_thread(args):
if dsc_path and tar_path:
for build_arch in build_archs:
build_all = build_archs[build_arch]
deb_paths += dpkg_binary(dsc_path, name, git, series, build_arch, build_all)
deb_paths += dpkg_binary(dsc_path, name, git, series, build_arch, build_all, pockets)
return (name, git, series, pockets, dsc_path, tar_path, deb_paths)

def build_packages(name):
Expand Down