Skip to content

Commit

Permalink
Merge branch 'develop' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
clemlesne committed Jul 17, 2023
2 parents 6704d8b + 8391f34 commit b9e217e
Show file tree
Hide file tree
Showing 4 changed files with 31 additions and 28 deletions.
42 changes: 21 additions & 21 deletions .github/workflows/pipeline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ env:
CONTAINER_REGISTRY_GHCR: ghcr.io
CONTAINER_REGISTRY_DOCKER_HUB: docker.io
# https://npmjs.com/package/@microsoft/sarif-multitool?activeTab=versions
SARIF_MULTITOOL_VERSION: 4.2.0
SARIF_MULTITOOL_VERSION: 4.2.2
# https://npmjs.com/package/snyk?activeTab=versions
SNYK_VERSION: 1.1182.0
SNYK_VERSION: 1.1187.0
# https://github.com/microsoft/azure-pipelines-agent/releases
AZP_AGENT_VERSION: 3.220.5
# https://github.com/PowerShell/PowerShell/releases
POWERSHELL_VERSION: 7.2.11
POWERSHELL_VERSION: 7.2.12
# https://github.com/krallin/tini/releases
TINI_VERSION: 0.19.0
# https://github.com/mikefarah/yq/releases
Expand All @@ -39,27 +39,27 @@ env:
# https://github.com/moby/buildkit/releases
BUILDKIT_VERSION: 0.11.6
# https://github.com/Azure/azure-cli/releases
AZURE_CLI_VERSION: 2.49.0
AZURE_CLI_VERSION: 2.50.0
# https://github.com/stedolan/jq/releases
JQ_VERSION: 1.6
# https://github.com/aws/aws-cli/releases
AWS_CLI_VERSION: 2.12.1
AWS_CLI_VERSION: 2.13.0
# https://cloud.google.com/sdk/docs/install
GCLOUD_CLI_VERSION: 435.0.1
GCLOUD_CLI_VERSION: 437.0.1
# https://github.com/git-for-windows/git/releases
GIT_VERSION_WIN: 2.41.0
# https://github.com/facebook/zstd/releases
ZSTD_VERSION_WIN: 1.5.5
# https://www.python.org/downloads/windows
PYTHON_VERSION_WIN: 3.11.4
# https://nodejs.org/en/download/releases
NODE_VERSION: 18.16.0
NODE_VERSION: 18.16.1
# https://github.com/helm/helm/releases
HELM_VERSION: 3.12.1
# https://github.com/oras-project/oras/releases
ORAS_VERSION: 1.0.0
# https://github.com/docker/buildx/releases
BUILDX_VERSION: 0.11.0
BUILDX_VERSION: 0.11.1
# https://github.com/hadolint/hadolint/releases
HADOLINT_VERSION: 2.12.0

Expand All @@ -77,7 +77,7 @@ jobs:
submodules: recursive

- name: SAST - Credentials
uses: trufflesecurity/trufflehog@v3.40.0
uses: trufflesecurity/trufflehog@v3.44.0
with:
base: ${{ github.event.repository.default_branch }}
head: HEAD
Expand Down Expand Up @@ -112,7 +112,7 @@ jobs:

# Required for running "npx" CLI
- name: Setup Node
uses: actions/setup-node@v3.6.0
uses: actions/setup-node@v3.7.0
with:
node-version: ${{ env.NODE_VERSION }}

Expand Down Expand Up @@ -168,7 +168,7 @@ jobs:
snyk.sarif
- name: Upload results to GitHub Security
uses: github/codeql-action/[email protected].0
uses: github/codeql-action/[email protected].4
with:
sarif_file: merged.sarif

Expand Down Expand Up @@ -216,7 +216,7 @@ jobs:

# Required for running "npx" CLI
- name: Setup Node
uses: actions/setup-node@v3.6.0
uses: actions/setup-node@v3.7.0
with:
node-version: ${{ env.NODE_VERSION }}

Expand All @@ -231,7 +231,7 @@ jobs:
make test
build-publish-linux:
name: Build & deploy Linux image "${{ matrix.os }}"
name: Build & deploy image (Linux ${{ matrix.os }})
needs:
- sast-creds
- sast-semgrep
Expand Down Expand Up @@ -278,15 +278,15 @@ jobs:
platforms: ${{ matrix.arch }}

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v2.7.0
uses: docker/setup-buildx-action@v2.9.1
with:
version: v${{ env.BUILDX_VERSION }}
driver-opts: |
image=moby/buildkit:v${{ env.BUILDKIT_VERSION }}
# Required for running "npx" CLI
- name: Setup Node
uses: actions/setup-node@v3.6.0
uses: actions/setup-node@v3.7.0
with:
node-version: ${{ env.NODE_VERSION }}

Expand Down Expand Up @@ -367,8 +367,8 @@ jobs:
type=gha
type=registry,ref=${{ env.CONTAINER_REGISTRY_GHCR }}/${{ env.CONTAINER_NAME }}
cache-to: |
type=gha,mode=max
type=registry,ref=${{ env.CONTAINER_REGISTRY_GHCR }}/${{ env.CONTAINER_NAME }},mode=max
type=gha
type=registry,ref=${{ env.CONTAINER_REGISTRY_GHCR }}/${{ env.CONTAINER_NAME }}
context: src/docker
file: src/docker/Dockerfile-${{ matrix.os }}
labels: ${{ steps.meta.outputs.labels }}
Expand Down Expand Up @@ -407,12 +407,12 @@ jobs:
snyk-*.sarif
- name: Upload results to GitHub Security
uses: github/codeql-action/[email protected].0
uses: github/codeql-action/[email protected].4
with:
sarif_file: merged.sarif

build-publish-win:
name: Build & deploy Windows image "${{ matrix.os }}"
name: Build & deploy image (Windows ${{ matrix.os }})
needs:
- sast-creds
- sast-semgrep
Expand Down Expand Up @@ -580,7 +580,7 @@ jobs:
${{ steps.tag.outputs.tag }}
- name: Upload results to GitHub Security
uses: github/codeql-action/[email protected].0
uses: github/codeql-action/[email protected].4
with:
sarif_file: snyk.sarif

Expand All @@ -601,7 +601,7 @@ jobs:
run: semgrep ci --sarif --output=semgrep.sarif

- name: Upload results to GitHub Security
uses: github/codeql-action/[email protected].0
uses: github/codeql-action/[email protected].4
with:
sarif_file: semgrep.sarif

Expand Down
File renamed without changes.
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

<!-- github.com badges -->

[![Project licence](https://img.shields.io/github/license/clemlesne/azure-pipelines-agent)](https://github.com/clemlesne/azure-pipelines-agent/blob/main/LICENCE)
[![Project license](https://img.shields.io/github/license/clemlesne/azure-pipelines-agent)](https://github.com/clemlesne/azure-pipelines-agent/blob/main/LICENSE)
[![Last release date](https://img.shields.io/github/release-date/clemlesne/azure-pipelines-agent)](https://github.com/clemlesne/azure-pipelines-agent/releases)
[![Workflow status](https://img.shields.io/github/actions/workflow/status/clemlesne/azure-pipelines-agent/pipeline.yaml?branch=main)](https://github.com/clemlesne/azure-pipelines-agent/actions/workflows/pipeline.yaml)
[![All releases download counter](https://img.shields.io/github/downloads/clemlesne/azure-pipelines-agent/total)](https://github.com/clemlesne/azure-pipelines-agent/pkgs/container/azure-pipelines-agent)
Expand Down Expand Up @@ -194,7 +194,7 @@ These methods can be used to build a container image, at the time of writing:
| [img](https://github.com/genuinetools/img#running-with-kubernetes), [BuildKit](https://github.com/moby/buildkit) | 🟩🟩🟩 | 🟩🟩🟥 | 🟩🟥🟥 | Local CLI | CLI to build the images. Can build different architectures on a single machine. Requires [Seccomp](https://en.wikipedia.org/wiki/Seccomp) disabled and [AppArmor](https://apparmor.net) disabled. |
| Docker in docker | 🟩🟩🟩 | 🟥🟥🟥 | 🟩🟩🟩 | Local CLI | Before Kubernetes 1.20, it was possible to build container images in the agent, using the Docker socket. This is not possible anymore, as Kubernetes [deprecated the Docker socket](https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker) in favor of the [Container Runtime Interface](https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes). |

We choose BuildKit for this project. [Its licence](https://raw.githubusercontent.com/moby/buildkit/v0.11.5/LICENSE) allows commercial use, and the project and mainly maintained, as the time of writing, by Docker, Netlix and Microsoft.
We choose BuildKit for this project. [Its license](https://raw.githubusercontent.com/moby/buildkit/v0.11.5/LICENSE) allows commercial use, and the project and mainly maintained, as the time of writing, by Docker, Netlix and Microsoft.

Linux systems are supported, but not Windows:

Expand Down Expand Up @@ -323,6 +323,7 @@ extraVolumeMounts:
- [ASP.NET Core Runtime](https://github.com/dotnet/aspnetcore)
- Python ([Python 3.8](https://www.python.org/downloads/release/python-380), [Python 3.9](https://www.python.org/downloads/release/python-390), [Python 3.10](https://www.python.org/downloads/release/python-3100), [Python 3.11](https://www.python.org/downloads/release/python-3110), depending of the system, plus C/Rust build tools for libs non pre-built on the platforms)
- Tools
- [git](https://github.com/git-for-windows/git)
- [gzip](https://www.gnu.org/software/gzip)
- [jq](https://github.com/stedolan/jq)
- [make](https://www.gnu.org/software/make)
Expand Down Expand Up @@ -431,4 +432,4 @@ This project is open source and maintained by people like you. If you need help

## [Code of conduct](./CODE_OF_CONDUCT.md)

## [Authors](./AUTHORS.md)
## [Authors](./AUTHORS.md)
10 changes: 6 additions & 4 deletions TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
- [Change Folder Buildkit](#Change-Folder-Buildkit)
- [Agent ran longer than the maximum time of 60 minutes](#Agent-ran-longer-than-the-maximum-time-of-60-minutes)

## Troubleshooting
## Troubleshooting

### Namespaces needs to be set to non-zero

### Namespaces needs to be set to non-zero
This error is due to the fact that BuildKit needs to create a new user namespace, and the default maximum number of namespaces is 0. Value is defined by `user.max_user_namespaces` ([documentation](https://man7.org/linux/man-pages/man7/namespaces.7.html)). You can fix it by setting the value to more than 1000. Issue notably happens on AWS Bottlerocket OS. [See related issue.](https://github.com/clemlesne/azure-pipelines-agent/issues/19)

We can update dynamically the host system settings with a DaemonSet:
Expand Down Expand Up @@ -43,15 +44,16 @@ spec:
privileged: true
```
### Change Folder Buildkit
### Change Folder Buildkit
If need Buildkit to write in another folder, then create the buildkitd.toml file and set the root variable. Example below (bash in the pipeline):
```bash
mkdir ~/.config/buildkit
echo 'root = "/app-root/.local/tmp/buildkit"' > ~/.config/buildkit/buildkitd.toml
```

### Agent ran longer than the maximum time of 60 minutes
### Agent ran longer than the maximum time of 60 minutes

If the pipeline takes longer than 60 minutes, you need to change two points. The first is the variable in the helm [pipelines.timeout](https://github.com/clemlesne/azure-pipelines-agent#Helm-values) to 7200 seconds (two hours) for example.

Expand Down

0 comments on commit b9e217e

Please sign in to comment.