Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build dateilager within the dockerfile using the nix environment #86

Merged
merged 1 commit into from
May 8, 2024

Conversation

airhorns
Copy link
Contributor

@airhorns airhorns commented May 4, 2024

This switches the build process to:

  • build DL within the dockerfile for repeatable builds, instead of copying in from outside
  • use the nix environment to setup the env for the build, so it's the same go version / protoc version etc etc in and outside of docker
  • use a multistage docker build so we have slim containers with just the binary at the end

I did all this because I was fighting to get a docker image built for linux_arm64, which is what orbstack needs to run the image locally in a k8s setup! It was annoying. This also adds linux_arm64 as a build target to the makefile, and teaches the dockerfile how to build the binary for the target architecture that the dockerfile is building for.

Yeesh.

@airhorns airhorns force-pushed the harry/multi-stage-docker branch from 10b9a4d to 656a326 Compare May 5, 2024 01:51
@airhorns airhorns force-pushed the harry/multi-stage-docker branch from 656a326 to bc25ab3 Compare May 5, 2024 02:00
@airhorns airhorns changed the base branch from main to harry/prereleasing May 5, 2024 02:00
Base automatically changed from harry/prereleasing to main May 5, 2024 02:02
@airhorns airhorns marked this pull request as ready for review May 5, 2024 02:27
@airhorns airhorns force-pushed the harry/multi-stage-docker branch from bc25ab3 to 42a6646 Compare May 5, 2024 02:54
@airhorns airhorns mentioned this pull request May 5, 2024
@airhorns airhorns requested a review from angelini May 5, 2024 16:46
go install github.com/bojand/ghz/cmd/[email protected]
go install github.com/gadget-inc/fsdiff/cmd/[email protected]
go install github.com/stamblerre/gocode@latest
go install golang.org/x/tools/cmd/goimports@latest
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stuck these in the nix env, and then removed the ones that now come bundled with vscode (gocode => gopls, goimports is bundled also). This way, the versions used are the same in dev and in docker

@@ -20,8 +33,11 @@ RUN mkdir -p /home/main/secrets
VOLUME /home/main/secrets/tls
VOLUME /home/main/secrets/paseto

COPY release/server_linux_amd64 server
COPY --from=build-stage /app/release/server_linux_${TARGETARCH} server
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the nice multistage thing so all the build deps aren't in the final image

protoc-gen-go-grpc
go-migrate
mkcert
];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the devShell build all the stuff you need to work on DL into your shell, but doesn't build DL itself. I always found that annoying with the original setup that I think Kira created that cd-ing into the dir built the whole project, which doesn't work so good when you are working on it and it might be busted.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm down to properly Nix convert the project, it was in a very half converted state for a while.

But I do really miss the ability to control package versions, if ever Nix updates a package before we want to we're SOL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible in nix, it's just annoying! You add a new dep at the top of a file for a nixpkgs sha that has the version of the thing you actually want, and then you refer to that version of nixpkgs in your exported packages! We did it in gadget a couple times when we didn't want to upgrade things.

Dockerfile Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
.dockerignore Outdated Show resolved Hide resolved
This switches the build process to:
 - build DL within the dockerfile for repeatable builds, instead of copying in from outside
 - use the nix environment to setup the env for the build, so it's the same go version / protoc version etc etc in and outside of docker
 - use a multistage docker build so we have slim containers with just the binary at the end

I did all this because I was fighting to get a docker image built for linux_arm64, which is what orbstack needs to run the image locally in a k8s setup! It was annoying. This also adds linux_arm64 as a build target to the makefile, and teaches the dockerfile how to build the binary for the target architecture that the dockerfile is building for.

Yeesh.
@airhorns airhorns force-pushed the harry/multi-stage-docker branch from da16b06 to f0fc487 Compare May 6, 2024 13:44
@airhorns airhorns requested a review from angelini May 6, 2024 13:46
@angelini angelini merged commit 070991a into main May 8, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants