Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BFD-2906: Automate Maven Release Process #2061

Conversation

malessi
Copy link
Contributor

@malessi malessi commented Nov 15, 2023

JIRA Ticket:
BFD-2906

User Story or Bug Summary:

As a BFD engineer I want to be able to have an automated release process that uses the maven release plugin to push build artifacts to CodeArtifact and ECR.


What Does This PR Do?

NOTE: This PR is being merged into an intermediate branch as additional changes are required before this process is usable

This PR:

  • Adds a new GitHub Actions workflow named Build Release that can be started manually (see workflow_dispatch).
    • This workflow executes the following steps:
      • First, this new workflow will execute the Maven Release plugin, which will:
        • Build all modules within the parent pom.xml
        • Upon successful build, update all pom.xmls to the version specified by the operator
        • Create a commit with these updated pom.xmls on the branch specified by the operator (master, in the typical case), and create a tag from this commit with the release version as the tag name
        • Update the pom.xmls again setting their versions to the development version specified by the operator (typically <releaseVersion + 1>-SNAPSHOT)
        • Create a commit with these SNAPSHOT pom.xmls on the branch specified (usually master)
        • The Docker images generated by Jib will then be uploaded to ECR
        • All artifacts generated by the Maven Release build will then be uploaded to CodeArtifact under the bfd-mgmt repository, versioned appropriately
      • Then, assuming a successful run of the Maven Release plugin, the workflow will then generate a new GitHub prerelease based upon the Git tag generated by the Maven Release plugin with all of the relevant data dictionary files uploaded as assets under the new Release
    • Note that all Git operations done by this Workflow use the bfd-release GitHub Application, which is configured to bypass branch protection rules on master
  • Updates the bfd-data-* modules to be included as part of the parent pom.xml so that they are versioned and built similarly to all other modules

This PR has been validated by:

  • Running the new Build Release workflow against a protected branch, verifying that the workflow runs successfully, all Git operations succeed (and protections are bypassed), all artifacts are uploaded to their respective repositories, and a corresponding GitHub Release is generated with the appropriate data dictionary files appended as assets
  • terraform planning the changes to mgmt, verifying that the plan does not include any unexpected changes and that all changes are valid

No changes to our infrastructure have been made as of this PR. I will apply mgmt when this PR is merged.

What Should Reviewers Watch For?

If you're reviewing this PR, please check for these things in particular:

  • Verify all PR security questions and checklists have been completed and addressed.

What Security Implications Does This PR Have?

Submitters should complete the following questionnaire:

  • If the answer to any of the questions below is Yes, then you must supply a link to the associated Security Impact Assessment (SIA), security checklist, or other similar document in Confluence here: N/A

    • Does this PR add any new software dependencies?
      • Yes
      • No
    • Does this PR modify or invalidate any of our security controls?
      • Yes
      • No
    • Does this PR store or transmit data that was not stored or transmitted before?
      • Yes
      • No
  • If the answer to any of the questions below is Yes, then please add @StewGoin as a reviewer, and note that this PR should not be merged unless/until he also approves it.

    • Do you think this PR requires additional review of its security implications for other reasons?
      • Yes
      • No

What Needs to Be Merged and Deployed Before this PR?

This PR cannot be either merged or deployed until the following prerequisite changes have been fully deployed:

  • Branch protection rules on master are converted to equivalent rules using Rulesets, with the bfd-release GitHub Application exempted from those rulesets

Submitter Checklist

I have gone through and verified that...:

  • I have named this PR and branch so they are automatically linked to the (most) relevant Jira issue. Ie: BFD-123: Adds foo
  • This PR is reasonably limited in scope, to help ensure that:
    1. It doesn't unnecessarily tie a bunch of disparate features, fixes, refactorings, etc. together.
    2. There isn't too much of a burden on reviewers.
    3. Any problems it causes have a small "blast radius".
    4. It'll be easier to rollback if that becomes necessary.
  • This PR includes any required documentation changes, including README updates and changelog / release notes entries.
  • The data dictionary has been updated with any field mapping changes, if any were made.
  • All new and modified code is appropriately commented, such that the what and why of its design would be reasonably clear to engineers, preferably ones unfamiliar with the project.
  • All tech debt and/or shortcomings introduced by this PR are detailed in TODO and/or FIXME comments, which include a JIRA ticket ID for any items that require urgent attention.
  • Reviews are requested from both:
    • At least two other engineers on this project, at least one of whom is a senior engineer or owns the relevant component(s) here.
    • Any relevant engineers on other projects (e.g. DC GEO, BB2, etc.).
  • Any deviations from the other policies in the DASG Engineering Standards are specifically called out in this PR, above.
    • Please review the standards every few months to ensure you're familiar with them.

@malessi malessi force-pushed the alessio/BFD-2906__maven-release-pipeline branch 11 times, most recently from ad74599 to 6aa57e1 Compare November 21, 2023 20:37
@malessi malessi changed the title Alessio/bfd 2906 maven release pipeline BFD-2906: Automate Maven Release Process Nov 21, 2023
@malessi malessi marked this pull request as ready for review November 21, 2023 20:59
Copy link
Contributor

@brandoncruz3 brandoncruz3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and verified in GHA. 🚀

Copy link

@StewGoin StewGoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussion with @mjburling today (I will allow him to summarize in a comment, etc...) I'm ok with this moving forward within the context provided by that forthcoming comment.

@lsmitchell
Copy link
Contributor

Is updating the documentation for our release process included in this story, or is it recorded somewhere?

@malessi
Copy link
Contributor Author

malessi commented Nov 27, 2023

Is updating the documentation for our release process included in this story, or is it recorded somewhere?

We could probably include that as part of the follow-up Story, BFD-2907. This Story is specifically for getting the Workflow created -- BFD-2907 is the part where we actually integrate this new Workflow into our release process.

Copy link
Contributor

@brianburton brianburton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work!

@malessi malessi marked this pull request as draft November 28, 2023 20:49
Comment on lines 175 to 181
CA_DATA_DICTIONARY_ASSETS: |
[
"bfd-server-war-${{ inputs.releaseVersion }}-v1-data-dictionary.csv",
"bfd-server-war-${{ inputs.releaseVersion }}-v2-data-dictionary.csv",
"bfd-server-war-${{ inputs.releaseVersion }}-v1-data-dictionary.json",
"bfd-server-war-${{ inputs.releaseVersion }}-v2-data-dictionary.json"
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is going to need some additional attention, post #2065

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this to include the new .xlsx Excel worksheet and to follow the updated naming conventions introduced in #2065. That should be enough, but let me know if I missed something.

@malessi malessi force-pushed the alessio/BFD-2906__maven-release-pipeline branch from 7a6ad9b to c497d17 Compare December 1, 2023 16:15
@malessi malessi force-pushed the alessio/BFD-2906__maven-release-pipeline branch from c497d17 to af9d3a5 Compare December 1, 2023 16:19
@mjburling mjburling merged this pull request into BFD-3078__release-automation-improvement Dec 1, 2023
17 checks passed
@mjburling mjburling deleted the alessio/BFD-2906__maven-release-pipeline branch December 1, 2023 21:31
mjburling added a commit that referenced this pull request Dec 13, 2023
Co-authored-by: Brian Burton <[email protected]>
Co-authored-by: Michael J Burling <[email protected]>
mjburling added a commit that referenced this pull request Dec 14, 2023
Co-authored-by: Brian Burton <[email protected]>
Co-authored-by: Michael J Burling <[email protected]>
mjburling added a commit that referenced this pull request Dec 18, 2023
Co-authored-by: Brian Burton <[email protected]>
Co-authored-by: Michael J Burling <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants