json + matrix + output + jq + toJSON

[bukowa]

Hello! I don't want to go into details right now but will do if nessecary.

Literally I spent several hours trying to pass jq output into github action matrix via toJSON function.

I think this documentation is not enough to understand how should i pass the data.

{\"include\":[{\"project\":\"foo\",\"config\":\"Debug\"},{\"project\":\"bar\",\"config\":\"Release\"}]}"

Documentation says nothing about format and other requirements.
How do i make it work (best practice) while the output is randomly generated in other step and passed into jq filter ?
Could we update documentation with working example of running job with matrix from output filtered by jq?

[bukowa]

Works for me:

# i used this  to set output
run: |
  set -eux
  DATA=$(echo ${{ steps.changed-files-specific.outputs.all_changed_files }} | jq -c '. | map(split("/")[0]) | unique')
  echo "matrix=$DATA" >> "$GITHUB_OUTPUT"

# and this to load
strategy:
  matrix:
    charts: ${{ fromJSON(needs.changed-files.outputs.matrix) }}

[anapsix]

I got tired of fighting with escaping quotes in bash, and ended up using shell: python

worflow example

name: "Workflow with matrix from output"

on:
  workflow_dispatch:
    inputs:
      widgets:
        type: choice
        description: "Selected widget."
        required: true
        default: "all"
        options:
        - "all"
        - "one"
        - "two"
        - "three"

env:
  INPUT_WIDGETS: ${{ inputs.widgets || 'all' }}

jobs:
  prepare:
    name: Prepare output
    runs-on: ubuntu-latest
    outputs:
      widgets: ${{ steps.compute_matrix.outputs.widgets }}
    steps:
    -
      name: Checkout
      uses: actions/checkout@v4
      with:
        sparse-checkout: |
          .github/workflows/test.yml
    -
      name: Compute matrix
      id: compute_matrix
      shell: python
      run: |
        #!/usr/bin/env python3

        import os
        import yaml
        import json

        INPUT_WIDGETS = os.getenv('INPUT_WIDGETS')
        GITHUB_OUTPUT = os.getenv('GITHUB_OUTPUT')
        GITHUB_WORKSPACE = os.getenv('GITHUB_WORKSPACE')
        WORKFLOW_FILE = f'{GITHUB_WORKSPACE}/.github/workflows/test.yml'

        class CustomLoader(yaml.SafeLoader):
            pass

        def str_constructor(loader, node):
            return loader.construct_scalar(node)

        CustomLoader.add_constructor(
            'tag:yaml.org,2002:bool',
            str_constructor
        )

        print('::group::Getting widgets value')
        if INPUT_WIDGETS == 'all':
            try:
                with open(WORKFLOW_FILE, 'r') as file:
                    data = yaml.load(file, Loader=CustomLoader)
                    input_widgets = data['on']['workflow_dispatch']['inputs']['widgets']['options']
            except Exception as e:
                print(f'::error::Failed to read or parse the workflow file: {e}')
                exit(1)

            input_widgets.remove('all')
            widgets = json.dumps(input_widgets)

        else:
            widgets = json.dumps([INPUT_WIDGETS])

        print('::endgroup::')
        print('::group::Recording widgets output')
        try:
            with open(GITHUB_OUTPUT, 'w') as file:
                file.write(f'widgets={widgets}')
        except Exception as e:
            print(f'::error::Failed to write to the GitHub output file: {e}')
            exit(1)

        print('::endgroup::')
        print('::group::Checking GitHub output file content')
        try:
            with open(GITHUB_OUTPUT, 'r') as file:
                print(file.read())
        except Exception as e:
            print(f'::error::Failed to read the GitHub output file: {e}')
            exit(1)

        print('::endgroup::')
  execute:
    name: Execute matrix
    needs: prepare
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 2
      matrix:
        widget: ${{ fromJSON(needs.prepare.outputs.widgets) }}
    steps:
    -
      name: Build widgets
      run: |
        echo "building for ${{ matrix.widget }}"