Skip to content

Commit

Permalink
Flatten directory dependencies for hash
Browse files Browse the repository at this point in the history
  • Loading branch information
colluca committed Sep 27, 2024
1 parent 47624bc commit c8d3381
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 5 deletions.
15 changes: 13 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
uses: ./
with:
working-directory: test
target: test
target: test1
- name: Check non-recursive prerequisites
run: |
test "${{ steps.non-recursive-prerequisites.outputs.prerequisites }}" = "prereq"
Expand All @@ -27,9 +27,20 @@ jobs:
uses: ./
with:
working-directory: test
target: test
target: test1
flags: --recursive
- name: Check recursive prerequisites
run: |
test "${{ steps.recursive-prerequisites.outputs.prerequisites }}" = "preprereq1 preprereq2"
echo ${{ steps.recursive-prerequisites.outputs.hash }}
- name: Hash directory prerequisite
id: hash-dir-prerequisite
uses: ./
with:
working-directory: test
target: test2
flags: --recursive
- name: Check directory prerequisite hash
run: |
test "${{ steps.hash-dir-prerequisite.outputs.prerequisites }}" = "dirprereq"
test "${{ steps.hash-dir-prerequisite.outputs.hash }}" = "c64a77871513eadd52ec0e278bfbac1db866b77f66ebe91725b378b8bf43e29c"
14 changes: 12 additions & 2 deletions list-make-prerequisites.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from anytree import Node, RenderTree
import argparse
import hashlib
from pathlib import Path
import re
import subprocess
import sys
Expand Down Expand Up @@ -137,9 +138,18 @@ def hash_files(file_list):
# Create a new SHA-256 hash object
hasher = hashlib.sha256()

# If any prerequisite is a directory, replace it with its (recursive) contents
expanded_file_list = []
for file_name in file_list:
# Open each file in binary mode
try:
file = Path(file_name)
if file.is_dir():
expanded_file_list.extend([str(f) for f in file.rglob('*') if f.is_file()])
else:
expanded_file_list.append(file_name)

# Hash file contents
for file_name in expanded_file_list:
try:
with open(file_name, 'rb') as f:
# Read the file content in chunks to avoid memory issues with large files
while chunk := f.read(8192): # 8192 bytes per chunk
Expand Down
4 changes: 3 additions & 1 deletion test/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
test: prereq
test1: prereq
cp $< $@

prereq: preprereq1 preprereq2
cp $< $@
cat preprereq2 >> $@

test2: dirprereq
1 change: 1 addition & 0 deletions test/dirprereq/dir/prereq.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
prereq

0 comments on commit c8d3381

Please sign in to comment.