Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve HardlinkDir performance #91

Draft
wants to merge 5 commits into
base: harry/cached-csi
Choose a base branch
from
Draft

Improve HardlinkDir performance #91

wants to merge 5 commits into from

Conversation

airhorns
Copy link
Contributor

@airhorns airhorns commented May 6, 2024 •
edited
Loading

Trying to make HardlinkDir faster

On main:

go test -bench Hardlink -run=^# -benchtime=30s -cpuprofile out.prof
goos: darwin
goarch: arm64
pkg: github.com/gadget-inc/dateilager/test
BenchmarkHardlinkDir-12    	       7	5607450970 ns/op
PASS
ok  	github.com/gadget-inc/dateilager/test	44.303s
⏎

This was referenced May 6, 2024
Merged
Merged
Merged
Merged
@airhorns Graphite App
Copy link
Contributor Author

airhorns commented May 6, 2024

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @airhorns and the rest of your teammates on Graphite Graphite

@airhorns airhorns mentioned this pull request May 6, 2024
Merged
@airhorns airhorns force-pushed the harry/cached-csi branch 2 times, most recently from 48731e7 to 71bdea3 Compare May 8, 2024 04:05
@angelini angelini force-pushed the harry/cached-csi branch 2 times, most recently from 3d9d580 to 8c93d32 Compare May 8, 2024 12:22
angelini and others added 5 commits May 8, 2024 19:17
@angelini
Merge pull request #89 from gadget-inc/harry/cached-csi
942e67d 
`cached` K8S CSI driver
@airhorns
Update nix flake vendor hash -- we added new deps that need to be inc…
1003675 
…luded
@airhorns
Add a test and benchmark for the hardlink dir walker
545589a
@airhorns
Don't MkdirAll for every directory in the hardlink tree
b5ba504 
We know `fs.Walk` does a traversal that gives us parents before children, so we don't need to re-make all the grandparents every time, with the exception of the root, which we make special.
@airhorns
use filepath.WalkDir to avoid stat-ing files unnecessarily
3b51048
@airhorns airhorns force-pushed the harry/faster-hardlink branch from fed2e39 to 3b51048 Compare May 11, 2024 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@airhorns @angelini