Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.33.0 #10636

Draft
wants to merge 54 commits into
base: release
Choose a base branch
from
Draft

Release 0.33.0 #10636

wants to merge 54 commits into from

Conversation

lidel
Copy link
Member

@lidel lidel commented Dec 20, 2024

lidel and others added 30 commits October 30, 2024 17:44
…10566)

* fix(autotls): store certificates at the location from the repo path
* docs(autotls): cert storale and other caveats

---------

Co-authored-by: Marcin Rataj <[email protected]>
docker image was missing, we also don't want to default to ipfs-desktop
* chore: stop using go-homedir

The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.

- `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`
- `homedir.Expand` is replaced by fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.

Additional functionality, such as `DirWritable` and `FileExists` was moved into or included in the `github.com/ipfs/kubo/misc/fsutil` package.
history of 
https://github.com/ipfs/kubo/actions/workflows/docker-image.yml 
shows the job takes ~5m, so 15m is plenty if something goes wrong.
* chore: update to boxo without goprocess
* Use boxo fix for registering metrics
* chore: switch to boxo main with PR 723

---------

Co-authored-by: Marcin Rataj <[email protected]>
Fix Typos and Improve Clarity in Documentation
* feat(bootstrap): /dnsaddr/va1.bootstrap.libp2p.io

this adds
https://github.com/libp2p/js-libp2p-amino-dht-bootstrapper
instance at /dnsaddr/va1.bootstrap.libp2p.io
to ensure Kubo does not depend on single language stack
for initial bootstrap

* docs: document where defaults live

* test: updated bootstrappers
The issue is that for dht providing to work correctly, at least two nodes must
have dht routing.

Before, dht providing in the test always failed, but the test succeeded when
the tested-CID was the one that the failure happened with (the first in the
list for ProvideMany).
…10601)

Obtain actual daemon output on the screen while
debugging tests. 
Might be useful in the future even though it needs to
manually set verbose somewhere.
dependabot bot and others added 24 commits November 27, 2024 19:41
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.6.0 to 5.0.7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@b9fd7d1...015f24e)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.0.7 to 5.1.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@015f24e...7f8b4b4)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Gillis <[email protected]>
[skip changelog] deps: update nopfs
…10623)

* pinmfs: mitigate slow mfs writes when it triggers

This mitigates slow mfs writes when the pinmfs daemon calls mfs.RootNode()

When writing lots of files to MFS, this call triggers a mfs directory cache
sync operations. The cache grows forever and is unbounded. The more files
added to a directory, the longer it takes. In the meantime, writing to mfs is
locked.

The pinmfs, even when no remote pinning services are configured, will trigger
this issue. When RootNode() takes more than 30 seconds, the issue will be triggered
continuously causing a write-deadlock onto MFS.

This commit does not fix the fact that if you write 1M items into an MFS
directory, the first time that the directory is traversed it will still have
to sync those 1M items. It does at least prevent writes from stalling after
about ~6000 items.

* pinmfs: fix test
* Upgrade to Boxo v0.26.0
* Update docs/changelogs/v0.33.md
* cmd/files: flush parent folders

This is a mitigation to increased MFS memory usage in the course of many writes operations.

The underlying issue is the unbounded growth of the mfs directory cache in
boxo. In the latest boxo version, this cache can be cleared by calling Flush()
on the folder. In order to trigger that, we call Flush() on the parent folder
of the file/folder where the write-operations are happening.

To flushing the parent folder allows it to grow unbounded. Then, any read
operation to that folder or parents (i.e. stat), will trigger a sync-operation to match
the cache to the underlying unixfs structure (and obtain the correct node-cid).

This sync operation must visit every item in the cache. When the cache has grown too much,
and the underlying unixfs-folder has switched into a HAMT, the operation can take minutes.

Thus, we should clear the cache often and the Flush flag is a good indicator
that we can let it go. Users can always run with --flush=false and flush at
regular intervals during their MFS writes if they want to extract some performance.

Fixes #8694, #10588.

* cmd/files: docs and changelog for --flush changes
…ons (#10614)

* feat: expose BlockKeyCacheSize and enable WriteThrough when bloom filter disabled
* import/config: add BatchMaxSize and BatchMaxNodes
* config: make BlockKeyCacheSize an OptionalInteger
* config: add and wire datastore.WriteThrough option
* config: omitempty on BlockKeyCacheSize
* changelog: rewrite entry about new options for the datastore
* config: add docs for BatchMaxNodes and BatchMaxSize
* config: make WriteThrough an optional Flag
* changelog: improve description of new datastore/import options
* refactor: DefaultWriteThrough as bool
* chore: boxo v0.26.0
* docs: config and changelog fixes
* feat(libp2p): enable shared TCP listeners

* docs: switch mentions of /ws to /tcp/4001

* feat: AutoTLS.AutoWSS

This adds AutoTLS.AutoWSS flag that is set to true by default.

It will check if Addresses.Swarm contain explicit /ws listener,
and if not found, it will append one per every /tcp listener

This way existing TCP ports are reused without any extra configuration,
but we don't break user's who have custom / explicit /ws listener
already.

I also moved logger around, to include Addresses.Swarm inspection
results in `autotls` logger.

* chore: go-libp2p v0.38.1

https://github.com/libp2p/go-libp2p/releases/tag/v0.38.0
https://github.com/libp2p/go-libp2p/releases/tag/v0.38.1

* docs: AutoTLS.AutoWSS and go-libp2p v0.38.x

* chore: p2p-forge/client v0.2.0

https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0

* fix: disable libp2p.ShareTCPListener() in PNET

* chore(ci): timeout sharness after 15m

average successful run is  <9 minutes, no need to wait for 20
https://github.com/ipfs/kubo/actions/workflows/sharness.yml?query=is%3Asuccess

---------

Co-authored-by: Andrew Gillis <[email protected]>
Co-authored-by: Marcin Rataj <[email protected]>
@lidel lidel mentioned this pull request Dec 20, 2024
47 tasks
@lidel lidel added the skip/changelog This change does NOT require a changelog entry label Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip/changelog This change does NOT require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants