Skip to content

chore: merge new changes from ipfs/kubo master #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 450 commits into
base: master
Choose a base branch
from

Conversation

alvin-reyes
Copy link

No description provided.

gammazero and others added 30 commits October 22, 2024 11:26
* use boxo v0.24.2
* update changelog
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.4.0 to 4.6.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@6d79887...b9fd7d1)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…10566)

* fix(autotls): store certificates at the location from the repo path
* docs(autotls): cert storale and other caveats

---------

Co-authored-by: Marcin Rataj <[email protected]>
…10566)

* fix(autotls): store certificates at the location from the repo path
* docs(autotls): cert storale and other caveats

---------

Co-authored-by: Marcin Rataj <[email protected]>
(cherry picked from commit 1ca0ae0)
(cherry picked from commit d4ae7fa)
docker image was missing, we also don't want to default to ipfs-desktop
(cherry picked from commit e17dc21)
* chore: stop using go-homedir

The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.

- `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`
- `homedir.Expand` is replaced by fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.

Additional functionality, such as `DirWritable` and `FileExists` was moved into or included in the `github.com/ipfs/kubo/misc/fsutil` package.
* chore: stop using go-homedir

The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.

- `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`
- `homedir.Expand` is replaced by fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.

Additional functionality, such as `DirWritable` and `FileExists` was moved into or included in the `github.com/ipfs/kubo/misc/fsutil` package.

(cherry picked from commit 4009ad3)
this should help with debugging
ipshipyard/p2p-forge#8
on boxes with debug log level
(and hide noise from normal users)
history of 
https://github.com/ipfs/kubo/actions/workflows/docker-image.yml 
shows the job takes ~5m, so 15m is plenty if something goes wrong.
gammazero and others added 30 commits April 30, 2025 22:01
* datastore: metrics optional and off by default

When ipfs is initialized, the datastore metrics wrapper is not configured by default as it previously was. To enable datastore metrics during initialization, specifying the appropriate `--profile` option.

To enable datastore metrics tracking wrapper, initialize with datastore profile name + "-measure" suffix. For example:
```
ipfs init --profile flatfs-measure
```

Closes #10767

* fix sharness tests for new datastore dafaults

* Add sharness test to check metrics added by flatfs-measure profile

* Document updated metrics in changelog

* update config doc with new profiles

* docs(changelog): separate section

* initialize non-measure pebbleds with FormatMajorVersion config

* docs: fix typos, add docs link

---------

Co-authored-by: gammazero <[email protected]>
Co-authored-by: Marcin Rataj <[email protected]>
* feat: add Bitswap configuration and related tests
* fix: update Bitswap function to use 'provide' parameter for server enablement
* docs: update changelog for Bitswap functionality changes
* fix: update Bitswap server enablement logic and improve related tests
* fix: rename BitswapConfig to Bitswap and update references
* docs: config and changelog
* fix: `ipfs cat` panic when `Bitswap.Enabled=false`

Fixes panic described in:
#10782 (comment)

---------

Co-authored-by: gystemd <[email protected]>
Co-authored-by: gammazero <[email protected]>
Co-authored-by: Giulio Piva <[email protected]>
Co-authored-by: Marcin Rataj <[email protected]>
* Feat: http retrieval as experimental feature

This introduces the http-retrieval capability as an experimental feature.

It can be enabled in the configuration `Experimental.HTTPRetrieval.Enabled = true`.

Documentation and changelog to be added later.

* refactor: HTTPRetrieval.Enabled as Flag

* docs(config): HTTPRetrieval section

* refactor: reusable MockHTTPContentRouter

* feat: HTTPRetrieval.TLSInsecureSkipVerify

allows self-signed certificates in tests

* feat(config): HTTPRetrieval.MaxBlockSize

* test: end-to-end HTTPRetrieval.Enabled

this spawns two http services on localhost:
1. HTTP router that returns HTTP provider when /routing/v1/providers/cid  i queried
2. HTTP provider that returns a block when /ipfs/cid is queried
3. Configures Kubo to use (1) instead of cid.contact

this seems to work (running test with DEBUG=true shows (1) was queried
for the test CID and returned multiaddr of (2), but Kubo never requested
test CID block from (2) – needs investigation

* fix: enable /routing/v1/peers for non-cid.contact

we artificially limited every delegated routing endpoint because of
cid.contact being limited to one endpoint

* feat: Routing.DelegatedRouters

make it easy to override the hardcoded implicit HTTP routeur URL
without having to set the entire custom Router.Routers and
Router.Methods

(http_retrieval_client_test.go still needs to be fixed in future commit)

* test: flag remaining work

* docs: review feedback

* refactor: providerQueryMgr with bitswapNetworks

this fixes two regressions:

(1) introduced in #10717
    where we only used bitswapLib2p query manager
    (this is why E2E did not act on http provider)

(2) introduced in #10765
    where it was not possible to set binary peerID in IgnoreProviders
    (we changed to []string)

* refactor: Bitswap.Libp2pEnabled

replaces Bitswap.Enabled with Bitswap.Libp2pEnabled
adds tests that confirm it is possible to disable libp2p bitswap fully
and only keep http in client mode

also, removes the need for passing empty blockstore in client-only mode

* docs: changelog

---------

Co-authored-by: Marcin Rataj <[email protected]>
* Add MFS command line options, extend existing mount functions for MFS, set defaults.

* Directory listing and file stat.

* Add a read-only MFS view.

* Add mkdir and interface checks.

* Add remove and rename functionality.

* Implement all required write interfaces.

* Adjust mount  functions for other architechtures.

* Merge branch 'master' into feat/10710-mfs-fuse-mount

* Write a basic read/write test.

* Write more basic tests, add a mutex to the file object, fix modtime.

* Add a concurrency test, remove mutexes from file and directory structures.

* Refactor naming(mfdir -> mfsdir) and add documentation.

* Add CID retrieval through ipfs_cid xattr.

* Add docs, add xattr listing, fix bugs for mv and stat, refactor.

* Add MFS command line options, extend existing mount functions for MFS, set defaults.

* docs phrasing

* docs: Mounts.MFS

* docs: warn about lazy-loaded DAGs

* test: TEST_FUSE=1 ./t0030-mount.sh -v

---------

Co-authored-by: Guillaume Michel <[email protected]>
Co-authored-by: guillaumemichel <[email protected]>
Co-authored-by: Marcin Rataj <[email protected]>
* repo: optionally wait to acquire repo lock

Retry acquiring repo lock for a specified amount of time. Retry once per second until time is expired ro lock is acquired.

Lock wait timeout is specified using the environment variable `IPFS_WAIT_REPO_LOCK` and assigning it a value that is parsable as a golang `time.Duration`. For example:

```
IPFS_WAIT_REPO_LOCK="5s"
```

Closes #10482

* document IPFS_WAIT_REPO_LOCK

---------

Co-authored-by: Marcin Rataj <[email protected]>
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.4.0 to 5.4.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@0565863...ad3126e)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: 5.4.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: add or force update .github/workflows/stale.yml

* chore: add or force update .github/workflows/generated-pr.yml
* fix(config): explicit Provider.Enabled flag

Adds missing config option described in
#10803

* refactor: remove Experimental.StrategicProviding

removing experiment, replaced with Provider.Enabled

* test(cli): routing [re]provide

updated and added tests for manually triggering provide and reprovide
and making them respect global configuration flag to avoid
inconsistent behaviors

* docs: improve DelegatedRouters

* refactor: default DefaultProviderWorkerCount=16

- simplified default for both
- 16 is safer for non-accelerated DHT client
- acceletated DHT performs better without limit anyway - updated docs
* fix(config): explicit Provider.Enabled flag

Adds missing config option described in
#10803

* refactor: remove Experimental.StrategicProviding

removing experiment, replaced with Provider.Enabled

* test(cli): routing [re]provide

updated and added tests for manually triggering provide and reprovide
and making them respect global configuration flag to avoid
inconsistent behaviors

* docs: improve DelegatedRouters

* refactor: default DefaultProviderWorkerCount=16

- simplified default for both
- 16 is safer for non-accelerated DHT client
- acceletated DHT performs better without limit anyway - updated docs

(cherry picked from commit 2ab3f58)
* fix(fusei/ux): check if paths exist, print err

* fix(fuse): ipns 'could not resolve'

error type changed when code got extracted to boxo, but it was not
caught because of FUSE tests do not cover IPNS in online mode

Closes #8095
Closes #2167
Closes #3013

* docs: clarify opt-in
* fix(fusei/ux): check if paths exist, print err

* fix(fuse): ipns 'could not resolve'

error type changed when code got extracted to boxo, but it was not
caught because of FUSE tests do not cover IPNS in online mode

Closes #8095
Closes #2167
Closes #3013

* docs: clarify opt-in

(cherry picked from commit f84fb28)
* chore: update to go-log/v2

go-log v2 has been out for quite a while now and it is time to deprecate v1.

Replace all use of go-log with go-log/v2
Makes /api/v0/log/tail useful over HTTP
Updates dependencies that have moved to go-lov/v2
Removes support for ContextWithLoggable as this is not needed for tracing-like functionality
- Replaces: PR #8765
- Closes issue #8753
- Closes issue #9245
- Closes issue #10809

Other fixes:
* update go-ipfs-cmds
* update http logs test
* fix test
* Read/send one line of log data at a time
* Update -log-level docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.