Skip to content

fix(handleFileUpload): reliably pin and announce uploaded directory CID #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

akhileshthite
Copy link
Member

@akhileshthite akhileshthite commented Apr 22, 2025

When uploading large files via handleFileUpload, the CID is generated locally but never shows up on the public DHT (IPFS Checker reports “0 working providers”).

What we tried

  • After unixfs.addAll(), we now
    • Pin the root CID (node.pins.add(rootCid))
    • Manually advertise it (await node.routing.provide(rootCid))

Console:

Pinned root CID: bafybeidthg7n5lg67eawgyxtliftty2aeebs3a4fhw7wcbf2trlz2f7iem
Pinned root CID: bafkreibdhnynbrqiihdwcauqwywx4a7kkggw2ixnoa6yuhehlcz5d5ylfa
Provided CID on DHT: bafybeidthg7n5lg67eawgyxtliftty2aeebs3a4fhw7wcbf2trlz2f7iem

Files:

ipfs-handler:

async function handleFileUpload(request, sendResponse) {

libp2p: https://github.com/p2plabsxyz/peersky-browser/blob/fix/enable-dht-provider/src/protocols/helia/libp2p.js

Current behavior:

Despite those calls, the checker still reports zero providers for the CID:
Http gateway: https://bafybeidthg7n5lg67eawgyxtliftty2aeebs3a4fhw7wcbf2trlz2f7iem.ipfs.dweb.link/

IPFS check: https://check.ipfs.network/?cid=bafybeidthg7n5lg67eawgyxtliftty2aeebs3a4fhw7wcbf2trlz2f7iem

❌ Found 0 working providers (out of 4 provider records sampled from Amino DHT and IPNI) that could be connected to and had the CID available over Bitswap:
	12D3KooWDngsXgk7s5SkpxqdZMuxEUbbbz1yAHgrJut7iFGpSWD8
		Connected: ❌ failed to dial: failed to dial 12D3KooWDngsXgk7s5SkpxqdZMuxEUbbbz1yAHgrJut7iFGpSWD8: all dials failed
		  * [/ip4/10.0.0.53/tcp/55117] gater disallows connection to peer
		  * [/ip4/10.0.0.53/tcp/55119/ws] gater disallows connection to peer
		  * [/ip4/127.0.0.1/tcp/55117] gater disallows connection to peer
		  * [/ip4/127.0.0.1/tcp/55119/ws] gater disallows connection to peer
		  * [/ip6/::1/tcp/55118] gater disallows connection to peer
		  * [/ip6/::1/tcp/55120/ws] gater disallows connection to peer
		  * [/ip6/75a::3:0:73e:802/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] gater disallows connection to peer
		  * [/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] dial backoff
		Peer Multiaddrs:
			/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit
		Found in: Amino DHT
	12D3KooWSDMpT13dPkRdQmrv3cyPNwcsum2n1cvPdUJHBW9riG7i
		Connected: ❌ failed to dial: failed to dial 12D3KooWSDMpT13dPkRdQmrv3cyPNwcsum2n1cvPdUJHBW9riG7i: all dials failed
		  * [/ip4/10.0.0.53/tcp/56532] gater disallows connection to peer
		  * [/ip4/10.0.0.53/tcp/56534/ws] gater disallows connection to peer
		  * [/ip4/127.0.0.1/tcp/56532] gater disallows connection to peer
		  * [/ip4/127.0.0.1/tcp/56534/ws] gater disallows connection to peer
		  * [/ip6/::1/tcp/56533] gater disallows connection to peer
		  * [/ip6/::1/tcp/56535/ws] gater disallows connection to peer
		  * [/ip6/1e::3:0:2:be0e/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] gater disallows connection to peer
		  * [/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] dial backoff
		Peer Multiaddrs:
			/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit
		Found in: Amino DHT
	12D3KooWL1NJ9jqpH8kkmw1w2kEgAFSuKfD5oBGSAeoDUMUpjubu
		Connected: ❌ failed to dial: failed to dial 12D3KooWL1NJ9jqpH8kkmw1w2kEgAFSuKfD5oBGSAeoDUMUpjubu: no addresses
		Found in: Amino DHT
	12D3KooWEwwHXJBZiTcey62EbRK7iBMSFnXJ8WSWQbDbGYpmCaBo
		Connected: ❌ failed to dial: failed to dial 12D3KooWEwwHXJBZiTcey62EbRK7iBMSFnXJ8WSWQbDbGYpmCaBo: no addresses
		Found in: Amino DHT

RAW OUTPUT:
[
  {
    "ID": "12D3KooWDngsXgk7s5SkpxqdZMuxEUbbbz1yAHgrJut7iFGpSWD8",
    "ConnectionError": "failed to dial: failed to dial 12D3KooWDngsXgk7s5SkpxqdZMuxEUbbbz1yAHgrJut7iFGpSWD8: all dials failed\n  * [/ip4/10.0.0.53/tcp/55117] gater disallows connection to peer\n  * [/ip4/10.0.0.53/tcp/55119/ws] gater disallows connection to peer\n  * [/ip4/127.0.0.1/tcp/55117] gater disallows connection to peer\n  * [/ip4/127.0.0.1/tcp/55119/ws] gater disallows connection to peer\n  * [/ip6/::1/tcp/55118] gater disallows connection to peer\n  * [/ip6/::1/tcp/55120/ws] gater disallows connection to peer\n  * [/ip6/75a::3:0:73e:802/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] gater disallows connection to peer\n  * [/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] dial backoff",
    "Addrs": [
      "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit"
    ],
    "ConnectionMaddrs": null,
    "DataAvailableOverBitswap": {
      "Duration": 0,
      "Found": false,
      "Responded": false,
      "Error": ""
    },
    "Source": "Amino DHT"
  },
  {
    "ID": "12D3KooWL1NJ9jqpH8kkmw1w2kEgAFSuKfD5oBGSAeoDUMUpjubu",
    "ConnectionError": "failed to dial: failed to dial 12D3KooWL1NJ9jqpH8kkmw1w2kEgAFSuKfD5oBGSAeoDUMUpjubu: no addresses",
    "Addrs": [],
    "ConnectionMaddrs": null,
    "DataAvailableOverBitswap": {
      "Duration": 0,
      "Found": false,
      "Responded": false,
      "Error": ""
    },
    "Source": "Amino DHT"
  },
  {
    "ID": "12D3KooWEwwHXJBZiTcey62EbRK7iBMSFnXJ8WSWQbDbGYpmCaBo",
    "ConnectionError": "failed to dial: failed to dial 12D3KooWEwwHXJBZiTcey62EbRK7iBMSFnXJ8WSWQbDbGYpmCaBo: no addresses",
    "Addrs": [],
    "ConnectionMaddrs": null,
    "DataAvailableOverBitswap": {
      "Duration": 0,
      "Found": false,
      "Responded": false,
      "Error": ""
    },
    "Source": "Amino DHT"
  },
  {
    "ID": "12D3KooWSDMpT13dPkRdQmrv3cyPNwcsum2n1cvPdUJHBW9riG7i",
    "ConnectionError": "failed to dial: failed to dial 12D3KooWSDMpT13dPkRdQmrv3cyPNwcsum2n1cvPdUJHBW9riG7i: all dials failed\n  * [/ip4/10.0.0.53/tcp/56532] gater disallows connection to peer\n  * [/ip4/10.0.0.53/tcp/56534/ws] gater disallows connection to peer\n  * [/ip4/127.0.0.1/tcp/56532] gater disallows connection to peer\n  * [/ip4/127.0.0.1/tcp/56534/ws] gater disallows connection to peer\n  * [/ip6/::1/tcp/56533] gater disallows connection to peer\n  * [/ip6/::1/tcp/56535/ws] gater disallows connection to peer\n  * [/ip6/1e::3:0:2:be0e/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] gater disallows connection to peer\n  * [/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit] dial backoff",
    "Addrs": [
      "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ/p2p-circuit"
    ],
    "ConnectionMaddrs": null,
    "DataAvailableOverBitswap": {
      "Duration": 0,
      "Found": false,
      "Responded": false,
      "Error": ""
    },
    "Source": "Amino DHT"
  }
]

@akhileshthite akhileshthite added bug Something isn't working priority: high For important issues that affect many users or major functionality of the project ipfs/ipns Related to ipfs protocol labels Apr 22, 2025
@akhileshthite akhileshthite self-assigned this Apr 22, 2025
@2color
Copy link

2color commented Apr 22, 2025

Based on the results you shared from IPFS Check, it appears that the CID is successfully advertised to the DHT, but that the node providing is either offline or unreachable because it's behind NAT. Currently, NAT traversal is only supported in js-libp2p (used by Helia) by the webrtc network transport.

From the looks of it, your Helia nodes are only listening on TCP and WS. My recommendation would be to add a webrtc listener as follows: https://github.com/ipfs/helia/blob/313e2c10a71c4f42533c3f077c6720563dd09ce5/packages/helia/src/utils/libp2p-defaults.browser.ts#L46-L51

But if you are running these Helia nodes behind NAT, you are likely to experience such problems, because NAT traversal fails on some networks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ipfs/ipns Related to ipfs protocol priority: high For important issues that affect many users or major functionality of the project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants