Better handling of file extensions in the browser

[agmap]

Many files are not wrapped in a directory. The file extension is therefore missing.
But browsers seem to extract the file extension from the magic numbers / file signatures.

Json files
When I open a json file in ipfs.io, it gets the correct file extension.
When I open a json file in inbrowser.link its in an HTML page. I can’t download it as json.
For example:
https://ipfs.io/ipfs/bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy
https://bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy.ipfs.inbrowser.link/

The same applies to image files:
If I open an image in ipfs.io, the image gets the correct file extension
If I open an image file in inbrowser.link, it is in an HTML page and I cannot download it as an image from the browser.
Example:
https://ipfs.io/ipfs/bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce
https://bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce.ipfs.inbrowser.link/

[SgtPooki]

@agmap can you clarify what browser you are using and how you are trying to "download it as X" ?

I just deployed v1.8.2 onto inbrowser.link so it's got a lot of updates.

In Brave, if I right click an image and "save" it works as expected. However, if I go to `File > Save page As" I am seeing the experience you've described..

I am not sure if browsers handle resources returned from service-workers differently, but we can investigate

[SgtPooki]

@agmap I saved the raw HEX of the files from both ipfs.io and inbrowser.link to .bin files (copy full hex, save to file by pasting) and then compared them and they are not different, so I am wondering if this is something we can control...

@lidel any ideas?

Compare command ran:

cmp -l ipfs-io-ipfs-bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce.bin bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce-ipfs-inbrowser-link.bin

# no output because they're the same

[agmap]

I am using chrome desktop in windows 11.

For the example of the json:
Right click --> "save as"

• ipfs.io: I am getting the the *.json (bafkre…eqyy.json) --> also pretty print
• inbrowser.link: I am getting an *.html (bafkrei…eqyy.ipfs.inbrowser.link.html) --> not pretty print

For the example of the image:
Right click --> "save image as"

• ipfs.io: I am getting the image as cid.png (bafybei….blhnce.png)
• inbrowser.link: I am getting an *.html (download.html)

[lidel]

Smells like something related to Content-Type and/or Content-Disposition header in HTTP response.
They inform browser how to render content and what filename is used in "Save As".

• https://specs.ipfs.tech/http-gateways/path-gateway/#cache-control-response-header
• https://specs.ipfs.tech/http-gateways/path-gateway/#content-disposition-response-header

We likely should have JSON/PNG tests in gateway-conformance for this – filled ipfs/gateway-conformance#234 so we confirm or close that gap.

Did quick test (Firefox) for first two links from the original report and inbrowser.link seems to return invalid content-type HTTP header:

inbrowser.link (SW)	dweb.link (Rainbow)

@SgtPooki do you know if it is a bug in verified-fetch, or browser not displaying HTTP header from Response correctly ?

[SgtPooki]

@SgtPooki do you know if it is a bug in verified-fetch, or browser not displaying HTTP header from Response correctly ?

I will investigate if I can reproduce when running npm run build && node packages/gateway-conformance/dist/src/demo-server.js from helia-verified-fetch to see if this is a verified-fetch problem, or a sw-gateway problem.

[SgtPooki]

verified-fetch is for sure returning content-type of html for json... but that is because of https://github.com/ipfs/helia-verified-fetch/blob/a041bdba954610beeacb6c768b0c4907633ae4fb/packages/gateway-conformance/src/fixtures/content-type-parser.ts#L7

sw-gateway does this at

service-worker-gateway/src/lib/content-type-parser.ts

Line 5 in 17421d6

export const defaultMimeType = 'text/html'

Here are logs for the request of bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy (json):

  helia:trustless-gateway-block-broker:127.0.0.1 GET http://127.0.0.1:8080/ipfs/bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy?format=raw 200 +0ms
  helia:trustless-gateway:session:bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy:trace got block for bafkreieze572daxva52asutm4u2bgtjtifur2a4d2aiur7lymjcwhoeqyy from http://127.0.0.1:8080/ +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace set _fileSize to 759 +41ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace requestRangeStart and requestRangeEnd are null +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace set request body with fileSize 759 +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace returning body unmodified for non-range, or invalid range, request +0ms
  content-type-parser contentTypeParser called for fileName: undefined +0ms
  content-type-parser no detectedType +1ms
  content-type-parser checking for svg +0ms
  helia:verified-fetch:raw-plugin:trace setting content type to "text/html; charset=utf-8" +43ms
  helia:verified-fetch:trace checking for content disposition +43ms
  helia:verified-fetch:trace download not requested +0ms
  helia:verified-fetch:trace no filename specified in query +0ms
  helia:verified-fetch:trace no content disposition specified +0ms

This has to do with https://www.npmjs.com/package/@sgtpooki/file-type (fork of https://www.npmjs.com/package/file-type) not handling content-type for text.

if it's json, we will need to read the entire contents to determine if it's json.. we do allow passing a custom content-type parser to helia-verified-fetch, so we could parse the entire content to determine if it's json.

---

Here are logs for the request of bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce (image):

  helia:verified-fetch:dag-pb-plugin got async iterator for bafybeiblgxritjybxffn5te6aupebq7m77xm7xiqqkiq7eete7wrblhnce/ +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace set _fileSize to null +1ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace requestRangeStart and requestRangeEnd are null +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace set request body with fileSize null +0ms
  helia:verified-fetch:helia:verified-fetch:byte-range-context:trace returning body unmodified for non-range, or invalid range, request +0ms
  content-type-parser contentTypeParser called for fileName: undefined, byte size=262144 +8s
  content-type-parser detectedType: image/png +0ms
  helia:verified-fetch:dag-pb-plugin:trace contentTypeParser returned image/png +1ms
  helia:verified-fetch:dag-pb-plugin:trace setting content type to "image/png" +0ms
  helia:verified-fetch:trace checking for content disposition +1ms
  helia:verified-fetch:trace download not requested +0ms
  helia:verified-fetch:trace no filename specified in query +0ms
  helia:verified-fetch:trace no content disposition specified +0ms