Store webmentions using a known serialisation format

[paulrobertlloyd]

Using getMentionsForPage, normalised objects are returned, but not in a specified or common format.

Mention data is currently normalised in a format that approximates JF2, but not for deeply nested properties, and uses non-standard values for type. This means it is not possible to query processed mentions, return mention data and convert it to a different format.

The use case for being able to covert mention data to different formats can be seen in indieweb/webmention-ecosystem#2, where there is a desire to return mentions with either mf2, JF2 or JSON Feed serialisations. I’d like to return mentions as JF2, at least initially, and potentially other formats in the future.

Unfortunately, the object normalisation takes place when mentions are processed, with the original mf2 parsing discarded. This means it's not possible to convert the mention data to JF2 (or mf2) with libraries that parse mf2, or without employing additional processing to convert the current serialisation to adhere to JF2.

Potential options

• Store mf2 when processing mentions, and only do the conversion when using getMentionsForPage.
• Use JF2 when processing mentions, and ensure the normalisation follows that specification.
• Provide an option at the handler level to determine which serialisation format mentions are stored to the database with OR provide an option on the getMentionsForPage function to return a desired format (mf2 or JF2).

It would be useful to have a function to return all processed mentions, not just those for a given target, but I will create a separate PR for that.

[vandie]

What's the use case?
Can you please also not label things as bugs when that's not what they are 😅

[paulrobertlloyd]

The bug label gets added by the new issue workflow and can’t be removed by an issue’s author.

I may have been a bit hasty with this issue; am trying to figure out how mentioned get saved/normalised. As I look at the different example webmentions that I’m testing with have been saved, it appears as if the normalisation only happens at the top level, and not for nested properties.

I have a library mf2tojf2 that normalises deeply nested MF2 objects; I can try creating a PR that uses that module?

[paulrobertlloyd]

I’ve updated my original comment to better describe my issue (hopefully!) and provide a link to some example use cases.

In the meantime, I’m going to implement the authorship spec on mf2tojf2, as it may be helpful to those converting from mf2 and JF2 for use in webmention parsing and processing.

[vandie]

I think looking at it, option 2 makes the most sense but would be a breaking change both for users of this library as well as potentially for any storage implementations.

You're right in saying that this package currently uses a none standard format as it was built originally for my own use at a time when JF2 wasn't as common (with any similarities coming from the fact they're both rooted in MF2)