Encoding issue (?) in the message sent by Markdown.LinkClicked

[VincentDemery]

I have what seems to be an encoding issue with the message sent by Markdown.LinkClicked.

Consider the code below, with accented characters. "tété" shows up correctly in the Markdown widget. Clicking on it, and thus sending the message.href "tété" replaces the text by "t%C3%A9t%C3%A9", where it should be "tété". I do not know how the message handles the string, or what is the encoding used.

#!/usr/bin/python3

from textual import on
from textual.app import App, ComposeResult
from textual.widgets import Markdown

class MyApp(App):
    def compose(self) -> ComposeResult:
        self.md = Markdown(markdown="[tété](tété)")
        yield self.md
            
    def on_markdown_link_clicked(self, message: Markdown.LinkClicked):
        self.md.update(message.href)

if __name__ == "__main__":
    app = MyApp()
    app.run()

It will be helpful if you run the following command and paste the results:

textual diagnose

I am sorry but this command returns textual not found. I use Archlinux with

• python 3.11.5-2
• python-textual 0.36.0-1

[davep]

I am sorry but this command returns textual not found.

You likely haven't installed textual-dev.

[VincentDemery]

Thanks. I have a hard time installing it (not in the AUR, and pip tells me that "error: externally-managed-environment"). If it is possible not to install it that would be easier for me.

[davep]

It's fine if you can't, although you should be able to install it the same way you would Textual. But don't worry about it if you can't; it's unlikely the issue you're asking about is down to your operating system and the like. :-)

[TomJGooding]

Clicking on it, and thus sending the message.href "tété" replaces the text by "t%C3%A9t%C3%A9", where it should be "tété". I do not know how the message handles the string, or what is the encoding used.

This is URL encoding (AKA percent-encoding) which will convert non-ASCII characters to a percent sign followed by two hex digits.

For example, https://www.google.com/search?q=tété is not actually a valid URL, your browser needs to encode this to https://www.google.com/search?q=t%C3%A9t%C3%A9

[TomJGooding]

Thanks. I have a hard time installing it (not in the AUR, and pip tells me that "error: externally-managed-environment"). If it is possible not to install it that would be easier for me.

You might find this video helpful: Arch Linux's Python Protects Me From Myself

If you plan on developing Textual apps, I would instead recommend installing textual and textual-dev using pip in a virtual environment.

[VincentDemery]

Thanks for your help with textual-dev; indeed I consider using a virtual environment. As you could see for my Github profile, I do not develop much, so I would need to maintain this environment in top of my global environment.

although you should be able to install it the same way you would Textual.

Actually the Arch User Repository contains python-textual but not python-textual-dev.

Back to the original problem, using urllib.parse.unquote allows me to solve my problem (found here):

#!/usr/bin/python3

from urllib.parse import unquote

from textual import on
from textual.app import App, ComposeResult
from textual.widgets import Markdown

class MyApp(App):
    def compose(self) -> ComposeResult:
        self.md = Markdown(markdown="[tété](tété)")
        yield self.md
            
    def on_markdown_link_clicked(self, message: Markdown.LinkClicked):
        self.md.update(unquote(message.href))

if __name__ == "__main__":
    app = MyApp()
    app.run()

However, could you explain where this conversion happens and whether this is the expected behavior?

[TomJGooding]

It looks like this encoding is done by the markdown parser used by Textual.

from pprint import pprint

from markdown_it import MarkdownIt

parser = MarkdownIt("gfm-like")
markdown = "[tété](tété)"
for token in parser.parse(markdown):
    pprint(token)

...
      children=[Token(type='link_open',
                      tag='a',
                      nesting=1,
                      attrs={'href': 't%C3%A9t%C3%A9'},
...

It makes sense to me that href should be a valid encoded URL.

[willmcgugan]

I guess the consensus would be to unquote the href in the message?

[github-actions]

Don't forget to star the repository!

Follow @textualizeio for Textual updates.

[jspv]

I believe this change has the side effect of unquoting already encoded characters that should not be unquoted. See #5426. This breaks signed URLs as the characters don't match the signature. (Ideally, href should be the identical link that is in the markdown.)

[VincentDemery]

Je suis en vacances (bien méritées), je serai de retour le 6 janvier. I am on vacation, I will be back on January 6th. Vincent -- Vincent Demery ***@***.***>