French TTS (`espnet-tts-fr`) struggles with numerical numbers

[notkaramel]

As title suggests, the French TTS is having a hard time generating audio for large number input, in numeric form. Here in this example, the text output from the translation is (from Etherscan):

• "Graphique de croissance de l'offre d'Ether. x Axis,Time,du 30 juillet 2015 au 15 juin 2023. y Axis,Ether Supply,du 72049306.59375 au 120532870.54151082"

Here is the audio generated: French TTS Rendering (via Google Drive).

Notice that the service struggles to say "2015", "2023", and most of the remaining large numbers.

[notkaramel]

French TTS also struggles with describing data. Example from a pie chart from https://www.inspq.qc.ca/covid-19/donnees/age-sexe

Audio: Pie Chart Rendering (via Google Drive)

Note that the French TTS struggles to say concatenated numbers or with decimal point/comma, like (in bold):

• "40-49 ans, 15,76 pour cent"
• "50-59 ans, 12,70 pour cent"
• ...

[notkaramel]

Weird case: French TTS can not say "2" or "3" in French.

Graphic:

Text (in French): "Cette photo intérieure contient les objets ou personnes suivants: 3 livres, 3 téléviseurs, 2 personnes, 2 chaises, un téléphone portable, une souris, et un clavier."

Audio: Photo rendering (via Google Drive)

Notice how French TTS said "3" as "et tres" (I think), and "2" as "two". "2 personnes" should have been "deux personnes" but it was "two personnes".

[jeffbl]

Put in Jul10 sprint for investigation of possible solutions. Particularly important for charts!

[notkaramel]

Moved to sprint Jul31 to investigate and resolve during this sprint instead

[notkaramel]

• In order to help French TTS speaks numbers, I added the num2words library. It allows number conversion to multiple language, would definitely benefit future addition of other languages with similar implementation.

• There are quite a wide range of possibilities/cases that need to be taken into consideration:

  NOTE: the logic flow is: break down a long string of text into segments; for each segment, we check if the string could potentially a number, or if it's a syntax that involves numbers; if positive, we then convert the string in numerical form into string of text then perform TTS.

  • Case 0 (simplist): the number is an integer or a float ➡️ convert directly to text
  • Case 1: Float number but the decimal separator is a comma "," instead of a dot ".". This is due to multilang-support uses LMM trained on Metropolitan French where they have a different notation system. Here, we check by replace the separator then attempt to type cast to a float number. If able to, we return the text result.
  • Case 2: Range. Can see above from the pie chart example with age range, where "20-29" could not be directly intepreted as numbers.
  • Case 3 (edge): The number is next to a comma/dot. Occured with the Etherium example above: "2023.". Since we are processing substrings of the whole sentence, solving Case 0 also solve this case.
  • Case 3: TBD?

• Those are the cases for the examples of this issue, and surely I am not covering every possible cases that may occur.

[notkaramel]

Moved (again) to sprint Aug 21st because I'm a bit behind schedule and Regex took me a while to understand. Apologies!