Support test sets from sign language task?

[bricksdont]

Hi Matt and team

I just published the full test sets from the WMT shared task on sign language translation: https://www.wmt-slt.com/

We have the data in WMT XML format here:

https://files.ifi.uzh.ch/cl/archiv/2022/easier/wmtslt/test/dsgs-de/slttest2022.dsgs-de.refs.xml
https://files.ifi.uzh.ch/cl/archiv/2022/easier/wmtslt/test/de-dsgs/slttest2022.de-dsgs.refs.xml

(dsgs= Swiss German Sign language)

I'd like to know whether this can somehow be added to SacreBLEU. A potential issue is that either the source or reference segments are not text, but URLs to videos on our file server. Ideas: SacreBLEU could download individual videos and place them in a folder, or ignore the videos but allow users to evaluate against the text references for dsgs-de.

Let me know your thoughts and kind regards
Mathias