MultiPic (Dunabeitia et al. 2022)

[LinguList]

@AnnikaTjuka,we want to add a rather large database on picture norms. I will here illustrate what I did so far in the branch dunabeitia, and @alzkuc will follow up, also in that branch for now, later merging. If you, @AnnikaTjuka, can later check, it may be interesting, as we will add some 100 variables, if not more, for a couple of new languages as well.

[LinguList]

First, @alzkuc, please check how to run NoRaRe (there are instructions, also recent ones on https://calc.hypotheses.org).

Then, please install pynorare:

pip install pynorare

And then git-clone pynorare, and checkout the branch that we work on:

git clone git@github.com:concepticon/norare-data
cd norare-data
git branch dunabeitia
git pull origin dunabeitia

Then run the norare to test if it works:

norare download Dunabeitia-2022-MultiPic
norare map Dunabeitia-2022-MultiPic

Then check from within Python:

from pynorare import NoRaRe
from pathlib import Path

nora = NoRaRe(Path(""))
multipic = nora.datasets["Dunabeitia-2022-MultiPic"]
languages = ["german", "italian"]
vecs = {language: [] for language in languages}
for concept in multipic.concepts.values():
    for language in languages:
        if language + "_shannon_diversity" in concept:
            vecs[language] += [concept[language + "_shannon_diversity"]]
        else:
            vecs[language] += [0] # careful this is not optimal!

# compute correlations
from scipy.stats import kendalltau
kendalltau(vecs["german"], vecs["italian"])

[LinguList]

Okay, once you have done this, @alzkuc, please fill in the gaps. You must essentially add more languages in the code datasets/Dunabeitia-2022-MultiPic/norare.py, where it says:

    languages = OrderedDict({
            "Czech": "CZECH", 
            "Finnish": "FINNISH",
            "Italian": "ITALIAN",
            "Korean": "KOREAN",
            "German": "GERMAN",
            "Catalan": "CATALAN",
            "Hebrew": "HEBREW",
            })

And then also in the JSON, you must please add more lines (I hope this is self-explaining!)

Then rerun the norare map procedure!

[LinguList]

Please also edit norare-data/datasets.tsv. Later, we must add all columns to norare-data/norare.tsv, but we can do so later. (adding bibtex also, etc.)

[alzkuc]

All of this is done now: datasets.tsv has been updated, norare.py includes all the added languages, and the Dunabeitia dataset has been completed with all the languages and their corresponding norms, along with an updated metadata file.

[AnnikaTjuka]

@alzkuc Could you please check if the command norare stats runs without an error?

[alzkuc]

@alzkuc Could you please check if the command norare stats runs without an error?

scheint in Ordnung zu sein :-) @AnnikaTjuka

[LinguList]

@alzkuc, I found errors in the data. The screenshot shows that we have percentages > 100. This should not have passed during conversion. I have corrected the errors already, but problems remain, and it is useful to discuss what happened here, since to my knowledge it should not have been possible to compile the data with the code you pushed in norare.py.

[LinguList]

There are several problems.

If you do the data download automatically, you obtain the raw data by the authors (I upload the file again here, as it is shared with a free license).

MultiPic Dataset Open Materials.csv

This file contains language names that you do not have in norare.py, yet without matching the filenames, we cannot retrieve them.

Check here:

    # define languages we want to keep
    languages = OrderedDict({
...
            "Dutch": "DUTCH",
            "Mandarin": "MANDARIN",            
            "CypriotGreek": "CGREEK",
...
            "Arabic": "ARABIC",
            "Malaysian": "MALAYSIAN",
...
            "English": "ENGLISH",
            "Flemish": "FLEMISH",
            })

MultiPic has "Belgian Dutch", not "Flemish", and "Mandarin Chinese", etc. If you run norare map Dunabeitia-2022-MultiPic, it won't map these. They will be empty, yet your data showed them, so I dont' know if you shared the state of the norare.py script that you were using to compile the data.

[LinguList]

Even more difficult is "English". Here, we list English as the first entry in the metadata. So you leave it out when you list English later on:

          {"name": "BASQUE", "datatype": "string"},
          {"name": "BASQUE_RESPONSE_NUMBER", "datatype": "integer"},
          {"name": "BASQUE_SHANNON_DIVERSITY", "datatype": {"base": "float", "min": 0.0}},
          {"name": "BASQUE_MODAL_RESPONSE_PERCENTAGE", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},
          {"name": "BASQUE_DUNNO_PERCENTAGE", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},
          {"name": "BASQUE_IDIOSYNCRATIC_RESPONSE_PERCENTAGE", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},
          {"name": "BASQUE_FAMILIARITY", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},
          {"name": "ENGLISH_RESPONSE_NUMBER", "datatype": "integer"},
          {"name": "ENGLISH_SHANNON_DIVERSITY", "datatype": {"base": "float", "min": 0.0}},
          {"name": "ENGLISH_MODAL_RESPONSE_PERCENTAGE", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},
          {"name": "ENGLISH_DUNNO_PERCENTAGE", "datatype": {"base": "float", "min": 0.0, "max": 100.0}},

But this means the code will write English anyway, adding one column too much to the data, given this line:

    # fill the table
    table = []
    for code, (cid, cgl, eng) in mapping.items():
        row = [cid, cgl, code, eng]
        for language in languages:
            if code in responses[language]:
                row += responses[language][code]
            else:
                row += ["", "", "", "", "", "", ""]
        table += [row]

[LinguList]

So I cannot run norare map right now, but I managed to correct it. I am afraid, however, that the initial tests on correlations are also influenced here!

[LinguList]

My fixes are important now. First, I put English first, both in norare.py and in the metadata-json file. Then, I adjust the language names in norare.py to account for the mappings we selected. Then, I re-ran norare map Dunabeitia-2022-MultiPic. Now, the data looks fine to me, but I noted a general problem with the data in terms of missing items and similar problems. My impression is that the data -- as interesting as it seems -- suffers from some problems, which result in the problems that we observe. Essentially, there does not seem to be any clear signal with respect to elicitation variation (the term I prefer now) per concept.

[LinguList]

This shows that we must clearly and carefully check the data upon each step that we use.

[alzkuc]

There are several problems.

If you do the data download automatically, you obtain the raw data by the authors (I upload the file again here, as it is shared with a free license).

MultiPic Dataset Open Materials.csv

This file contains language names that you do not have in norare.py, yet without matching the filenames, we cannot retrieve them.

Check here:

# define languages we want to keep
languages = OrderedDict({

...
"Dutch": "DUTCH",
"Mandarin": "MANDARIN",
"CypriotGreek": "CGREEK",
...
"Arabic": "ARABIC",
"Malaysian": "MALAYSIAN",
...
"English": "ENGLISH",
"Flemish": "FLEMISH",
})
MultiPic has "Belgian Dutch", not "Flemish", and "Mandarin Chinese", etc. If you run norare map Dunabeitia-2022-MultiPic, it won't map these. They will be empty, yet your data showed them, so I dont' know if you shared the state of the norare.py script that you were using to compile the data.

Dieses tsv. scheint mir neu zu sein. Also da können Unterschiede entstehen. Da sind jetzt auch weitere neue Varietäten mit dabei: Cantonese, Galician...

[LinguList]

Der FEhler mit ENGLISH musste aber auch bei Dir auftauchen, @alzkuc, denn der kann gar nicht von norare.py akzeptiert werden.

Aber der Fehler ist jetzt behoben. Wichtig ist eben, den Download auch zu machen. Wenn da neue Sprachen sind, können wir die erweitern.