migration ignores documents already in DB

[GracefulLemming]

Modifies fetch_sheet to skip documents that already exist in the database. To determine if a document exists, this function uses the query document_id_from_name to compare the short name from a metadata sheet with all short names in the database. If there is a match, it skips the rest of the spreadsheet. Otherwise, it adds the sheet as normal.

This is an imprecise solution meant to facilitate the creation of a new edited collection in the short-term. One major limitation of this solution is its inability to add new words or annotation layers to an existing document. In the future, we will likely want the possibility for more granular comparisons to allow merging of new info in sheets with existing info in the database

[netlify]

✅ Deploy Preview for dailp canceled.

Name	Link
🔨 Latest commit	23f92a0
🔍 Latest deploy log	https://app.netlify.com/sites/dailp/deploys/67c0f2912a36550008cbb060

[nole2701]

Looks good to me. I was wondering though why do you need the 1 second wait on line 212. And when it comes to updating documents that exist but have changed, is it not viable to just overwrite them?

[GracefulLemming]

Looks good to me. I was wondering though why do you need the 1 second wait on line 212. And when it comes to updating documents that exist but have changed, is it not viable to just overwrite them?

@nole2701 Im actually not sure why that wait is there, although I suspect it has to do with Google rate limiting. I will look into this more later!

As for updating existing documents, it is not viable to simply overwrite them entirely.
Consider this example AnnotatedForm in a document:

Source	Romanized	Translation	Comment
ᎤᎨᏩᎴᏓᏃ	ugewaledano	personal name	name of Sequoyah's brother

If we simply rewrote the document, it would delete the word. Then, migration would recreate the word:

Source	Romanized	Translation	Comment
ᎤᎨᏩᎴᏓᏃ	ugewaledano	personal name

Notice that comments do not persist under this model. Neither would attached audio or any changes to syllabary, phonetics, etc. made in the writing environment.

One possible solution is just replace the source, romanization, and translation fields from the spreadsheet when migration runs. However, this would still overwrite contributions in the writing environment which is less than ideal.

[netlify]

✅ Deploy Preview for dailp canceled.

Name	Link
🔨 Latest commit	ee33677
🔍 Latest deploy log	https://app.netlify.com/sites/dailp/deploys/681d11e6d8c3590008d096db