DC1 Round 1

[lliming]

• East publishes
• West publishes
• Inspect event stream and compare catalogue contents

[lukaszlacinski]

Both West and East published 500 Items to their STAC Transaction APIs. In effect, it should be 1,000 Items in the West Search Index. There is only 999 Items because one Item in the Items published by West and East is duplicated, https://github.com/ESGF/esgf-roadmap/blob/ska-add-paths/data-challenge-docs/West-CMIP6-paths.txt#L1.

[lliming]

Identified a few issues.

1. The testing datasets used by East and West had one overlap, so there were only 999 unique datasets instead of 1000. It appears that both catalogs caught the duplicate and it only appears once in each catalog.
2. At least one metadata field had capitalization inconsistencies due to a limitation in CEDA's ElasticSearch implementation.
3. The assets in all datasets appear inconsistently when viewed in the discovery API. This is caused by a bug in Globus's Globus Search backend and the way it restructures assets in the index (and restores the original structure when retrieving the item in the discovery API).
4. The West (Globus) event consumer crashed after running a long time due to mishandling an expired access token.

[sturoscy-personal]

This is more or less completed. Need to write up results #45