Potential silent failures ?

[julianblue]

Hi @bitner ,
I have been using pypgstac for the stac ingest pipeline for a few weeks now and it drastically reduced system load, which is fantastic !
That said, I noticed a few rare cases where the ingest succeeds but the item never appears in the pgstac db.
Here is an excerpt of the code to publish items:

    # Connect to the database and ingest the item
    db = PgstacDB(dsn=dsn)
    loader = Loader(db=db)

    # Determine the insert mode based on overwrite parameter
    insert_mode = Methods.upsert if overwrite else Methods.insert

    logger.debug(f"Ingesting STAC item {stac_item.id} into collection {selected_collection}")
    loader.load_items([item_dict], insert_mode=insert_mode)
    logger.info(f"STAC item {stac_item.id} ingested successfully")

    # Close the database connection
    db.disconnect()
    db.close()`

We publish single items with each execution, and this issue seemed to happen under a bit of load. Is this linked to partitions as we send single daily items ?

Thanks in advance.