Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

throw NoSuchFeaturesException if no data available for extents (CARD4L) #78

Open
bossie opened this issue Sep 26, 2022 · 1 comment
Open

Comments

@bossie
Copy link
Collaborator

bossie commented Sep 26, 2022

org.openeo.geotrellissentinelhub.BatchProcessingService will throw a NoSuchFeaturesException in case the SHub catalog returns no hits for the specified extents, at least in the non-CARD4L case. This needs to be done for CARD4L too. There's already a test that could use a better name in the form of org.openeo.geotrellissentinelhub.BatchProcessingServiceTest#startCard4LBatchProcessesForXXX.

Don't forget to translate it to an appropriate error code (Open-EO/openeo-geopyspark-driver#224)

@bossie
Copy link
Collaborator Author

bossie commented Dec 12, 2022

This problem manifested itself recently (j-7f841b9888c247929df5f69bfb053c6d) as a misleading error in the async_task handler:

failed to handle polling batch processes for batch job j-7f841b9888c247929df5f69bfb053c6d:
Traceback (most recent call last):
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/async_task.py", line 228, in main
    batch_jobs.poll_sentinelhub_batch_processes(job_info, sentinel_hub_client_alias, vault_token)
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/backend.py", line 993, in poll_sentinelhub_batch_processes
    batch_processes = reduce(partial(dict_merge_recursive, overwrite=True),
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/backend.py", line 994, in <genexpr>
    (batch_request_details(dependency) for dependency in dependencies))
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/backend.py", line 976, in batch_request_details
    [batch_process_dependency['batch_request_id']])
KeyError: 'batch_request_id'

Relevant lines in openeo-geopyspark-driver:
https://github.com/Open-EO/openeo-geopyspark-driver/blob/577be55c90fd1e948ebd2a338092cd46f9fba5bb/openeogeotrellis/backend.py#L994-L995

Earlier logs:

saved newly scheduled CARD4L batch processes [] for near future use (key ('SENTINEL1_GRD', (3.6675152618923206, 50.763255632654214, 3.838210383127976, 50.903414110950855), 'EPSG:4326', '2014-11-18T00:00:00+00:00', '2014-12-31T00:00:00+00:00', ('VV', 'dataMask', 'localIncidenceAngle'), 'COPERNICUS_30', (('sar:instrument_mode', (('eq', 'IW'),)), ('sat:orbit_state', (('eq', 'ASCENDING'),)))))

Indeed, the list of batch request IDs in the job's dependencies is empty:

[
  {
    "collection_id": "SENTINEL1_GRD",
    "batch_request_ids": [],
    "results_location": "s3://openeo-sentinelhub/10226947-dda5-416c-9d7d-e0aaead7cce2",
    "card4l": true
  }
]

An empty list is falsy, resulting in the KeyError. This can be averted by throwing a NoSuchFeaturesException earlier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant