Pipeline Error Ingestion pipeline raised an unexpected exception! <class 'KeyError'>: 'platformResource'

[IALASIA24]

Hello,

We are currently using Kubernetes with a stable setup, utilizing chart datahub-0.4.24 and app version v0.14.0.2, along with the prerequisite chart datahub-prerequisites-0.0.18. However, after upgrading to chart datahub-0.5.5, app version v0.15.0, and the prerequisite chart datahub-prerequisites-0.1.7, we began encountering this type of error with our recipes.

We try it with differents CLI version, 0.15.0, 0.14.0

I will share some logs with you:

Execution finished with errors.
{'exec_id': '21d857b2-3927-4033-bc11-14846b56c5fa',
 'infos': ['2025-02-19 03:00:00.132522 INFO: Starting execution for task with name=RUN_INGEST',
           "2025-02-19 05:56:38.482273 INFO: Failed to execute 'datahub ingest', exit code 1",
           '2025-02-19 05:56:38.483332 INFO: Caught exception EXECUTING task_id=21d857b2-3927-4033-bc11-16b56c5fa, name=RUN_INGEST, '
           'stacktrace=Traceback (most recent call last):\n'
           '  File "/datahub-ingestion/.venv/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 140, in execute_task\n'
           '    task_event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
           '    return future.result()\n'
           '  File "/datahub-ingestion/.venv/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 282, in '
           'execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
 'errors': []}

~~~~ Ingestion Report ~~~~
{
  "cli": {
    "cli_version": "0.14.0",
-----------------------------------------------------------
      "lineage_start_time": "2025-02-18 15:05:12.588000+00:00 (14 hours, 51 minutes and 16.23 seconds ago)",
      "lineage_end_time": "2025-02-19 03:02:10.810536+00:00 (2 hours, 54 minutes and 18.01 seconds ago)",
      "stateful_lineage_ingestion_enabled": true,
      "usage_start_time": "2025-02-18 00:00:00+00:00 (1 day, 5 hours and 56 minutes ago)",
      "usage_end_time": "2025-02-19 03:02:10.810536+00:00 (2 hours, 54 minutes and 18.01 seconds ago)",
      "stateful_usage_ingestion_enabled": true,
      "start_time": "2025-02-19 03:02:10.844329 (2 hours, 54 minutes and 17.97 seconds ago)",
      "running_time": "2 hours, 54 minutes and 17.96 seconds",
      "failures": [
        {
          "title": "Pipeline Error",
          "message": "Ingestion pipeline raised an unexpected exception!",
          "context": [
            "<class 'KeyError'>: 'platformResource'"
          ]
        }
      ],
      "warnings": [],
      "infos": []


---------------------------------------------------------
~~~~ Ingestion Logs ~~~~
[earlier logs truncated...]
 'profile_table_selection_criteria': {},
 'selected_profile_tables': {},
 'profiling_skipped_invalid_partition_ids': {},
 'profiling_skipped_invalid_partition_type': {},
 'profiling_skipped_partition_profiling_disabled': [],
 'num_usage_workunits_emitted': 0,
 'total_query_log_entries': 0,
 'num_read_events': 183395,
 'num_query_events': 4032560,
 'num_view_query_events': 14225,
 'num_view_query_events_failed_sql_parsing': 10294,
 'num_view_query_events_failed_table_identification': 565,
 'num_filtered_read_events': 2090890,
 'num_filtered_query_events': 0,
 'num_usage_query_hash_collisions': 0,
 'num_operational_stats_workunits_emitted': 3997,
 'snapshots_scanned': 0,
 'read_reasons_stat': {'JOB': 183395},
 'operation_types_stat': {'CREATE_TABLE_AS_SELECT': 3171,
                          'SELECT': 1454,
                          'MERGE': 8172,
                          'UPDATE': 1154,
                          'DELETE': 2358,
                          'INSERT': 2774,
                          'EXPORT_DATA': 9,
                          'CREATE_VIEW': 2,
                          'CREATE_TABLE': 94,
                          'ALTER_TABLE': 5,
                          'TRUNCATE_TABLE': 188},

I would greatly appreciate any assistance or guidance on how to resolve this issue.

Thank you in advance!