Skip to content

Add mutli-team dimensions to provider metrics #68996

Description

@ferruzzi

Follow-up to #68294 which added team name to all relevant core metrics, this one is for mob-coding the providers.

Provider Assignee Status PR
amazon @justinpakzad ✅ #69072
celery @SameerMesiah97 ✅ #69092
edge3 worker @shivaam ✅ #69149
edge3 execurtor @shivaam ✅ #69005
k8s @SameerMesiah97 ✅ #69046
openlineage @SameerMesiah97 ✅ #69109

Assigned: 👀
PR: 🚧
Done: ✅


Provider metrics to be updated:

amazon (3):

  • batch_executor.adopt_task_instances.duration
  • ecs_executor.adopt_task_instances.duration
  • lambda_executor.adopt_task_instances.duration

celery (2):

  • celery.task_timeout_error
  • celery.execute_command.failure

edge3 worker (10):

  • edge_worker.status
  • edge_worker.connected
  • edge_worker.maintenance
  • edge_worker.jobs_active
  • edge_worker.concurrency
  • edge_worker.free_concurrency
  • edge_worker.num_queues
  • edge_worker.heartbeat_count
  • edge_worker.ti.start
  • edge_worker.ti.finish

edge3 executor (1):

  • edge_executor.sync.duration

kubernetes (8):

  • kubernetes_executor.pod_creation_status
  • kubernetes_executor.pod_deletion_status
  • kubernetes_executor.pod_patching_status
  • kubernetes_executor.clear_not_launched_queued_tasks.duration
  • kubernetes_executor.adopt_task_instances.duration
  • kubernetes_executor.pod_creation
  • kubernetes_executor.pod_deletion
  • kubernetes_executor.pod_patching

openlineage (4):

  • ol.emit.failed
  • ol.event.size
  • ol.emit.attempts
  • ol.extract

Prior work examples:

Assets: #68367
Deadlines: #68589
Executors: #68593
Scheduler: #68594
Dag Processor: #68599

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions