add jira-ticket parameter to paasta spark-run

[sidsatyelp]

What this change does

In order to more accurately attribute the costs of feature development to infrastructure costs we are making a change to how we submit adhoc spark jobs. Users will need to pass an additional parameter --jira-ticket in paasta spark-run. The value of this parameter should be the top level jira ticket being used to track your project. This will allow us to aggregate the entire project's spark development costs over time.

Will work only after the flag spark.yelp.jira_ticket.enabled is set to true in srv-configs and the new version with these changes are included in paasta and spark-tools. Before we enable this we will communicate about this to spark users at Yelp.

Related PRs and documentation

https://github.yelpcorp.com/sysgit/srv-configs/pull/44570
Yelp/service_configuration_lib#161
New release of service_configuration_lib being used here.
https://github.yelpcorp.com/python-packages/spark_tools/pull/136
https://yelpwiki.yelpcorp.com/spaces/AML/pages/402885641/--jira-ticket+in+spark-run

Testing

# Shouldn't launch because we don't provide jira_ticket
paasta spark-run --aws-profile=dev-cloud-economics --cmd "spark-submit /code/integration_tests/s3.py"
# output: https://fluffy.yelpcorp.com/i/XDdDpJB7vrXLvcqVNkLBxvBdtTFl44RX.html

# Should launch because we provide jira_ticket
paasta spark-run --aws-profile=dev-cloud-economics --jira-ticket=ABC-123 --cmd "spark-submit /code/integration_tests/s3.py"
# output: https://fluffy.yelpcorp.com/i/1VCTvzjk0wK6mbC97KF8JPRqt2WmsXTd.html

jira_ticket label is being added to executors
https://app.cloudzero.com/explorer?activeCostType=real_cost&granularity=hourly&partitions=k8slabel%3Aspark_yelp_com_jira_ticket&dateRange=Last%2024%20Hours&k8slabel%3Aspark_yelp_com_user=sids&showRightFlyout=filters

Tech Spec
Jira Ticket

[nemacysts]

from past experience with blocking workflows that require tickets, i'm a little skeptical of the data quality we'll get from this over something coarser (but more automatic) like team-based allocation based on the launching user, but i'm happy to be proven wrong :)

(note: ML Compute owns spark-run in its entirety - you don't need CI shipit for this PR since this doesn't touch any CODEOWNER'd files)

[nemacysts]

probably worth replacing this with a y/ link :)