Surface Spark Connect drvier container status

[nabuskey]

Purpose of this PR

This surfaces Spark driver container status to SparkConnect CR. I think it's more user friendly to be able to see the status of the driver container just by looking at the CR's status field.

Example outputs:

No Error

status:
  server:
    driverContainerStatus: 'Running:  Driver container is running at 2025-07-10 23:53:08
      +0000 UTC'
    podName: spark-connect-server
    serviceName: spark-connect-server

With Error*

status:
  server:
    driverContainerStatus: 'Waiting: CrashLoopBackOff Driver container is waiting:
      back-off 1m20s restarting failed container=spark-kubernetes-driver pod=spark-connect-server_default(61a96816-3e03-49fa-9cf7-1aa734cb4076)'
    podName: spark-connect-server
    serviceName: spark-connect-server

Proposed changes:

• add status.server.driverContainerStatus field
• Add basic information about the driver container

Change Category

• Bugfix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that could affect existing functionality)
• Documentation update

Rationale

I was playing with spark connect CR and it was a bit annoying to figure out what was happening to the driver container.

Checklist

• I have conducted a self-review of my own code.
• I have updated documentation accordingly.
• I have added tests that prove my changes are effective or that my feature works.
• Existing unit tests pass locally with my changes.

Additional Notes

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yuchaoran2011 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ChenYi015]

This line is duplicated with line 316.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This pull request has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.