Add support to configure multiple service ports

[shivamerla]

• Default named port of "api" will be added with legacy .spec.port attribute
• Ability to setup multiple named ports "api", "metrics" etc.
• auto detection of metrics and api ports for probes and metrics collection

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[varunrsekar]

we can probably use rendertypes.ServiceParams as input here and simplify this to always prefer "api" namedPort from the ServiceParams.Ports list

[shivamerla]

Each resource is rendered independently during reconciliation. That will add a dependency that we always have to render serviceParams before setting up deployment or servicemonitor.

[shengnuo]

This implies that .spec.expose.service is required, but it is not currently.
Maybe we should make it a required field or add a default value?

[shivamerla]

yes, we can make this as a required field, earlier there was just single port param and default for that, so it was always present.

[varunrsekar]

case where length is 1 and default case where length > 1 should have similar behavior:

• Prefer "api"
• Otherwise, pick first port
  Isn't this the case? Defaulting to "api" when other options exist doesnt seem right to me.

Suggested change

	switch len(serviceSpec.Ports) {
	case 1:
	port := serviceSpec.Ports[0]
	if port.Name != "" {
	return intstr.FromString(port.Name)
	}
	return intstr.FromInt(int(port.Port))
	case 0:
	// Default to "api" as the operator always adds a default named port with 8000
	return intstr.FromString("api")
	default:
	// Multiple ports: Prefer "api"
	if portName := selectNamedPort(serviceSpec, "api"); portName != "" {
	return intstr.FromString(portName)
	}
	// Default when multiple ports exist
	return intstr.FromString("api")
	}
	}
	switch len(serviceSpec.Ports) {
	case 0:
	// Default to "api" as the operator always adds a default named port with 8000
	return intstr.FromString("api")
	default:
	// Multiple ports: Prefer "api"
	if portName := selectNamedPort(serviceSpec, "api"); portName != "" {
	return intstr.FromString(portName)
	}
	// Default when multiple ports exist
	port := serviceSpec.Ports[0]
	if port.Name != "" {
	return intstr.FromString(port.Name)
	}
	return intstr.FromInt(int(port.Port))
	}
	}

[shivamerla]

We just can't assume to use the first port when multiple ports are given. For e.g. in case of Customizer, if "api" and "internal" ports are used with different names, we can't assume "internal" ports for liveness/startup probes etc. Hence i was always enforcing that a named port of "api" should be specified in that case. Probably better things is to validate this during reconciliation and error out when required named ports are not provided.

[varunrsekar]

similarly here: Shouldn't we be getting the first port from the list if "metrics" and "api" dont exist?

[shivamerla]

same logic as above, in documentation we have to specify that named ports for "api" or "metrics" should be provided when they are served differently by the underlying service. Randomly assigning would not work.

[shengnuo]

trying to understand the logic.. in this case, why it shouldn't produce api like the test case in L63?

[shivamerla]

when say user provides input as below:

  expose:
    service:
      type: ClusterIP
      ports:
        - port: 8000
           protocol: TCP

then, in the deployment spec, we don't add any named ports like "api". Hence we use the port number that is given to us. we will add a default named port ("api: 8000" ) when no ports are given in the config at all.