type key is missing for prediction_pb2.Metric with type COUNTER

[tandav]

Describe the bug

A SeldonMessage with metrics of type COUNTER is missing metric type key, unlike other metric types (GAUGE, TIMER)

To reproduce

1. Let's create a SeldonMessage with a metric of type COUNTER:

from seldon_core.proto import prediction_pb2
from seldon_core.metrics import validate_metrics
from seldon_core.utils import seldon_message_to_json

msg = prediction_pb2.SeldonMessage()
msg.meta.metrics.extend([
    prediction_pb2.Metric(type='COUNTER', key='mycounter', value=1),
])

Let's see whats inside the metrics. A type: COUNTER field is missing and validate_metrics function says metrics are invalid:

>>> msg.meta.metrics
[key: "mycounter"
value: 1.0
]
>>> validate_metrics(seldon_message_to_json(msg)['meta']['metrics'])
False

This is different from GAUGE and TIMER metrics, which have type key and passes validate_metrics test:

msg = prediction_pb2.SeldonMessage()
msg.meta.metrics.extend([
    prediction_pb2.Metric(type='GAUGE', key='mygauge', value=22.0),
])
>>> msg.meta.metrics
[key: "mygauge"
type: GAUGE
value: 22.0
]
>>> validate_metrics(seldon_message_to_json(msg)['meta']['metrics'])
True

msg = prediction_pb2.SeldonMessage()
msg.meta.metrics.extend([
    prediction_pb2.Metric(type='TIMER', key='mytimer', value=1),
])
>>> msg.meta.metrics
[key: "mytimer"
type: TIMER
value: 1.0
]
>>> validate_metrics(seldon_message_to_json(msg)['meta']['metrics'])
True

Expected behaviour

prediction_pb2.Metric should not drop type key for COUNTER metrics.

context

I've encountered this problem when fixing custom metric logic in PR #4863. I'm trying to call client_custom_metrics for _raw methods to update seldon_metrics and concatenate custom metrics from msg (raw methods response) with user_model.metrics() (if implemented) and I get metrics validation errors for UserObjectLowLevelGrpc in test_metrics. Currently client_custom_metrics is not called for _raw methods and validation passes