Knative Serving release v0.7.0

Meta

serving.knative.dev/v1beta1 (requires K8s 1.14+ due to #4533)

• In 0.6 we expanded our v1alpha1 API to include our v1beta1 fields. In this release, we are contracting the set of fields we store for v1alpha1 to that subset (and disallowing those that don’t fit). With this, we can leverage the “same schema” CRD-conversion supported by Kubernetes 1.11+ to ship v1beta1.

HPA-based scaling on concurrent requests

• We previously supported using the HPA “class” autoscaler to enable Knative services to be scaled on CPU and Memory. In this release, we are adding support for using the HPA to scale them on the same “concurrent requests” metrics used by our default autoscaler.
• HPA still does not yet support scaling to zero, and more work is needed to expose these metrics to arbitrary autoscaler plugins, but this is exciting progress!

Non-root containers

• This release, all of the containers we ship run as a “nonroot” user. This includes the queue-proxy sidecar injected into the user pod. This enables the use of stricter “Pod Security Policies” with knative/serving.

Breaking Changes

• Previously deprecated status fields are no longer populated.
• Build and Manual (deprecated in 0.6) are now unsupported
• The URLs generated for Route tags by default have changed, see the tagTemplate section below for how to avoid this break.

Autoscaling

Support concurrency-based scaling on the HPA (thanks @markusthoemmes).

Metric-scraping and decision-making has been separated out of the Knative internal autoscaler (KPA). The metrics are now also available to the HPA.

Dynamically change autoscaling metrics sample size based on pod population (thanks @yanweiguo).

Depending on how many pods the specific revision has, the autoscaler now scrapes a computed number of pods to gain more confidence in the reported metrics while maintaining scalability.

Fixes:

• Added readiness probes to the autoscaler #4456 (thanks @vagababov)
• Adjust activator’s throttling behavior based on activator scale (thanks @shashwathi and @andrew-su).
• Revisions wait until they have reached “minScale” before they are reported “Ready” (thanks @joshrider).

Core API

Expose v1beta1 API #4199 (thanks @mattmoor)

This release exposes resources under serving.knative.dev/v1beta1.

Non-root containers #3237 (thanks @bradhoekstra and @dprotaso)

This release, all of the containers we ship run as a “nonroot” user. This includes the queue-proxy sidecar injected into the user pod. This enables the use of stricter “Pod Security Policies” with knative/serving.

Allow users to specify their container name #4289 (thanks @mattmoor)

This will default to user-container, which is what we use today, and that default may be changed for config-defaults to a Go template with access to the parent resource’s (e.g. Service, Configuration) ObjectMeta fields.

Projected volume support #4079 (thanks @mattmoor)

Based on community feedback, we have added support for mounting ConfigMaps and Secrets via the projected volume type.

Drop legacy status fields #4197 (thanks @mattmoor)

A variety of legacy fields from our v1alpha1 have been dropped in preparation to serve these same objects over v1beta1.

Build is unsupported #4099 (thanks @mattmoor)

As mentioned in the 0.6 release notes, support for just-in-time builds has been removed, and requests containing a build will now be rejected.

Manual is unsupported #4188 (thanks @mattmoor)

As mentioned in the 0.6 release notes, support for manual mode has been removed, and requests containing it will now be rejected.

V1beta1 clients and conformance testing #4369 (thanks @mattmoor)

We have generated client libraries for v1beta1 and have a v1beta1 version of the API conformance test suite under ./test/conformance/api/v1beta1.

Defaulting based conversion #4080 (thanks @mattmoor)

Objects submitted with the old v1alpha1 schema will be upgraded via our “defaulting” logic in a mutating admission webhook.

New annotations for queue-proxy resource limits #4151 (thanks @raushan2016)

The queue.sidecar.serving.knative.dev/resourcePercentage annotation now allows setting the percetnage of user container resources to be used for the queue-proxy.

Annotation propagation #4363, #4367 (thanks @vagababov)

Annotations now propagate from the Knative Service object to Route and Configuration.

Fixes:

• Improve our Ready/Generation handling across resources #4185 (thanks @mattmoor)
• Fix Revision GC #4187, #4245 (thanks @nak3, @greghaynes)
• Surface pod schedule errors in Revision #4191 (thanks @shashwathi)
• Allow container.name in RevisionTemplate #4289 (thanks @mattmoor)
• Fix pulling older schema 1 manifests #4430 (thanks @jonjohnsonjr)

Test:

• Add multiple namespace test #4108 (thanks @andrew-su)
• Separate Conformance tests by type #4145 (thanks @tzununbekov)
• Upgrade test improvements #4211, #4267 (thanks @jonjohnsonjr)
• Add conformance test case for user set headers #4411 (thanks @dgerd)

Networking

Reconcile annotations from Route to ClusterIngress #4087 (thanks @vagababov)

This allows ClusterIngress class annotation to be specified per-Route instead of cluster wide through a config-network setting.

Introduce tagTemplate configuration #4292 (thanks @mattmoor)

This allows operators to configure the names that are given to the services created for tags in Route.
This also changes the default to transpose the tag and route name, which is a breaking change to the URLs these received in 0.6. To avoid this break, you can set tagTemplate: {{.Name}}-{{.Tag}} in config-network.

Enable use of annotations in domainTemplate #4210 (thanks @raushan2016)

User can now provide custom subdomain via label serving.knative.dev/subDomain.

Allow customizing max allowed request timeout #4172 (thanks @mdemirhan)

This introduces a new config entry max-revision-timeout-seconds in config-defaults to set the max allowed request timeout.

Set Forwarded header on request #4376 (thanks @tanzeeb)

The Forwarded header is constructed and appended to the headers by the queue-proxy if only legacy x-forwarded-* headers are set.

Fixes:

• Enable short names for cluster-local Service without relying on sidecars #3824 (thanks @tcnghia)
• Better surfacing of ClusterIngress Status #4288 #4144 (thanks @tanzeeb, @nak3)
• SKS private service uses random names to avoid length limitation #4250 (thanks @vagababov)

Monitoring

Set memory request for zipkin pods #4353 (thanks @sebgoa)

This lowers the memory necessary to schedule the zipkin pod.

Collect /var/log without fluentd sidecar #4156 (thanks @JRBANCEL)

This allows /var/log collection without the need to load fluentd sidecar, which is large and significantly increases pod startup time.

Enable queue-proxy metrics scraping by Prometheus. #4111 (thanks @mdemirhan)

The new metrics exposed by queue proxy are now exposed as part of the pod spec and Prometheus can now scrape these metrics.

Fixes:

• Fix 'Revision CPU and Memory Usage' Grafana dashboard #4106 (thanks @JRBANCEL)
• Fix 'Scaling Debugging' Grafana dashboard. #4096 (thanks @JRBANCEL)
• Remove embedded jaeger-operator and include as dependency instead #3938 (thanks @objectiser)
• Fix HTTP request dashboards #4418 (thanks @mdemirhan)

Knative Serving release v0.6.1

Meta

New API Shape

We have approved a proposal for the “v1beta1” API shape for knative/serving. These changes will make the Serving resources much more familiar for experienced Kubernetes users, unlock the power of Route to users of Service, and enable GitOps scenarios with features like “bring-your-own-Revision-name”. We will be working towards this over the next few releases.

In this release we have backported the new API surface to the v1alpha1 API as the first part of the transition to v1beta1 (aka “lemonade”). The changes that will become breaking in 0.7+ are:

• Service and Configuration will no longer support “just-in-time” Builds.
• Service will no longer support “manual” mode.

You can see the new API surface in use throughout our samples in knative/docs, but we will continue to support the majority of the legacy surface via v1alpha1 until we turn it down.

Overhauled Scale-to-Zero

We have radically changed the mechanism by which we scale to zero. The new architecture creates a better separation of concerns throughout the Serving resource model with fewer moving parts, and enables us to address a number of long-standing issues (some in this release, some to come). See below for more details.

Auto-TLS (alpha, opt-in)

We have added support for auto-TLS integration! The default implementation builds on cert-manager to provision certificates (e.g. via Let’s Encrypt), but similar to how we have made Istio pluggable, you can swap out cert-manager for other certificate provisioning systems. Currently certificates are provisioned per-Route, but stay tuned for wildcard support in a future release. This feature requires Istio 1.1, and must be explicitly enabled.

Moar Controller Decoupling

We have started to split the “pluggable” controllers in Knative into their own controller processes so that folks looking to replace Knative sub-systems can more readily remove the bundled default implementation. For example, to install Knative Serving without the Istio layer run:

kubectl apply -f serving.yaml \
  -l networking.knative.dev/ingress-provider!=istio

Note that we may see some error due to kubectl not understanding the yaml for Istio objects (even if they are filtered out by the label selector). It is safe to ignore the errors no matches for kind "Gateway" in version "networking.istio.io/v1alpha3".

You can also use this to omit the optional Auto-TLS controller based on cert-manager with:

kubectl apply -f serving.yaml \
  -l networking.knative.dev/certificate-provider!=cert-manager

Autoscaling

Move the Knative PodAutoscaler (aka “KPA”) from the /scale sub-resource for scaling to a PodScalable “duck type”. This enables us to leverage informer caching, and the expanded contract will enable the ServerlessService (aka “SKS”) to leverage the PodSpec to do neat optimizations in future releases. (Thanks @mattmoor)

We now ensure that our “activator” component has been successfully wired in before scaling a Revision down to zero (aka “positive hand-off”, #2949). This work was enabled by the Revision-managed activation work below. (Thanks @vagababov)

New annotations autoscaling.knative.dev/window, autoscaling.knative.dev/panicWindowPercentage, and autoscaling.knative.dev/panicThresholdPercentage allow customizing the sensitivity of KPA-class PodAutoscalers (#3103). (Thanks @josephburnett)

Added tracing to activator to get more detailed and persistently measured performance data (#2726). This fixes #1276 and will enable us to troubleshoot performance issues, such as cold start. (Thanks @greghaynes).

Fixed a Scale to Zero issue with Istio 1.1 lean installation (#3987) by reducing the idle timeouts in default transports (#3996) (Thanks @vagababov) which solves the k8's service not being terminated when the endpoint changes.

Resolved an issue which prevented disabling Scale to Zero (#3629) with fix (#3688) (Thanks @yanweiguo) which takes enable-scale-to-zero from configmap into account in KPA reconciler when doing scale. If minScale annotation is not set or set to 0 and enable-scale-to-zero is set to false, keep 1 pod as minimum.

Fix the autoscaler bug that make rash decision when the autoscaler restarts (#3771). This fixes issues #2705 and #2859. (Thanks @hohaichi)

Core API

We have an approved v1beta1 API shape! As above, we have started down the path to v1beta1 over the next several milestones. This milestone landed the v1beta1 API surface as a supported subset of v1alpha1. See above for more details. (Thanks to the v1beta1 task force for many hours of hard work on this).

We changed the way we perform validation to be based on a “fieldmask” of supported fields. We will now create a copy of each Kubernetes object limited to the fields we support, and then compare it against the original object; this ensures we are deliberate with which resource fields we want to leverage as the Kubernetes API evolves. (#3424, #3779) (Thanks @dgerd). This was extended to cleanup our internal API validations (#3789, #3911) (Thanks @mattmoor).

status.domain has been deprecated in favor of status.url. (#3970) (Thanks @mattmoor) which uses the apis.URL for our URL status fields, resolving the issue "Unable to get the service URL" (#1590)

Added the ability to specify default values for the matrix of {cpu, mem} x {request, limit} via our configmap for defaults. This also removes the previous CPU limit default so that we fallback on the configured Kubernetes defaults unless this is specifically specified by the operator. (#3550, #3912) (Thanks @mattmoor)

Dropped the use of the configurationMetadataGeneration label (#4012) (thanks @dprotaso), and wrapped up the last of the changes transitioning us to CRD sub-resources (#643).

Networking

Overhauled the way we scale-to-zero! (Thanks @vagababov) This enables us to have Revisions managing their own activation semantics, implement positive hand-off when scaling to zero, and increase the autoscaling controller’s resync period to be consistent with our other controllers.

Added support for automatically configuring TLS certificates! (Thanks @ZhiminXiang) See above for more details.

We have stopped releasing Istio yamls. It was never our intention for knative/serving to redistribute Istio, and prior releases exposed our “dev”-optimized Istio yamls. Users should consult either the Istio or vendor-specific documentation for how to get a “supported” Istio distribution. (Thanks @mattmoor)

We have started to adopt a flat naming scheme for the named sub-routes within a Service or Route. The old URLs will still work for now, but the new URLs will appear in the status.traffic[*].url fields. (Thanks @andrew-su)

Support the installation of Istio 1.1 (#3515, #3353) (Thanks @tcnghia)

Fixed readiness probes with Istio mTLS enabled (#4017) (Thanks @mattmoor)

Monitoring

Activator now reports request logs (#3781) with check-in (#3927) (Thanks @mdemirhan)

Test and Release

Assorted Fixes

• label serving.knative.dev/release: devel should have the release name/number instead of devel (#3626) fixed with Export TAG to fix our annotation manipulation. (#3995) (Thanks @mattmoor)

• Always install istio from HEAD for upgrade tests (#3522) (Thanks @jonjohnsonjr) fixing errors with upgrade / downgrade testing of knative (#3506)

• Additional runtime conformance test coverage (9 new tests), improvements to existing conformance tests, and v1beta1 coverage. (Thanks @andrew-su, @dgerd, @yt3liu, @mattmoor, @tzununbekov)

Knative Serving release v0.6.0

Meta

New API Shape

We have approved a proposal for the “v1beta1” API shape for knative/serving. These changes will make the Serving resources much more familiar for experienced Kubernetes users, unlock the power of Route to users of Service, and enable GitOps scenarios with features like “bring-your-own-Revision-name”. We will be working towards this over the next few releases.

In this release we have backported the new API surface to the v1alpha1 API as the first part of the transition to v1beta1 (aka “lemonade”). The changes that will become breaking in 0.7+ are:

• Service and Configuration will no longer support “just-in-time” Builds.
• Service will no longer support “manual” mode.

You can see the new API surface in use throughout our samples in knative/docs, but we will continue to support the majority of the legacy surface via v1alpha1 until we turn it down.

Overhauled Scale-to-Zero

We have radically changed the mechanism by which we scale to zero. The new architecture creates a better separation of concerns throughout the Serving resource model with fewer moving parts, and enables us to address a number of long-standing issues (some in this release, some to come). See below for more details.

Auto-TLS (alpha, opt-in)

We have added support for auto-TLS integration! The default implementation builds on cert-manager to provision certificates (e.g. via Let’s Encrypt), but similar to how we have made Istio pluggable, you can swap out cert-manager for other certificate provisioning systems. Currently certificates are provisioned per-Route, but stay tuned for wildcard support in a future release. This feature requires Istio 1.1, and must be explicitly enabled.

Moar Controller Decoupling

We have started to split the “pluggable” controllers in Knative into their own controller processes so that folks looking to replace Knative sub-systems can more readily remove the bundled default implementation. For example, to install Knative Serving without the Istio layer run:

kubectl apply -f serving.yaml \
  -l networking.knative.dev/ingress-provider!=istio

Note that we may see some error due to kubectl not understanding the yaml for Istio objects (even if they are filtered out by the label selector). It is safe to ignore the errors no matches for kind "Gateway" in version "networking.istio.io/v1alpha3".

You can also use this to omit the optional Auto-TLS controller based on cert-manager with:

kubectl apply -f serving.yaml \
  -l networking.knative.dev/certificate-provider!=cert-manager

Autoscaling

Move the Knative PodAutoscaler (aka “KPA”) from the /scale sub-resource for scaling to a PodScalable “duck type”. This enables us to leverage informer caching, and the expanded contract will enable the ServerlessService (aka “SKS”) to leverage the PodSpec to do neat optimizations in future releases. (Thanks @mattmoor)

We now ensure that our “activator” component has been successfully wired in before scaling a Revision down to zero (aka “positive hand-off”, #2949). This work was enabled by the Revision-managed activation work below. (Thanks @vagababov)

New annotations autoscaling.knative.dev/window, autoscaling.knative.dev/panicWindowPercentage, and autoscaling.knative.dev/panicThresholdPercentage allow customizing the sensitivity of KPA-class PodAutoscalers (#3103). (Thanks @josephburnett)

Added tracing to activator to get more detailed and persistently measured performance data (#2726). This fixes #1276 and will enable us to troubleshoot performance issues, such as cold start. (Thanks @greghaynes).

Fixed a Scale to Zero issue with Istio 1.1 lean installation (#3987) by reducing the idle timeouts in default transports (#3996) (Thanks @vagababov) which solves the k8's service not being terminated when the endpoint changes.

Resolved an issue which prevented disabling Scale to Zero (#3629) with fix (#3688) (Thanks @yanweiguo) which takes enable-scale-to-zero from configmap into account in KPA reconciler when doing scale. If minScale annotation is not set or set to 0 and enable-scale-to-zero is set to false, keep 1 pod as minimum.

Fix the autoscaler bug that make rash decision when the autoscaler restarts (#3771). This fixes issues #2705 and #2859. (Thanks @hohaichi)

Core API

We have an approved v1beta1 API shape! As above, we have started down the path to v1beta1 over the next several milestones. This milestone landed the v1beta1 API surface as a supported subset of v1alpha1. See above for more details. (Thanks to the v1beta1 task force for many hours of hard work on this).

We changed the way we perform validation to be based on a “fieldmask” of supported fields. We will now create a copy of each Kubernetes object limited to the fields we support, and then compare it against the original object; this ensures we are deliberate with which resource fields we want to leverage as the Kubernetes API evolves. (#3424, #3779) (Thanks @dgerd). This was extended to cleanup our internal API validations (#3789, #3911) (Thanks @mattmoor).

status.domain has been deprecated in favor of status.url. (#3970) (Thanks @mattmoor) which uses the apis.URL for our URL status fields, resolving the issue "Unable to get the service URL" (#1590)

Added the ability to specify default values for the matrix of {cpu, mem} x {request, limit} via our configmap for defaults. This also removes the previous CPU limit default so that we fallback on the configured Kubernetes defaults unless this is specifically specified by the operator. (#3550, #3912) (Thanks @mattmoor)

Dropped the use of the configurationMetadataGeneration label (#4012) (thanks @dprotaso), and wrapped up the last of the changes transitioning us to CRD sub-resources (#643).

Networking

Overhauled the way we scale-to-zero! (Thanks @vagababov) This enables us to have Revisions managing their own activation semantics, implement positive hand-off when scaling to zero, and increase the autoscaling controller’s resync period to be consistent with our other controllers.

Added support for automatically configuring TLS certificates! (Thanks @ZhiminXiang) See above for more details.

We have stopped releasing Istio yamls. It was never our intention for knative/serving to redistribute Istio, and prior releases exposed our “dev”-optimized Istio yamls. Users should consult either the Istio or vendor-specific documentation for how to get a “supported” Istio distribution. (Thanks @mattmoor)

We have started to adopt a flat naming scheme for the named sub-routes within a Service or Route. The old URLs will still work for now, but the new URLs will appear in the status.traffic[*].url fields. (Thanks @andrew-su)

Support the installation of Istio 1.1 (#3515, #3353) (Thanks @tcnghia)

Fixed readiness probes with Istio mTLS enabled (#4017) (Thanks @mattmoor)

Monitoring

Activator now reports request logs (#3781) with check-in (#3927) (Thanks @mdemirhan)

Test and Release

Assorted Fixes

• label serving.knative.dev/release: devel should have the release name/number instead of devel (#3626) fixed with Export TAG to fix our annotation manipulation. (#3995) (Thanks @mattmoor)

• Always install istio from HEAD for upgrade tests (#3522) (Thanks @jonjohnsonjr) fixing errors with upgrade / downgrade testing of knative (#3506)

• Additional runtime conformance test coverage (9 new tests), improvements to existing conformance tests, and v1beta1 coverage. (Thanks @andrew-su, @dgerd, @yt3liu, @mattmoor, @tzununbekov)

Knative Serving release v0.5.2

Autoscaling

Bucketize autoscaling metrics by timeframe (#3289 - thanks @markusthoemmes)

Stats are averaged in each specific timeframe vs. averaged over the whole window. Fixes #2977.

Prevent double counting by the autoscaler (#3477 - thanks @hohaichi)

Let the queue report a concurrency metric for proxied requests, and the autoscaler discount this concurrency when making scaling decisions. Fixes bug (#3301).

Scrape queue-proxy metrics in autoscaler (#3149, thanks @yanweiguo)

Scrape queue-proxy metrics in autoscaler instead of pushing metrics from queue-proxy to autoscaler via websocket connection. Remove the websocket usage in queue-proxy. Fixes bugs #2203, #1927.

Bug Fixes / Cleanups

• Doc update for scale annotations lifetime (#3461 - thanks @yanweiguo) resolving bug (#2720)
• Surface events for all reconciliation failures (#3462 - thanks @mattmoor)---fixes #2941.
• Only add KPA label to a K8S service of a revision if KPA is used (#3498 - thanks @yanweiguo)---fixes #2606 (HPA and KPA controllers are getting wrong class PAs)

Core API

Surface events on reconciliation failures (thanks @mattmoor)

Changes boilerplate to surface reconciliation failures in addition to existing logging entries already in place. (#3462) which fixes bug (#2941).

Several Revision default values are now configurable (thanks @mattmoor)

A new ConfigMap in knative-serving called config-defaults enables the customization of the default CPU resource request and request timeout (#3546).

Add a url: to Service/Route status (thanks @dgerd)

A new url: field in the status.traffic block of Service and Route is displayed to enable discovery of the generated URL for name: sub-routes (#3526). This field is only present when name: is used for a traffic target. See the url: field for Route and Service in spec.md

More conformance tests:

• Validating user securityContext is reflected in container. (thanks @dgerd)
  • This change also adds the group information to the runtime test image, but we do not validate this as 1. it is not currently part of the runtime contract and 2. Setting group is currently an alpha feature that does not work on many Kubernetes clusters. See kubernetes/enhancements#213 - (#3423)
• Validating that metadata.generateName can be used instead of metadata.name (thanks @brandone-mabey)
  • Add testing to ensure that metadata.generateName can be used instead of metadata.name to specify names for services, routes, and configurations. (#3292) fixing (#3183)

Bug Fixes / Cleanups:

• CPU requests is now defaulted in the webhook (thanks @mattmoor) (#3529)
• Fixed broken deploy.sh **(thanks @adrchunha) **(#3367)
• Dropped functional dependency on 'configurationMetadaraGeneration' Label **(thanks @pivotal-joshua-rider) **(#3325)

Networking

Define 'Certificate' CRD for Auto TLS (thanks @ZhiminXiang)

As part of work in #1964, define an internal CRD Certificate to make TLS provision more pluggable by having the Certificate Controller request an SSL certificate according to the information in Certificate object that is being created. (#3050)

Hide /quitquitquit endpoint of queue-proxy #3279 (thanks @andrew-su)

We hide the /quitquitquit endpoint of queue-proxy, so that it can only be lameducked by the kubelet during shutdown.

GRPC streaming now works on cold start #3239 (thanks @mattmoor)

After a service is scaled-to-zero, a streaming gRPC request should return successfully.

Istio 1.0.7

Update Istio template to 1.0.7.

Health check on cold starts #2856 (thanks @mattmoor)

Activator avoids unnecessary retries by performing health-checking of Revision service after cold start.

Bug fixes / Cleanups:

• Fix the authority bug (affected GRPC) #3370 (thanks @tanzeeb)
• Eliminate the request size limit on cold start #3359 (thanks @mattmoor)
• Add Activator readiness probe for improved availability #2857 (thanks @mattmoor)

Monitoring

Write request logs from queue proxy #3561 (thanks @mdemirhan)

Queue proxy now has the ability to write request logs to stdout. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Write request metrics from queue proxy #3596 (thanks @yanweiguo)

Queue proxy now has the ability to generate request count and latency metrics. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Knative Serving release v0.5.1

Autoscaling

Bucketize autoscaling metrics by timeframe (#3289 - thanks @markusthoemmes)

Stats are averaged in each specific timeframe vs. averaged over the whole window. Fixes #2977.

Prevent double counting by the autoscaler (#3477 - thanks @hohaichi)

Let the queue report a concurrency metric for proxied requests, and the autoscaler discount this concurrency when making scaling decisions. Fixes bug (#3301).

Scrape queue-proxy metrics in autoscaler (#3149, thanks @yanweiguo)

Scrape queue-proxy metrics in autoscaler instead of pushing metrics from queue-proxy to autoscaler via websocket connection. Remove the websocket usage in queue-proxy. Fixes bugs #2203, #1927.

Bug Fixes / Cleanups

• Doc update for scale annotations lifetime (#3461 - thanks @yanweiguo) resolving bug (#2720)
• Surface events for all reconciliation failures (#3462 - thanks @mattmoor)---fixes #2941.
• Only add KPA label to a K8S service of a revision if KPA is used (#3498 - thanks @yanweiguo)---fixes #2606 (HPA and KPA controllers are getting wrong class PAs)

Core API

Surface events on reconciliation failures (thanks @mattmoor)

Changes boilerplate to surface reconciliation failures in addition to existing logging entries already in place. (#3462) which fixes bug (#2941).

Several Revision default values are now configurable (thanks @mattmoor)

A new ConfigMap in knative-serving called config-defaults enables the customization of the default CPU resource request and request timeout (#3546).

Add a url: to Service/Route status (thanks @dgerd)

A new url: field in the status.traffic block of Service and Route is displayed to enable discovery of the generated URL for name: sub-routes (#3526). This field is only present when name: is used for a traffic target. See the url: field for Route and Service in spec.md

More conformance tests:

• Validating user securityContext is reflected in container. (thanks @dgerd)
  • This change also adds the group information to the runtime test image, but we do not validate this as 1. it is not currently part of the runtime contract and 2. Setting group is currently an alpha feature that does not work on many Kubernetes clusters. See kubernetes/enhancements#213 - (#3423)
• Validating that metadata.generateName can be used instead of metadata.name (thanks @brandone-mabey)
  • Add testing to ensure that metadata.generateName can be used instead of metadata.name to specify names for services, routes, and configurations. (#3292) fixing (#3183)

Bug Fixes / Cleanups:

• CPU requests is now defaulted in the webhook (thanks @mattmoor) (#3529)
• Fixed broken deploy.sh **(thanks @adrchunha) **(#3367)
• Dropped functional dependency on 'configurationMetadaraGeneration' Label **(thanks @pivotal-joshua-rider) **(#3325)

Networking

Define 'Certificate' CRD for Auto TLS (thanks @ZhiminXiang)

As part of work in #1964, define an internal CRD Certificate to make TLS provision more pluggable by having the Certificate Controller request an SSL certificate according to the information in Certificate object that is being created. (#3050)

Hide /quitquitquit endpoint of queue-proxy #3279 (thanks @andrew-su)

We hide the /quitquitquit endpoint of queue-proxy, so that it can only be lameducked by the kubelet during shutdown.

GRPC streaming now works on cold start #3239 (thanks @mattmoor)

After a service is scaled-to-zero, a streaming gRPC request should return successfully.

Istio 1.0.7

Update Istio template to 1.0.7.

Health check on cold starts #2856 (thanks @mattmoor)

Activator avoids unnecessary retries by performing health-checking of Revision service after cold start.

Bug fixes / Cleanups:

• Fix the authority bug (affected GRPC) #3370 (thanks @tanzeeb)
• Eliminate the request size limit on cold start #3359 (thanks @mattmoor)
• Add Activator readiness probe for improved availability #2857 (thanks @mattmoor)

Monitoring

Write request logs from queue proxy #3561 (thanks @mdemirhan)

Queue proxy now has the ability to write request logs to stdout. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Write request metrics from queue proxy #3596 (thanks @yanweiguo)

Queue proxy now has the ability to generate request count and latency metrics. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Knative Serving release v0.5.0

Autoscaling

Bucketize autoscaling metrics by timeframe (#3289 - thanks @markusthoemmes)

Stats are averaged in each specific timeframe vs. averaged over the whole window. Fixes #2977.

Prevent double counting by the autoscaler (#3477 - thanks @hohaichi)

Let the queue report a concurrency metric for proxied requests, and the autoscaler discount this concurrency when making scaling decisions. Fixes bug (#3301).

Scrape queue-proxy metrics in autoscaler (#3149, thanks @yanweiguo)

Scrape queue-proxy metrics in autoscaler instead of pushing metrics from queue-proxy to autoscaler via websocket connection. Remove the websocket usage in queue-proxy. Fixes bugs #2203, #1927.

Bug Fixes / Cleanups

• Doc update for scale annotations lifetime (#3461 - thanks @yanweiguo) resolving bug (#2720)
• Surface events for all reconciliation failures (#3462 - thanks @mattmoor)---fixes #2941.
• Only add KPA label to a K8S service of a revision if KPA is used (#3498 - thanks @yanweiguo)---fixes #2606 (HPA and KPA controllers are getting wrong class PAs)

Core API

Surface events on reconciliation failures (thanks @mattmoor)

Changes boilerplate to surface reconciliation failures in addition to existing logging entries already in place. (#3462) which fixes bug (#2941).

Several Revision default values are now configurable (thanks @mattmoor)

A new ConfigMap in knative-serving called config-defaults enables the customization of the default CPU resource request and request timeout (#3546).

Add a url: to Service/Route status (thanks @dgerd)

A new url: field in the status.traffic block of Service and Route is displayed to enable discovery of the generated URL for name: sub-routes (#3526). This field is only present when name: is used for a traffic target. See the url: field for Route and Service in spec.md

More conformance tests:

• Validating user securityContext is reflected in container. (thanks @dgerd)
  • This change also adds the group information to the runtime test image, but we do not validate this as 1. it is not currently part of the runtime contract and 2. Setting group is currently an alpha feature that does not work on many Kubernetes clusters. See kubernetes/enhancements#213 - (#3423)
• Validating that metadata.generateName can be used instead of metadata.name (thanks @brandone-mabey)
  • Add testing to ensure that metadata.generateName can be used instead of metadata.name to specify names for services, routes, and configurations. (#3292) fixing (#3183)

Bug Fixes / Cleanups:

• CPU requests is now defaulted in the webhook (thanks @mattmoor) (#3529)
• Fixed broken deploy.sh **(thanks @adrchunha) **(#3367)
• Dropped functional dependency on 'configurationMetadaraGeneration' Label **(thanks @pivotal-joshua-rider) **(#3325)

Networking

Define 'Certificate' CRD for Auto TLS (thanks @ZhiminXiang)

As part of work in #1964, define an internal CRD Certificate to make TLS provision more pluggable by having the Certificate Controller request an SSL certificate according to the information in Certificate object that is being created. (#3050)

Hide /quitquitquit endpoint of queue-proxy #3279 (thanks @xtreme-andrew-su)

We hide the /quitquitquit endpoint of queue-proxy, so that it can only be lameducked by the kubelet during shutdown.

GRPC streaming now works on cold start #3239 (thanks @mattmoor)

After a service is scaled-to-zero, a streaming gRPC request should return successfully.

Istio 1.0.7

Update Istio template to 1.0.7.

Health check on cold starts #2856 (thanks @mattmoor)

Activator avoids unnecessary retries by performing health-checking of Revision service after cold start.

Bug fixes / Cleanups:

• Fix the authority bug (affected GRPC) #3370 (thanks @tanzeeb)
• Eliminate the request size limit on cold start #3359 (thanks @mattmoor)
• Add Activator readiness probe for improved availability #2857 (thanks @mattmoor)

Monitoring

Write request logs from queue proxy #3561 (thanks @mdemirhan)

Queue proxy now has the ability to write request logs to stdout. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Write request metrics from queue proxy #3596 (thanks @yanweiguo)

Queue proxy now has the ability to generate request count and latency metrics. This functionality already exists in Istio mesh and the reason for the duplication here is to move towards eliminating our dependency on Istio's mesh functionality.

Knative Serving release v0.4.1

Meta

Support for ConfigMap and Secret Volumes (Sample)

We have expanded our ConfigMap and Secret support to also allow them to be mounted as volumes. This was one of one of the most commonly requested features.

HTTP2 / GRPC support (Sample)

Knative Service can now serve HTTP2 and gRPC/HTTP2. Users will need to name their container port ‘h2c’ in order to enable this feature. There is an outstanding issue with streaming RPC cold-starts (#3239), but folks are encouraged to try this out and give feedback.

Enable websocket upgrades (Sample)

We have plumbed through the various layers of Serving the ability to upgrade an inbound HTTP connection to support Websockets. (#2910, #3240)

Better preserve ConfigMap updates

Starting with this release, you should be able to simply kubectl apply -f serving.yaml to update knative/serving. Previously this would cause problems for any ConfigMap resources customized via kubectl edit cm, such as configuring a different default domain.
To address this large problem with updates, we have moved all of our default ConfigMap settings into our code. The ConfigMap will only contain a detailed _example: | block outlining how to use each ConfigMap with (validated!) examples. To try this out, see kubectl edit cm -knative-serving config-autoscaler.

Autoscaling

Cold-start improvements (thanks @greghaynes)

• Immediately send stats from activator to autoscaler (#2659)
• Immediately scale up when autoscaler gets stats from activator (#2961, #2963)

Add “throttling” to the activator (thanks @vvraskin)

The activator now throttles the traffic it sends to pods to respect the containerConcurrency setting of the Revision. This is to avoid overloading a single pod with a low concurrency setting when a big burst of traffic is received when scaled to zero. (#3082, #3089)

Use actual pod counts in the autoscaler (thanks @yanweiguo)

This release starts using actual ready pods count to calculate scaling up rate instead of the observed pods count from queue-proxy metrics data.

Core API

Support for ConfigMap and Secret Volumes (thanks @mattmoor)

See above

Support creating Services in “release” mode (thanks @vagababov)

Starting in 0.4 you can use the sentinel string @latest in “release” mode to send a designated portion of the main traffic to the latest ready revision stamped out by the Service’s Configuration (#2819)

Phase 2 of CRD sub-resources (thanks @dprotaso)

This release contains the second phase of CRD sub-resource adoption, which consists of three parts: start using generateName for Revision names, stop incrementing spec.generation on updates, and using metadata.generation for the value of our Configuration generation label on Revisions. (#3179, knative/pkg#234, #3204)

Service captures author info in annotation (thanks @vagababov)

The email of a Service’s creator is now captured in serving.knative.dev/creator and in subsequent updates serving.knative.dev/lastModifier (#3213)

(Ongoing) Progress on implementing the Runtime Contract (thanks @dgerd)

We have triaged a significant backlog of issues related to Conformance (tracked here: https://github.com/knative/serving/projects/20), and started to close a number of the gaps in our Runtime Contract conformance and validation.

Build extensibility now uses “Aggregated Cluster Roles” (thanks @mattmoor)

We have split out the part of the ClusterRole that Serving defined to enable it to consume knative/build and changed Serving to use an aggregated cluster role to glue together the set of capabilities that it needs. To use

Bug Fixes / Cleanups

• Make Conditions with Severity: Error less scary (#3038)
• Make our resources work properly when “finalizers” are present (#3036)
• Fix several “readiness” races (#2954, #2430, #2735)
• Strengthen our validation logic (#3034, #3019, #1299)
• Better support for on-cluster registries both with and without certificates (#2266)
• Consistently set status.observedGeneration on resource status (#3076)

Networking

Removed the Knative Istio IngressGateway (thanks @tcnghia)

This release removes the Knative Istio IngressGateway, for better compatibility with multiple versions of Istio and also reduce the number of needed LoadBalancer. Users upgrading from 0.3.0 needs to reconfigure their DNS to point to the istio-ingressgateway Service IP address before upgrading, and remove the knative-ingressgateway Service & Deployment.

HTTP2 / GRPC support (thanks @tanzeeb)

See above

Configurable default ClusterIngress controller (thanks @tcnghia)

This release adds an option in the config-network ConfigMap to change the default ClusterIngress controller from Istio if another controller is available.

Default TCP ReadinessProbe for user container (thanks @markusthoemmes)

Due to Istio proxy intercepting TCP handshakes, users won’t be able to specify meaningful TCP readiness check. We folded this TCP handshake into queue-proxy health check so that even if users don’t specify any Readiness Probe, some basic TCP probe will be provided automatically.

Enable websocket upgrades (thanks @tcnghia)

See above

Bug Fixes

• More reliably clean up ClusterIngress resources (#2570)
• Support non-default cluster service domain (#2892)
• Change the Envoy connectionTimeout Istio uses to reduce 503s (#2988)

Monitoring

No changes in 0.4

Knative Serving release v0.4.0

Meta

Support for ConfigMap and Secret Volumes (Sample)

We have expanded our ConfigMap and Secret support to also allow them to be mounted as volumes. This was one of one of the most commonly requested features.

HTTP2 / GRPC support (Sample)

Knative Service can now serve HTTP2 and gRPC/HTTP2. Users will need to name their container port ‘h2c’ in order to enable this feature. There is an outstanding issue with streaming RPC cold-starts (#3239), but folks are encouraged to try this out and give feedback.

Enable websocket upgrades (Sample)

We have plumbed through the various layers of Serving the ability to upgrade an inbound HTTP connection to support Websockets. (#2910, #3240)

Better preserve ConfigMap updates

Starting with this release, you should be able to simply kubectl apply -f serving.yaml to update knative/serving. Previously this would cause problems for any ConfigMap resources customized via kubectl edit cm, such as configuring a different default domain.
To address this large problem with updates, we have moved all of our default ConfigMap settings into our code. The ConfigMap will only contain a detailed _example: | block outlining how to use each ConfigMap with (validated!) examples. To try this out, see kubectl edit cm -knative-serving config-autoscaler.

Autoscaling

Cold-start improvements (thanks @greghaynes)

• Immediately send stats from activator to autoscaler (#2659)
• Immediately scale up when autoscaler gets stats from activator (#2961, #2963)

Add “throttling” to the activator (thanks @vvraskin)

The activator now throttles the traffic it sends to pods to respect the containerConcurrency setting of the Revision. This is to avoid overloading a single pod with a low concurrency setting when a big burst of traffic is received when scaled to zero. (#3082, #3089)

Use actual pod counts in the autoscaler (thanks @yanweiguo)

This release starts using actual ready pods count to calculate scaling up rate instead of the observed pods count from queue-proxy metrics data.

Core API

Support for ConfigMap and Secret Volumes (thanks @mattmoor)

See above

Support creating Services in “release” mode (thanks @vagababov)

Starting in 0.4 you can use the sentinel string @latest in “release” mode to send a designated portion of the main traffic to the latest ready revision stamped out by the Service’s Configuration (#2819)

Phase 2 of CRD sub-resources (thanks @dprotaso)

This release contains the second phase of CRD sub-resource adoption, which consists of three parts: start using generateName for Revision names, stop incrementing spec.generation on updates, and using metadata.generation for the value of our Configuration generation label on Revisions. (#3179, knative/pkg#234, #3204)

Service captures author info in annotation (thanks @vagababov)

The email of a Service’s creator is now captured in serving.knative.dev/creator and in subsequent updates serving.knative.dev/lastModifier (#3213)

(Ongoing) Progress on implementing the Runtime Contract (thanks @dgerd)

We have triaged a significant backlog of issues related to Conformance (tracked here: https://github.com/knative/serving/projects/20), and started to close a number of the gaps in our Runtime Contract conformance and validation.

Build extensibility now uses “Aggregated Cluster Roles” (thanks @mattmoor)

We have split out the part of the ClusterRole that Serving defined to enable it to consume knative/build and changed Serving to use an aggregated cluster role to glue together the set of capabilities that it needs. To use

Bug Fixes / Cleanups

• Make Conditions with Severity: Error less scary (#3038)
• Make our resources work properly when “finalizers” are present (#3036)
• Fix several “readiness” races (#2954, #2430, #2735)
• Strengthen our validation logic (#3034, #3019, #1299)
• Better support for on-cluster registries both with and without certificates (#2266)
• Consistently set status.observedGeneration on resource status (#3076)

Networking

Removed the Knative Istio IngressGateway (thanks @tcnghia)

This release removes the Knative Istio IngressGateway, for better compatibility with multiple versions of Istio and also reduce the number of needed LoadBalancer. Users upgrading from 0.3.0 needs to reconfigure their DNS to point to the istio-ingressgateway Service IP address before upgrading, and remove the knative-ingressgateway Service & Deployment.

HTTP2 / GRPC support (thanks @tanzeeb)

See above

Configurable default ClusterIngress controller (thanks @tcnghia)

This release adds an option in the config-network ConfigMap to change the default ClusterIngress controller from Istio if another controller is available.

Default TCP ReadinessProbe for user container (thanks @markusthoemmes)

Due to Istio proxy intercepting TCP handshakes, users won’t be able to specify meaningful TCP readiness check. We folded this TCP handshake into queue-proxy health check so that even if users don’t specify any Readiness Probe, some basic TCP probe will be provided automatically.

Enable websocket upgrades (thanks @tcnghia)

See above

Bug Fixes

• More reliably clean up ClusterIngress resources (#2570)
• Support non-default cluster service domain (#2892)
• Change the Envoy connectionTimeout Istio uses to reduce 503s (#2988)

Monitoring

No changes in 0.4

Knative Serving release v0.3.0

Meta

More regular releases

0.3 is the first release of our new schedule of releasing every 6 weeks.
This release is a smaller delta than 0.2 because of this shorter development cycle.

Kubernetes 1.11 is now required

We now use the Kubernetes /status sub-resource support, which went Beta in K8s 1.11.

Autoscaling

More aggressive scale to zero

We will now scale Revisions down to zero pods after only 30 seconds of inactivity.

Support for HPA-class autoscaler (Kubecon Demo)

You may now opt to use the Kubernetes HPA for autoscaling Revisions, if you want to scale on CPU instead of request rate. HPA-class Revisions will not scale to zero.

Support target and window annotations for KPA-class autoscaler

You may now tune the default Knative autoscaler (KPA) with revision-level concurrency targets and different windows over which to calculate average concurrency.

Support mutable PodAutoscalers

You may now mutate PodAutoscaler specs in-place to adjust the scale bounds and other parameters. The changes will be picked up dynamically by KPA and HPA-class autoscalers.

Core API

Support of “resources” block (thanks @jszroberto)

You may now specify the resources section of a container spec to include reservations and limits on the resources your service may use. This also lets your Service gain access to GPUs, if available on your cluster.

Elide Builds for config-only changes

We will now skip the Build step of a deployment if only the configuration of a Revision changes (e.g. env var).

Revisions can now specify TimeoutSeconds

Requests that show no activity within the allotted timeout will be cancelled and respond with a 503.

Inactive isn’t failed!

Fixed an issue in 0.2 where Services, Configurations, and Revisions that were scaled to zero would appear to have failed. The “Active” condition is now treated as a purely informational condition.

A Container Port can now be specified for incoming traffic

You may now specify a single 'containerPort' to customize which port will receive request traffic. If unspecified, the default port (8080) will be assumed. As with previous releases, this value is available to the container through the environment variable '$PORT'.

More Global Resyncs

Editing the Serving ConfigMaps will now immediately trigger all existing resources to reconcile, and reflect any newly configured settings. This avoids the (up to 10 hour) delay that would otherwise exist waiting for the normal resync.

Networking

Use the default gateway istio-ingressgateway by default (@lichuqiang)

We have deprecated the Knative copy of the Istio ingress gateway.
Routes can be exposed to additional Gateways.

Routes without external domain are exposed only to local Gateways

Route that only have svc.cluster.local domain suffix will not be exposed to Istio ingress gateway by default.
Users can set the label
serving.knative.dev/visibility=cluster-local on a Route or Service to achieve the same effect.

Reconcile ClusterIngress' based on Ingress class annotation (@greghaynes)

ClusterIngress class annotation is taken into account when reconciling. The default ClusterIngress reconciler only reconcile when the type is Istio.

Bug fixes:

#2359 (@lichuqiang) Conflict between Routes having the same names causing issue.
#2582 (@markusthoemmes) Correct Revision Timeout meaning.

Monitoring

Metrics improvements

New metrics are added for Knative reconciler
Metric labels were renamed to be consistent across all Knative components
Knative control plane (reconciler, autoscaler & activator) metrics can now be exported to Stackdriver backend

Logging improvements

Commit id of the running build is added to the logs generated by Knative components

Knative Serving release v0.2.3

Cherry Picks

See: https://github.com/knative/serving/projects/6

Meta

Pluggability:

We have made significant progress “under the hood” encapsulating the major subsystems of knative/serving in an effort to support pluggability (e.g. replace Istio). Our core resources now configure these subsystems: Networking, Autoscaling, and Caching via new internal APIs.

Looser Coupling:

We have spent considerable effort working to ensure that all knative components are usable in isolation, and splitting out our release artifacts into the smallest units we can. For instance, you can now install and use knative/serving without knative/build, and even plug in alternative Build CRDs!

Our release now consists of:

• serving.yaml: just the knative/serving components.
• build.yaml: just knative/build’s 0.2.0 release
• monitoring*.yaml: a number of different configurations of our monitoring stack.
• istio*.yaml: two configurations of our Istio stack, one with sidecar injection and one without.
• release*.yaml: similar bundles to last release

Autoscaling

New shared autoscaler (special thanks to @glyn @markusthoemmes)

We have replaced the previous per-Revision autoscalers with a single shared autoscaler. This autoscaler is based on the same logic as the previous autoscaler, but has evolved to be purely metrics driven (including 0->1->0), eliminating the unnecessary Revision servingState field.

Introduced the ContainerConcurrency field

We have replaced ConcurrencyModel (Single or Multi) with an integer ContainerConcurrency field. This allows limiting concurrency to values other than 1 for certain use cases (e.g. limited thread pools).

• 1 is the new Single.
• 0 is the new Multi (no limit).

Example:

spec:
  containerConcurrency: 1
  container:
    image: docker.io/{username}/helloworld-go

ContainerConcurrency is now used to determine the autoscaler target concurrency.

Core API

Decoupled Build (special thanks to @imikushin)

Build is no longer required to run Serving, unless you plan to use it. The old style of expressing builds inline is still supported when Build is installed, but deprecated in favor of:

spec:
  build:
    apiVersion: build.knative.dev/v1alpha1
    kind: Build
    spec:
      # What was previously directly under "build:"

In addition, alternative Build implementations may be plugged in, the only requirement is that those Build resources indicate completion via:

status:
  conditions:
  - type: Succeeded
    status: True | False | Unknown

Revision GC (special thanks to @greghaynes)

A Configuration will now reclaim unroutable Revisions based on a few criteria:

• Time since its creation
• Time since it was last observed to be routable (via serving.knative.dev/lastPinned annotation heartbeat)
• Age (e.g. keep the last N)
• Is it the LatestReadyRevision of the Configuration?
• These knobs can be configured in config/config-gc.yaml.

Misc Features

• Service now supports the release and manual modes.
• Short names for Knative resources
• Custom columns in kubectl output (K8s 1.11+ only)
• Longer "resync" periods, with global resyncs on (some) configmap changes
• We now create caching.internal.knative.dev/Image resources to signal the images that are important to cache. Operators must install an extension to leverage these hints.

Networking

ClusterIngress abstraction (special thanks to @lichuqiang)

Route no longer depends directly on VirtualService, but an intermediate resource ClusterIngress which could be reconciled differently for different network platforms.

Istio Updates

• Moved to Istio 1.0.2.
• Hacks & workarounds needed due to 0.8.0 bugs are removed.
• Cluster-local access to Route (through route.ns.svc.cluster.local name) no longer requires the istio sidecar.

Bug fixes

• #2276 Route status incorrect for named traffic targets having the same revision #2234 Activator headers getting forwarded prevent scaling from 0 (@trisberg, @scothis)
• #2046 Activator can activate the wrong Service (@markusthoemmes)
• #2043 Activator leaks metrics publicly (@scothis, @tanzeeb)
• #1931 Queue-proxy does not work with custom cluster domain (@cizixs,@arvtiwar)
• #1907 Activator retries its own requests (@markusthoemmes).
• #1637 Placeholder k8s service reconciliation doesn't take defaulter into account #1229 Activator should retry with backoff (@markusthoemmes).
• #837 Activator deal with multiple ports (@markusthoemmes).

Monitoring

The monitoring namespace has changed to knative-monitoring

Bug fixes:

• #2255 Zipkin install without Elasticsearch
• #2201 Add service label to revisions (@pivotal-nader-ziada)
• #1474 Switch the fluentd configmap to one-per-Revision