Features Gates with SKS
Kubernetes can be extended with Feature Gates allowing you to control specific Kubernetes features with Exoscale SKS. Exoscale SKS supports Feature Gates for both Control Plane (SKS Cluster) and Data Plane (SKS NodePool).
Feature Gates support in Exoscale SKS allows to:
enable Feature Gates which are disabled by default;
disable previously enabled Feature Gates.
Feature Gates can be enabled or disabled at any time during the lifecycle of a SKS Cluster, i.e. when creating or updating a SKS Cluster. Changes to the Feature Gate list at the SKS Cluster level are transitively propagated to SKS Nodepool. However, SKS Nodepool must be upgraded after enabling or disabling Feature Gates for the change to be applied.
Feature Gates are supported by Exoscale SKS for all Kubernetes releases above and including 1.33.7, 1.34.3 and 1.35.0. Each Kubernetes version comes with a list of supported Feature Gates which are a subset of the Kubernetes Feature Gates. Only Feature Gates supported for a given Kubernetes version can be enabled.
At the API level, the feature_gates field is an array
of strings. The full list of Feature Gates must be given when modified. The
full list of Feature Gates given to the API endpoint specifies the list of
enabled Feature Gates. Any supported Feature
Gate not listed by the API endpoint should be
considered disabled.
Prerequisites
A running Exoscale SKS Cluster (Control Plane)
A running Exoscale SKS NodePool for any Feature Gates applying to the Data Plane (Kubelet)
Enabling Feature Gates
We can enable Feature Gates when we create or update a SKS Cluster.
For example, to enable the ImageVolume Feature Gates when creating a SKS
Cluster, we can use the following command via exo CLI:
➜ exo compute sks create cluster-001 --feature-gates ImageVolume
✔ Creating SKS cluster "cluster-001"... 1m9s
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ SKS CLUSTER │ │
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ ID │ ca0317bd-7ea9-4ffa-8720-89ee947761e2 │
│ Name │ cluster-001 │
│ Description │ │
│ Zone │ ch-gva-2 │
│ Creation Date │ 2026-01-19 15:50:39 +0000 UTC │
│ Auto-upgrade │ false │
│ Enable kube-proxy │ true │
│ Endpoint │ ca0317bd-7ea9-4ffa-8720-89ee947761e2.ppsks-ch-gva-2.exo.io │
│ Version │ 1.35.0 │
│ Service Level │ pro │
│ CNI │ calico │
│ Add-Ons │ exoscale-cloud-controller │
│ │ metrics-server │
│ Audit Enabled │ false │
│ Feature Gates │ ImageVolume │
│ State │ running │
│ Labels │ n/a │
│ Nodepools │ n/a │
┼───────────────────┼────────────────────────────────────────────────────────────┼Respectively, to also enable the MemoryQoS Feature Gate for the same example
cluster, we can update the cluster via the exo CLI command:
➜ exo compute sks update cluster-001 --feature-gates "ImageVolume,MemoryQoS"
✔ Updating SKS cluster "cluster-001"... 48s
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ SKS CLUSTER │ │
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ ID │ ca0317bd-7ea9-4ffa-8720-89ee947761e2 │
│ Name │ cluster-001 │
│ Description │ │
│ Zone │ ch-gva-2 │
│ Creation Date │ 2026-01-19 15:50:39 +0000 UTC │
│ Auto-upgrade │ false │
│ Enable kube-proxy │ true │
│ Endpoint │ ca0317bd-7ea9-4ffa-8720-89ee947761e2.ppsks-ch-gva-2.exo.io │
│ Version │ 1.35.0 │
│ Service Level │ pro │
│ CNI │ calico │
│ Add-Ons │ exoscale-cloud-controller │
│ │ metrics-server │
│ Audit Enabled │ false │
│ Feature Gates │ ImageVolume │
│ │ MemoryQoS │
│ State │ running │
│ Labels │ n/a │
│ Nodepools │ n/a │
┼───────────────────┼────────────────────────────────────────────────────────────┼Note
The full list of Feature Gates must be specified when modified via the API
endpoint hence why we include the previously enabled ImageVolume Feature
Gate. Only specifying MemoryQoS would have enabled the MemoryQoS Feature
Gate while the ImageVolume Feature Gate would have been disabled.
Tip
The --feature-gates option from exo CLI uses a comma-separated list of
strings to define a list of Feature Gates.
Disabling Feature Gates
We can disable previously enabled Feature Gates at any time by updating the given SKS Cluster.
Since Feature Gates support for SKS Cluster comes as a toggling list, disabling Feature Gates from an SKS Cluster means we have to update the given cluster with the cluster’s Feature Gate list minus the Feature Gates that we want to disable.
As an example, let’s use the cluster definition from the section above:
➜ exo compute sks get cluster-001
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ SKS CLUSTER │ │
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ ID │ ca0317bd-7ea9-4ffa-8720-89ee947761e2 │
│ Name │ cluster-001 │
│ Description │ │
│ Zone │ ch-gva-2 │
│ Creation Date │ 2026-01-19 15:50:39 +0000 UTC │
│ Auto-upgrade │ false │
│ Enable kube-proxy │ true │
│ Endpoint │ ca0317bd-7ea9-4ffa-8720-89ee947761e2.ppsks-ch-gva-2.exo.io │
│ Version │ 1.35.0 │
│ Service Level │ pro │
│ CNI │ calico │
│ Add-Ons │ exoscale-cloud-controller │
│ │ metrics-server │
│ Audit Enabled │ false │
│ Feature Gates │ ImageVolume │
│ │ MemoryQoS │
│ State │ running │
│ Labels │ n/a │
│ Nodepools │ n/a │
┼───────────────────┼────────────────────────────────────────────────────────────┼Following this example, to remove the MemoryQoS Feature Gate from cluster-001
we can use the following exo CLI command:
➜ exo compute sks update cluster-001 --feature-gates "ImageVolume"
✔ Updating SKS cluster "cluster-001"... 45s
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ SKS CLUSTER │ │
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ ID │ ca0317bd-7ea9-4ffa-8720-89ee947761e2 │
│ Name │ cluster-001 │
│ Description │ │
│ Zone │ ch-gva-2 │
│ Creation Date │ 2026-01-19 15:50:39 +0000 UTC │
│ Auto-upgrade │ false │
│ Enable kube-proxy │ true │
│ Endpoint │ ca0317bd-7ea9-4ffa-8720-89ee947761e2.ppsks-ch-gva-2.exo.io │
│ Version │ 1.35.0 │
│ Service Level │ pro │
│ CNI │ calico │
│ Add-Ons │ exoscale-cloud-controller │
│ │ metrics-server │
│ Audit Enabled │ false │
│ Feature Gates │ ImageVolume │
│ State │ running │
│ Labels │ n/a │
│ Nodepools │ n/a │
┼───────────────────┼────────────────────────────────────────────────────────────┼To reset the Feature Gate list from a given SKS Cluster, i.e. to disable all Feature Gates previously enabled, we update the SKS Cluster with an empty Feature Gate list.
Continuing with the example above, we disable all Feature Gates for the targeted SKS Cluster via the exo CLI command:
➜ exo compute sks update cluster-001 --feature-gates ""
✔ Updating SKS cluster "cluster-001"... 45s
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ SKS CLUSTER │ │
┼───────────────────┼────────────────────────────────────────────────────────────┼
│ ID │ ca0317bd-7ea9-4ffa-8720-89ee947761e2 │
│ Name │ cluster-001 │
│ Description │ │
│ Zone │ ch-gva-2 │
│ Creation Date │ 2026-01-19 15:50:39 +0000 UTC │
│ Auto-upgrade │ false │
│ Enable kube-proxy │ true │
│ Endpoint │ ca0317bd-7ea9-4ffa-8720-89ee947761e2.ppsks-ch-gva-2.exo.io │
│ Version │ 1.35.0 │
│ Service Level │ pro │
│ CNI │ calico │
│ Add-Ons │ exoscale-cloud-controller │
│ │ metrics-server │
│ Audit Enabled │ false │
│ Feature Gates │ │
│ State │ running │
│ Labels │ n/a │
│ Nodepools │ n/a │
┼───────────────────┼────────────────────────────────────────────────────────────┼Supported Feature Gates
Exoscale SKS allows a subset of all available Kubernetes Feature Gates. Mainly, those are Alpha-staged Feature Gates which are disabled by default. Beta-staged Feature Gates are enabled by default. More details are available in the Kubernetes feature stages documentation.
Below you can find the available Feature Gates for Exoscale SKS per Kubernetes version. For more details about each Feature Gate refer to the Feature Gates reference documentation.
Kubernetes v1.35
List of supported Feature Gates
AllowUnsafeMalformedObjectDeletionAnonymousAuthConfigurableEndpointsAnyVolumeDataSourceAPIResponseCompressionAPIServerIdentityAPIServerTracingAPIServingWithRoutineAuthorizeNodeWithSelectorsAuthorizeWithSelectorsCBORServingAndStorageClearingNominatedNodeNameAfterBindingCloudControllerManagerWatchBasedRoutesReconciliationCloudControllerManagerWebhookClusterTrustBundleClusterTrustBundleProjectionComponentFlagzComponentStatuszConsistentListFromCacheConstrainedImpersonationContainerCheckpointContainerRestartRulesContainerStopSignalsContextualLoggingCoordinatedLeaderElectionCPUCFSQuotaPeriodCPUManagerPolicyAlphaOptionsCPUManagerPolicyOptionsCRDValidationRatchetingCrossNamespaceVolumeDataSourceCSIMigrationPortworxCSIVolumeHealthCustomResourceFieldSelectorsDeploymentReplicaSetTerminatingReplicasDisableAllocatorDualWriteDisableNodeKubeProxyVersionDRAAdminAccessDRAConsumableCapacityDRADeviceBindingConditionsDRADeviceTaintRulesDRADeviceTaintsDRAExtendedResourceDRAPartitionableDevicesDRAPrioritizedListDRAResourceClaimDeviceStatusDynamicResourceAllocationEnvFilesEventedPLEGExternalServiceAccountTokenSignerGangSchedulingGenericWorkloadGracefulNodeShutdownGracefulNodeShutdownBasedOnPodPriorityHonorPVReclaimPolicyHostnameOverrideHPAConfigurableToleranceHPAScaleToZeroImageMaximumGCAgeImageVolumeInPlacePodLevelResourcesVerticalScalingInPlacePodVerticalScalingInPlacePodVerticalScalingAllocatedStatusInPlacePodVerticalScalingExclusiveCPUsInPlacePodVerticalScalingExclusiveMemoryInTreePluginPortworxUnregisterJobBackoffLimitPerIndexJobManagedByJobPodReplacementPolicyJobSuccessPolicyKubeletCgroupDriverFromCRIKubeletCrashLoopBackOffMaxKubeletEnsureSecretPulledImagesKubeletFineGrainedAuthzKubeletInUserNamespaceKubeletPodResourcesDynamicResourcesKubeletPodResourcesGetKubeletPSIKubeletSeparateDiskGCKubeletServiceAccountTokenForCredentialProvidersKubeletTracingLocalStorageCapacityIsolationFSQuotaMonitoringLogarithmicScaleDownLoggingAlphaOptionsMatchLabelKeysInPodAffinityMatchLabelKeysInPodTopologySpreadMaxUnavailableStatefulSetMemoryManagerMemoryQoSMultiCIDRServiceAllocatorMutableCSINodeAllocatableCountMutablePodResourcesForSuspendedJobsMutablePVNodeAffinityMutableSchedulingDirectivesForSuspendedJobsMutatingAdmissionPolicyNFTablesProxyModeNodeDeclaredFeaturesNodeInclusionPolicyInPodTopologySpreadNodeLogQueryNodeSwapNominatedNodeNameForExpectationOpenAPIEnumsPodAndContainerStatsFromCRIPodCertificateRequestPodDeletionCostPodLevelResourcesPodLifecycleSleepActionPodLifecycleSleepActionAllowZeroPodLogsQuerySplitStreamsPodObservedGenerationTrackingPodReadyToStartContainersConditionPodSchedulingReadinessPodTopologyLabelsAdmissionPortForwardWebsocketsPreferSameTrafficDistributionProcMountTypeQOSReservedRecoverVolumeExpansionFailureRecursiveReadOnlyMountsReduceDefaultCrashLoopBackOffDecayRelaxedDNSSearchValidationRelaxedEnvironmentVariableValidationRelaxedServiceNameValidationRemoteRequestHeaderUIDResourceHealthStatusRestartAllContainersOnContainerExitsRetryGenerateNameRotateKubeletServerCertificateRuntimeClassInImageCriAPISchedulerAsyncPreemptionSELinuxChangePolicySELinuxMountSELinuxMountReadWriteOncePodServiceAccountTokenJTIServiceAccountTokenNodeBindingServiceAccountTokenNodeBindingValidationServiceAccountTokenPodNodeInfoServiceTrafficDistributionSidecarContainersStorageCapacityScoringStorageVersionAPIStorageVersionHashStorageVersionMigratorStrictIPCIDRValidationStructuredAuthenticationConfigurationStructuredAuthorizationConfigurationSupplementalGroupsPolicyTaintTolerationComparisonOperatorsTopologyAwareHintsTopologyManagerPolicyAlphaOptionsTopologyManagerPolicyOptionsTranslateStreamCloseWebsocketRequestsUnknownVersionInteroperabilityProxyUserNamespacesHostNetworkSupportUserNamespacesSupportVolumeAttributesClassVolumeLimitScalingWatchListWindowsCPUAndMemoryAffinityWindowsGracefulNodeShutdownWindowsHostNetworkWinDSRWinOverlay
Kubernetes v1.34
List of supported Feature Gates
AllowUnsafeMalformedObjectDeletionAnonymousAuthConfigurableEndpointsAnyVolumeDataSourceAPIResponseCompressionAPIServerIdentityAPIServerTracingAPIServingWithRoutineAuthorizeNodeWithSelectorsAuthorizeWithSelectorsCBORServingAndStorageClearingNominatedNodeNameAfterBindingCloudControllerManagerWebhookClusterTrustBundleClusterTrustBundleProjectionComponentFlagzComponentSLIsComponentStatuszConsistentListFromCacheContainerCheckpointContainerRestartRulesContainerStopSignalsContextualLoggingCoordinatedLeaderElectionCPUCFSQuotaPeriodCPUManagerPolicyAlphaOptionsCPUManagerPolicyOptionsCRDValidationRatchetingCrossNamespaceVolumeDataSourceCSIMigrationPortworxCSIVolumeHealthCustomResourceFieldSelectorsDeploymentReplicaSetTerminatingReplicasDisableAllocatorDualWriteDisableNodeKubeProxyVersionDRAAdminAccessDRAConsumableCapacityDRADeviceBindingConditionsDRADeviceTaintsDRAExtendedResourceDRAPartitionableDevicesDRAPrioritizedListDRAResourceClaimDeviceStatusDynamicResourceAllocationEnvFilesEventedPLEGExternalServiceAccountTokenSignerGracefulNodeShutdownGracefulNodeShutdownBasedOnPodPriorityHonorPVReclaimPolicyHostnameOverrideHPAConfigurableToleranceHPAScaleToZeroImageMaximumGCAgeImageVolumeInPlacePodVerticalScalingInPlacePodVerticalScalingAllocatedStatusInPlacePodVerticalScalingExclusiveCPUsInPlacePodVerticalScalingExclusiveMemoryInTreePluginPortworxUnregisterJobBackoffLimitPerIndexJobManagedByJobPodReplacementPolicyJobSuccessPolicyKubeletCgroupDriverFromCRIKubeletCrashLoopBackOffMaxKubeletEnsureSecretPulledImagesKubeletFineGrainedAuthzKubeletInUserNamespaceKubeletPodResourcesDynamicResourcesKubeletPodResourcesGetKubeletPSIKubeletSeparateDiskGCKubeletServiceAccountTokenForCredentialProvidersKubeletTracingListFromCacheSnapshotLoadBalancerIPModeLocalStorageCapacityIsolationFSQuotaMonitoringLogarithmicScaleDownLoggingAlphaOptionsMatchLabelKeysInPodAffinityMatchLabelKeysInPodTopologySpreadMaxUnavailableStatefulSetMemoryManagerMemoryQoSMultiCIDRServiceAllocatorMutableCSINodeAllocatableCountMutatingAdmissionPolicyNFTablesProxyModeNodeInclusionPolicyInPodTopologySpreadNodeLogQueryNodeSwapNominatedNodeNameForExpectationOpenAPIEnumsPodAndContainerStatsFromCRIPodCertificateRequestPodDeletionCostPodLevelResourcesPodLifecycleSleepActionPodLifecycleSleepActionAllowZeroPodLogsQuerySplitStreamsPodObservedGenerationTrackingPodReadyToStartContainersConditionPodSchedulingReadinessPodTopologyLabelsAdmissionPortForwardWebsocketsPreferSameTrafficDistributionProcMountTypeQOSReservedRecoverVolumeExpansionFailureRecursiveReadOnlyMountsReduceDefaultCrashLoopBackOffDecayRelaxedDNSSearchValidationRelaxedEnvironmentVariableValidationRelaxedServiceNameValidationRemoteRequestHeaderUIDResourceHealthStatusRetryGenerateNameRotateKubeletServerCertificateRuntimeClassInImageCriAPISchedulerAsyncPreemptionSELinuxChangePolicySELinuxMountSELinuxMountReadWriteOncePodServiceAccountTokenJTIServiceAccountTokenNodeBindingServiceAccountTokenNodeBindingValidationServiceAccountTokenPodNodeInfoServiceTrafficDistributionSidecarContainersSizeMemoryBackedVolumesStatefulSetAutoDeletePVCStorageCapacityScoringStorageVersionAPIStorageVersionHashStorageVersionMigratorStrictIPCIDRValidationStructuredAuthenticationConfigurationStructuredAuthorizationConfigurationSupplementalGroupsPolicyTopologyAwareHintsTopologyManagerPolicyAlphaOptionsTopologyManagerPolicyOptionsTranslateStreamCloseWebsocketRequestsUnknownVersionInteroperabilityProxyUserNamespacesPodSecurityStandardsUserNamespacesSupportVolumeAttributesClassWatchListWindowsCPUAndMemoryAffinityWindowsGracefulNodeShutdownWindowsHostNetworkWinDSRWinOverlay
Kubernetes v1.33
List of supported Feature Gates
AllowUnsafeMalformedObjectDeletionAnonymousAuthConfigurableEndpointsAnyVolumeDataSourceAPIResponseCompressionAPIServerIdentityAPIServerTracingAPIServingWithRoutineAuthorizeNodeWithSelectorsAuthorizeWithSelectorsCBORServingAndStorageCloudControllerManagerWebhookClusterTrustBundleClusterTrustBundleProjectionComponentFlagzComponentSLIsComponentStatuszConsistentListFromCacheContainerCheckpointContainerStopSignalsContextualLoggingCoordinatedLeaderElectionCPUCFSQuotaPeriodCPUManagerPolicyAlphaOptionsCPUManagerPolicyOptionsCRDValidationRatchetingCrossNamespaceVolumeDataSourceCSIMigrationPortworxCSIVolumeHealthCustomResourceFieldSelectorsDeploymentReplicaSetTerminatingReplicasDevicePluginCDIDevicesDisableAllocatorDualWriteDisableNodeKubeProxyVersionDRAAdminAccessDRADeviceTaintsDRAPartitionableDevicesDRAPrioritizedListDRAResourceClaimDeviceStatusDynamicResourceAllocationEventedPLEGExternalServiceAccountTokenSignerGracefulNodeShutdownGracefulNodeShutdownBasedOnPodPriorityHonorPVReclaimPolicyHPAConfigurableToleranceHPAScaleToZeroImageMaximumGCAgeImageVolumeInPlacePodVerticalScalingInPlacePodVerticalScalingAllocatedStatusInPlacePodVerticalScalingExclusiveCPUsInTreePluginPortworxUnregisterJobBackoffLimitPerIndexJobManagedByJobPodReplacementPolicyJobSuccessPolicyKubeletCgroupDriverFromCRIKubeletCrashLoopBackOffMaxKubeletEnsureSecretPulledImagesKubeletFineGrainedAuthzKubeletInUserNamespaceKubeletPodResourcesDynamicResourcesKubeletPodResourcesGetKubeletPSIKubeletSeparateDiskGCKubeletServiceAccountTokenForCredentialProvidersKubeletTracingListFromCacheSnapshotLoadBalancerIPModeLocalStorageCapacityIsolationFSQuotaMonitoringLogarithmicScaleDownLoggingAlphaOptionsMatchLabelKeysInPodAffinityMatchLabelKeysInPodTopologySpreadMaxUnavailableStatefulSetMemoryManagerMemoryQoSMultiCIDRServiceAllocatorMutableCSINodeAllocatableCountMutatingAdmissionPolicyNFTablesProxyModeNodeInclusionPolicyInPodTopologySpreadNodeLogQueryNodeSwapOpenAPIEnumsPodAndContainerStatsFromCRIPodDeletionCostPodDisruptionConditionsPodLevelResourcesPodLifecycleSleepActionPodLifecycleSleepActionAllowZeroPodLogsQuerySplitStreamsPodObservedGenerationTrackingPodReadyToStartContainersConditionPodSchedulingReadinessPodTopologyLabelsAdmissionPortForwardWebsocketsPreferSameTrafficDistributionProcMountTypeQOSReservedRecoverVolumeExpansionFailureRecursiveReadOnlyMountsReduceDefaultCrashLoopBackOffDecayRelaxedDNSSearchValidationRelaxedEnvironmentVariableValidationRemoteRequestHeaderUIDResourceHealthStatusRetryGenerateNameRotateKubeletServerCertificateRuntimeClassInImageCriAPISchedulerAsyncPreemptionSELinuxChangePolicySELinuxMountSELinuxMountReadWriteOncePodServiceAccountTokenJTIServiceAccountTokenNodeBindingServiceAccountTokenNodeBindingValidationServiceAccountTokenPodNodeInfoServiceTrafficDistributionSidecarContainersSizeMemoryBackedVolumesStatefulSetAutoDeletePVCStatefulSetStartOrdinalStorageCapacityScoringStorageVersionAPIStorageVersionHashStorageVersionMigratorStrictIPCIDRValidationStructuredAuthenticationConfigurationStructuredAuthorizationConfigurationSupplementalGroupsPolicyTopologyAwareHintsTopologyManagerPolicyAlphaOptionsTopologyManagerPolicyOptionsTranslateStreamCloseWebsocketRequestsUnknownVersionInteroperabilityProxyUserNamespacesPodSecurityStandardsUserNamespacesSupportVolumeAttributesClassWatchListWindowsCPUAndMemoryAffinityWindowsGracefulNodeShutdownWindowsHostNetworkWinDSRWinOverlay
Limitations
The following limitations are enforced when using Feature Gates with SKS:
Feature Gates use default configuration only; i.e. it is currently not possible to set a custom configuration for a Feature Gate that allows one.
NodePools MUST be upgraded to provide the Feature Gates enabled for SKS.
Feature Gates are currently only supported for NodePool. Karpenter support should be added in the near future.
Last updated on