Skip to content

CORS-4212: AWS: Add the ability to configure throughput on GP3 volumes #9945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-bot merged 3 commits into from
Nov 22, 2025
Merged

CORS-4212: AWS: Add the ability to configure throughput on GP3 volumes #9945

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fdde8b0
AWS: Add the ability to configure throughput on GP3 volumes
jhixson74 Sep 17, 2025
0224f1a
Bump openshift/api for AWS GP3 changes
jhixson74 Nov 20, 2025
2e5a20b
go mod vendor for AWS GP3 changes
jhixson74 Nov 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions data/data/install.openshift.io_installconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,13 @@ spec:
(GiB).
minimum: 0
type: integer
throughput:
description: |-
Throughput to provision in MiB/s supported for the volume type. This
is currently only applicable to volumes of type gp3.
format: int64
minimum: 0
type: integer
type:
description: Type defines the type of the volume.
type: string
Expand Down Expand Up @@ -1739,6 +1746,13 @@ spec:
gibibytes (GiB).
minimum: 0
type: integer
throughput:
description: |-
Throughput to provision in MiB/s supported for the volume type. This
is currently only applicable to volumes of type gp3.
format: int64
minimum: 0
type: integer
type:
description: Type defines the type of the volume.
type: string
Expand Down Expand Up @@ -3197,6 +3211,13 @@ spec:
(GiB).
minimum: 0
type: integer
throughput:
description: |-
Throughput to provision in MiB/s supported for the volume type. This
is currently only applicable to volumes of type gp3.
format: int64
minimum: 0
type: integer
type:
description: Type defines the type of the volume.
type: string
Expand Down Expand Up @@ -4848,6 +4869,13 @@ spec:
(GiB).
minimum: 0
type: integer
throughput:
description: |-
Throughput to provision in MiB/s supported for the volume type. This
is currently only applicable to volumes of type gp3.
format: int64
minimum: 0
type: integer
type:
description: Type defines the type of the volume.
type: string
Expand Down
3 changes: 3 additions & 0 deletions docs/user/aws/customization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Beyond the [platform-agnostic `install-config.yaml` properties](../customization
* `rootVolume` (optional object): Defines the root volume for EC2 instances in the machine pool.
* `iops` (optional integer): The amount of provisioned [IOPS][volume-iops].
This is only valid for `type` `io1`.
* `throughput` (optional integer): The amount of throughput in MiB/s [Throughput Performance][volume-throughput].
This is only valid for `type` `gp3`.
* `size` (optional integer): Size of the root volume in gibibytes (GiB).
* `type` (optional string): The [type of volume][volume-type].
* `kmsKeyARN` (optional string): The [ARN of KMS key][kms-key] that should be used to encrypt the EBS volume.
Expand Down Expand Up @@ -119,4 +121,5 @@ sshKey: ssh-ed25519 AAAA...
[kms-key-default]: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_GetEbsDefaultKmsKeyId.html
[kms-key]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html
[volume-iops]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html
[volume-throughput]: https://docs.aws.amazon.com/ebs/latest/userguide/general-purpose.html#gp3-ebs-volume-type
[volume-type]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ require (
github.com/nutanix-cloud-native/cluster-api-provider-nutanix v1.7.0
github.com/nutanix-cloud-native/prism-go-client v0.5.0
github.com/onsi/gomega v1.38.2
github.com/openshift/api v0.0.0-20251117165054-348370f055bf
github.com/openshift/api v0.0.0-20251120220512-cb382c9eaf42
github.com/openshift/assisted-image-service v0.0.0-20240607085136-02df2e56dde6
github.com/openshift/assisted-service/api v0.0.0
github.com/openshift/assisted-service/client v0.0.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -701,8 +701,8 @@ github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQ
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/opencontainers/runtime-spec v1.2.0 h1:z97+pHb3uELt/yiAWD691HNHQIF07bE7dzrbT927iTk=
github.com/opencontainers/runtime-spec v1.2.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/openshift/api v0.0.0-20251117165054-348370f055bf h1:8VzLlQFneh4bnHA3SS+Bb9VWdVaR7WugtSeqIngMC3s=
github.com/openshift/api v0.0.0-20251117165054-348370f055bf/go.mod h1:d5uzF0YN2nQQFA0jIEWzzOZ+edmo6wzlGLvx5Fhz4uY=
github.com/openshift/api v0.0.0-20251120220512-cb382c9eaf42 h1:Mo2FlDdoCZ+BE2W4C0lNcxEDeIIhfsYFP6vj4Sggp8w=
github.com/openshift/api v0.0.0-20251120220512-cb382c9eaf42/go.mod h1:d5uzF0YN2nQQFA0jIEWzzOZ+edmo6wzlGLvx5Fhz4uY=
github.com/openshift/assisted-image-service v0.0.0-20240607085136-02df2e56dde6 h1:U6ve+dnHlHhAELoxX+rdFOHVhoaYl0l9qtxwYtsO6C0=
github.com/openshift/assisted-image-service v0.0.0-20240607085136-02df2e56dde6/go.mod h1:o2H5VwQhUD8P6XsK6dRmKpCCJqVvv12KJQZBXmcCXCU=
github.com/openshift/assisted-service/api v0.0.0-20250922204150-a52b83145bea h1:YhJ9iHKKT5ooAdVr8qq3BdudhTxP/WF0XYDT5gzi1ak=
Expand Down
1 change: 1 addition & 0 deletions pkg/asset/machines/aws/awsmachines.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ func GenerateMachines(clusterID string, in *MachineInput) ([]*asset.RuntimeFile,
Size: int64(mpool.EC2RootVolume.Size),
Type: capa.VolumeType(mpool.EC2RootVolume.Type),
IOPS: int64(mpool.EC2RootVolume.IOPS),
Throughput: ptr.To(mpool.EC2RootVolume.Throughput),
Encrypted: ptr.To(true),
EncryptionKey: mpool.KMSKeyARN,
},
Expand Down
13 changes: 9 additions & 4 deletions pkg/asset/machines/aws/awsmachines_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ func TestGenerateMachines(t *testing.T) {
},
SSHKeyName: ptr.To(""),
RootVolume: &capa.Volume{
Encrypted: ptr.To(true),
Encrypted: ptr.To(true),
Throughput: ptr.To(int64(0)),
},
UncompressedUserData: ptr.To(true),
Ignition: &capa.Ignition{
Expand Down Expand Up @@ -171,7 +172,8 @@ func TestGenerateMachines(t *testing.T) {
},
SSHKeyName: ptr.To(""),
RootVolume: &capa.Volume{
Encrypted: ptr.To(true),
Encrypted: ptr.To(true),
Throughput: ptr.To(int64(0)),
},
UncompressedUserData: ptr.To(true),
Ignition: &capa.Ignition{
Expand Down Expand Up @@ -227,8 +229,11 @@ func TestGenerateMachines(t *testing.T) {
fmt.Sprintf("%s-subnet-public-%s", stubClusterID, machineZone),
}}},
},
SSHKeyName: ptr.To(""),
RootVolume: &capa.Volume{Encrypted: ptr.To(true)},
SSHKeyName: ptr.To(""),
RootVolume: &capa.Volume{
Encrypted: ptr.To(true),
Throughput: ptr.To(int64(0)),
},
UncompressedUserData: ptr.To(true),
Ignition: &capa.Ignition{
StorageType: capa.IgnitionStorageTypeOptionUnencryptedUserData,
Expand Down
11 changes: 6 additions & 5 deletions pkg/asset/machines/aws/machines.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,11 +239,12 @@ func provider(in *machineProviderInput) (*machineapi.AWSMachineProviderConfig, e
BlockDevices: []machineapi.BlockDeviceMappingSpec{
{
EBS: &machineapi.EBSBlockDeviceSpec{
VolumeType: pointer.String(in.root.Type),
VolumeSize: pointer.Int64(int64(in.root.Size)),
Iops: pointer.Int64(int64(in.root.IOPS)),
Encrypted: pointer.Bool(true),
KMSKey: machineapi.AWSResourceReference{ARN: pointer.String(in.root.KMSKeyARN)},
VolumeType: pointer.String(in.root.Type),
VolumeSize: pointer.Int64(int64(in.root.Size)),
Iops: pointer.Int64(int64(in.root.IOPS)),
ThroughputMib: pointer.Int32(int32(in.root.Throughput)),
Encrypted: pointer.Bool(true),
KMSKey: machineapi.AWSResourceReference{ARN: pointer.String(in.root.KMSKeyARN)},
},
},
},
Expand Down
10 changes: 10 additions & 0 deletions pkg/types/aws/machinepool.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ func (a *MachinePool) Set(required *MachinePool) {
if required.EC2RootVolume.IOPS != 0 {
a.EC2RootVolume.IOPS = required.EC2RootVolume.IOPS
}
if required.EC2RootVolume.Throughput != 0 {
a.EC2RootVolume.Throughput = required.EC2RootVolume.Throughput
}
if required.EC2RootVolume.Size != 0 {
a.EC2RootVolume.Size = required.EC2RootVolume.Size
}
Expand Down Expand Up @@ -107,6 +110,13 @@ type EC2RootVolume struct {
// +optional
IOPS int `json:"iops"`

// Throughput to provision in MiB/s supported for the volume type. This
// is currently only applicable to volumes of type gp3.
//
// +kubebuilder:validation:Minimum=0
Copy link
Contributor

@JoelSpeed JoelSpeed Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The minimum is 125 in the API, why would it be different here?

Copy link
Member

@tthvo tthvo Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it will be addressed in #10132

Copy link
Member Author

@jhixson74 jhixson74 Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The minimum is 125 in the API, why would it be different here?

I based this work directly off of how IOPS was implemented. IOPS makes all the same assumptions and works the same way as Throughput. So I (wrongly) assumed if everyone was okay with how IOPS was implemented, then it would be okay for Throughput.

Copy link
Member

@tthvo tthvo Dec 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I had the same thought as you said, John 😞

As a side note, the AWS SDK takes 0 as no-opinion; that's probably why CAPA allows 0 [0] [1].

I couldn't find any discussions regarding the IOPS implementation, but I guess we need to eventually follow the API...

// +optional
Throughput int64 `json:"throughput"`
Copy link
Contributor

@JoelSpeed JoelSpeed Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be a pointer so that you can detect between someone trying to explicitly set 0 vs no opinion

Copy link
Member

@tthvo tthvo Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Joel, I opened #10132 to address this.


// Size defines the size of the volume in gibibytes (GiB).
//
// +kubebuilder:validation:Minimum=0
Expand Down
20 changes: 20 additions & 0 deletions pkg/types/aws/validation/machinepool.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ func ValidateMachinePool(platform *aws.Platform, p *aws.MachinePool, fldPath *fi
if p.EC2RootVolume.Type != "" {
allErrs = append(allErrs, validateVolumeSize(p, fldPath)...)
allErrs = append(allErrs, validateIOPS(p, fldPath)...)
allErrs = append(allErrs, validateThroughput(p, fldPath)...)
}

if p.EC2Metadata.Authentication != "" && !validMetadataAuthValues.Has(p.EC2Metadata.Authentication) {
Expand Down Expand Up @@ -108,6 +109,25 @@ func validateIOPS(p *aws.MachinePool, fldPath *field.Path) field.ErrorList {
return allErrs
}

func validateThroughput(p *aws.MachinePool, fldPath *field.Path) field.ErrorList {
allErrs := field.ErrorList{}
volumeType := strings.ToLower(p.EC2RootVolume.Type)
throughput := p.EC2RootVolume.Throughput

switch volumeType {
case "gp3":
if throughput != 0 && (throughput < 125 || throughput > 2000) {
allErrs = append(allErrs, field.Invalid(fldPath.Child("throughput"), throughput, "throughput must be between 125 MiB/s and 2000 MiB/s"))
}
default:
if throughput != 0 {
allErrs = append(allErrs, field.Invalid(fldPath.Child("throughput"), throughput, fmt.Sprintf("throughput not supported for type %s", volumeType)))
}
}

return allErrs
}

// ValidateAMIID check the AMI ID is set for a machine pool.
func ValidateAMIID(platform *aws.Platform, p *aws.MachinePool, fldPath *field.Path) field.ErrorList {
allErrs := field.ErrorList{}
Expand Down
1 change: 0 additions & 1 deletion vendor/github.com/openshift/api/config/v1/types_node.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 17 additions & 1 deletion vendor/github.com/openshift/api/features/features.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 14 additions & 1 deletion vendor/github.com/openshift/api/machine/v1beta1/types_awsprovider.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions vendor/github.com/openshift/api/machine/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion vendor/github.com/openshift/api/machine/v1beta1/zz_generated.swagger_doc_generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading