No method to apply quota on storage tags

[alsko-icom]

problem

A domain admin can add a data disk to their instance depending on quota but not on tags. If I am tagging a primary storage as "sas 10k iops" and another as "sata 1k iops" and I am offerign a different price for each, the quota offered to the tenant has no way to understand which of the two tags (or storage classifications) it should use. It only knows it is primary storage. Is there any way to tackle this already in place that I am missing?

versions

Cloudstack management 4.20, KVM hosts based on redhat 8.

The steps to reproduce the bug

1. add tags to primary storages based on performance
2. set the tags to new storage offerings
3. create instance for a tenant based on specific storage tag
4. assign the VM to the tenant and add additional storage quota so he can add data disk
5. customer adds data disk based on any of the available offerings, even more expensive than the one originally purchased

What to do about it?

Incorporate TAGS in quota. Alternatively I was thinking maybe I can set the disk offering to not be "public" or choose specific domain for a disk offering. This way we will create too many offerings to assign wherever needed and we still will have no way to limit how many gigs are used per TAG. Only which domain has access to whichever offering for quota based on primary or secondary, not tag.

[DaanHoogland]

I have no idea if this is possible yet @GutoVeronezi @JoaoJandre do you gys know the answer to this one?

[alsko-icom]

I guess maybe if its not achievable, I am "thinking in a smaller box" in terms of charging a customer according to what quotas I have assigned to him and tried to limit him to use. Instead of the a better idea you already implemented which is the metering service that cloudstack offers and bills customers after using whatever. Maybe my use case methodology is "old-school/obsolete" and it is better to allow everything to everyone and charge them based on what they use?

[DaanHoogland]

It may be if charging means real money, I can see how your use-case could be useful for govermental or educational institutions, or maybe for enterprise clouds.

[bernardodemarco]

Hello, @alsko-icom

This way we will create too many offerings to assign wherever needed and we still will have no way to limit how many gigs are used per TAG. Only which domain has access to whichever offering for quota based on primary or secondary, not tag.

The Apache CloudStack 4.20.0.0 release introduced the concept of tagged resource limits that enables a granular management of the limit of resources consumption. Basically, the functionality allows operators to control domains and accounts limits of tagged resources usage.

The first step to configure tagged resource limits is by configuring the resource.limit.host.tags and resource.limit.storage.tags global settings. They expect a comma-separated list of tags to be used to manage the limits of host and storage resources usage, respectively. Next, when accessing an account's Configure limits tab, for instance, it'll be possible to configure the limits for VMs, CPU and memory based on the host tags defined in the global setting; and the limits for primary storage and volumes based on the storage tags defined in the global setting.

The functionality was introduced in the PR #8362 and its specification is available at this link. They provide more detailed use cases examples that could be extremely helpful.

---

Furthermore, I am not sure whether this fits in your use case/requirement, but just FYI Apache CloudStack has the Quota Plugin, which is a native plugin for quoting resources usage (assigning monetary value to computational resources consumption). To configure the Quota Plugin, it needs first to be globally enabled (quota.enable.service global setting). The plugin can also be enabled or disabled for accounts through the quota.account.enabled account-level configuration.

The Quota Plugin allows the operators to create tariffs that define monetary values to be assigned over the usage of cloud resources. For instance, it is possible to create a tariff of 5 credits per GB per month for allocated volumes. To achieve that, the quotaTariffCreate API needs to be executed, specifying the usagetype ¹ equal to 6 (VOLUME) and the value equal to 5:

quota tariffcreate name="5 credits per GB per month" usagetype=6 value=5 

The tariffs also support activation rules. These rules consist of logical expressions, written in JavaScript, which should evaluate to either a boolean or a numeric value. If the expression evaluates to a boolean, the tariff's value will be applied based on the result. If it evaluates to a numeric value, that value will directly become the tariff's value.

The activation rules support preset variables, which are pre-created variables that gives more flexibility to operators implement their business rules. Each resource type has a set of preset variables². The VOLUME and SNAPSHOT usage types, for instance, have the variable value.storage.tags, which represents the tags of the primary storage in which the volume or snapshot is allocated.

Therefore, in the scenario described in the issue, in which there are two primary storages, one tagged with sas 10k iops and another with sata 1k iops, a tariff of the VOLUME usage type could be created, with an activation rule similar to:

if (value.storage.tags.includes('sas 10k iops')) {
    5
} else if (value.storage.tags.includes('sata 1k iops')) {
    10
}

The following quotaTariffCreate API could be executed:

quota tariffcreate name="Custom tariff based on storage tags" usagetype=6 value=0 activationrule="if (value.storage.tags.includes('sas 10k iops') { 5 } else if (value.storage.tags.includes('sata 1k iops') { 10 }"

Therefore, if volumes are allocated to primary storages with the sas 10k iops tag, the customers will be charged 5 credits per GB per month. On the other hand, if volumes are allocated to primary storages with the sata 1k iops tag, then the customers will be charged 10 credits per GB per month.

Currently, the Apache CloudStack documentation lacks in-depth documentation for the Quota Plugin; we need to work on that. However, I strongly advise reviewing the PR that introduced this feature (#5909) along with its specifications (#5891). They provide more detailed examples on the usage and configuration of the Quota Plugin that could be extremely helpful.

Footnotes

1. The listUsageTypes API can be executed to retrieve the available usage types. ↩

2. The quotaPresetVariablesList API can be executed to retrieve the preset variables available for a given usage type. ↩