Support for per-role quote policies

[kostko]

Add support for per-role quote policies in addition to the existing general policy (which would become the "default" policy for roles that don't have an override). This would allow a more relaxed general policy but stricter requirements for nodes that can access the key manager (eg. compute/observer nodes).

[martintomazic]

So existing:

oasis-core/go/common/node/sgx.go

Lines 35 to 36 in aeffe3b

	// Policy is the quote policy.
	Policy *quote.Policy `json:"policy,omitempty"`

becomes the "default policy", and we add additional PerRolePolicy map[node.RolesMask]*quote.Policy?

This would allow a more relaxed general policy but stricter requirements for nodes that can access the key manager (eg. compute/observer nodes).

Who will be then validated against this "relaxed" policy given that we currently only register compute, observer or keymanager (that all access or are keymanagers)?

Given this changes consensus validation rules (breaking), I assume I have to gate it behind future version? Also obviously all struct changes should be backward compatible...

[kostko]

becomes the "default policy", and we add additional PerRolePolicy map[node.RolesMask]*quote.Policy?

Yes, something like that.

Who will be then validated against this "relaxed" policy

Clients (that do not register). And then a more strict policy for compute and observer nodes.

Given this changes consensus validation rules (breaking), I assume I have to gate it behind future version? Also obviously all struct changes should be backward compatible...

Yes. This is why the existing policy field is reused and only new fields are introduced. So you don't really need additional versioning and can just gate this extra "per-role policy" field being set on the feature (and be careful to fail in the same way as we do currently if a non-existent field is set).

[martintomazic]

Per-Role Quote Policies Design Doc

Status

Draft: Context is WIP, everything else is re-viewable, although subject to changes (planning to improve).

Context

Explain the flow...

CapabilityTEE

SGXConstraints contrains what is valid attestation. Part of it is also Policy that effectively restricts TCB/platform and therefore ensures our enclaves are run on the Intel certified platform, with up to date TCB.

Quote Policy verification / aplication

Where is currently Quote.Verify used?

Consensus

First use case is when we verify Capability.TEE. Here we take QuoteBundle and Policy and first verify raw local Quote and finally ensure the produced TCB matches the actual policy expectation. This verification is critical for sheduler committee election, keymanager status generation and registration of a new role (with concrete roles) to
the consensus registry.

The second place where Policy is used is inside pcs.ResolveQuote. Here the node is producing capability TEE. It first generarates raw Quote. Prior to registering a node with a capability TEE, we still need to fetch TCBBundle (so that other can verify us), and we want to make sure that such. Capability will actually pass consensus and later enclave validation so that transaction are not failing.

Runtime

Ad missing flow: EnclaveRPC, RHP initialization, endorsed capability TEE.

Motivation

This would allow a more relaxed general policy but stricter requirements for nodes that can access the key manager (eg. compute/observer nodes).

Currently, if you run a ROFL-node EnclaveRPC gates ROFL-RONL via the RONL quote policy. RONL policy is normally quite restrictive, as it is expected to access key-manager**. However, not every ROFL needs access to key-manager. Maybe ROFL only want to submit transaction using RONL, in which case lowering default RONL policy requirements makes a lot of sense.

** For RONL to access key-manager it needs to pass key-manager quote policy, so technically default RONL policy could already be loose? On the other hand, as mentioned, the new design opens possibility for RONL quote policy to define the constraints needed to safely access keymanager, thus off-loading responsibility of setting up sufficient TCB to RONL developer.

Requirements

• R1: Do we want different attestation age and possibly enclave identities,
  depending on the role?
  • E.g. if you need security critical patch for RONL, that is not relevant for client nodes.
  • If so we may want to to have per-role SGXConstraints
• R2: Should keymanager respect default or keymanager role or both?
  • I would say both.
• R3 What happens if per-role policy is less restrictive then default one?
  • It shouldn't matter, for every policy parameter the most restrictive part should hold.
• R4: While default and per role policies in isolation may be valid, a combination
  may be impossible to satisfy.
  • E.g. if whitelist intersection is empty, unless both whitelists are empty (accept anything),
    then the constrains as a whole are impossible to satify.
  • Such policies should be rejected from being registered in the first place. Do we really want to try out all combos??
• R5 Do we allow adding constraints for roles that don't have SGX attestion, e.g. validator or possible roles that don't exist?

Solution space

If we don't care about R1, then adding additional PerRolePolicy map[node.RolesMask]*quote.Policy to SGXConstraints makes most sense as it avoids versioning as agreed.

Challenges

Effective policy vs slice of policies.

The easiest way for clear semantic is to say: for the attestation to be valid it must
always satisfy the default policy. Moreover, depending on the role it needs to satisfy additional policies.

The most straightforward way to ensure this semantic is to pass around a slice o policies and validate
attestation against all policies. Alternative would be to generate the "effective policy" from the SGXConstraints (depending on the role), and pass it around from there on.

Whilst producing effective policy seems like a lot of overhead and merging is very error prone, passing around slice of policies is not much better as we would have to change a ton of function signatures. Furthermore, if we want to avoid validating same thing multiple times, we would have to merge this policies in the inner functions anyways. Additionally, if we want to prevent invalid constraints (C4), we might need effective policy helpers anyways.

Finally, if we want key-manager to respect both default and it's key-manager policy (C2), RuntimeKeyManagerQuotePolicyUpdateRequest accepts a single policy. Probably to make EnclaveRPC work with multiple methods would be breaking as well.

As much as I wanted to avoid effective policy, it feels like a simpler solution (far from clean).

RHP chicken and an egg problem

During initialization, or even later on, the enclave verifies the Quote produced by the QE (set_quote inside attestation.rs). This happens prior to the node being registered to the consensus, meaning the enclave cannot fetch the role, needed to validate the quote. Whilst it can get the roles from the host protocol, they cannot be trusted.
What is a threat model if host would lie about the role?

Getting roles to provisioner.

To avoid registering with invalid attestation, during pcs.ResolveQuote (add collateral for remote verification), we already verify against the target policy. The cleanest way would be to get it from the common.QuotePolicy, but we would need to add the roles to the host.Config. Current set of roles are part of the registry worker (role providers). One option is to get roles out of it, alternative would be to map node configuration (modes, and some other parameters) to the actual roles that node is currently registering for. Another option would be to move this check out of ResolveQuote and do the verification just before registering the node (we have capability, and roles there), however this wold require redundant parsing.

How to test

• Unit test for merging.
• Integration test for TeeCapability.Verify, possible mocking.
• e2e with sgx for the main use case?
  • Adding a new runner to the CI, with outdated TCB seems like an overkill, but what are alternative?

[peternose]

Can we simplify and say that whoever want's to register, it needs to always satisfy the default policy. Other policies can be ignored, even if the node supports one or more roles. However, when the node tries to do an action in the name of a role, the per-role policy is taken instead of the default one. The default one is overridden and not merged with the per-role policy. This way, specific roles can have striker or looser policies, and, you would always verify against one policy, and not against effective one which is not needed anymore.

[kostko]

I agree that we should not merge policies in this case (as this can result in strange side effects). I don't think however that we should allow nodes that don't satisfy the per-role policies to even register. To not over-complicate I think there are only two good ways to do it:

• The default policy always needs to be satisfied. Policies for each role need to all be satisfied as well.
• Policies for each role need to all be satisfied, iff a policy is not overriden for a role, the default policy must be satisfied for that role.

I think the second one is more flexible as with the first one you need to make sure that the default policy is loose enough.

For key managers, there are no sub-roles (all nodes that join the key manager runtime committee must be key managers), so there is no sense to support multiple policies there.

Regarding where to be flexible, we could be flexible at the SGXConstraints level as that has some additional fields (like max attestation age which is specific to our validation while the quote policy is specific to Intel's policies). On the other hand, it doesn't make much sense to support different enclave identities between roles. So maybe we differentiate on SGXConstraints but don't allow the Enclaves to be specified in these per-role policies?

[martintomazic]

I don't think however that we should allow nodes that don't satisfy the per-role policies to even register.

Agree, as the consensus should be source of truth for valid attestations, so that light clients don't have to verify, but instead trust via merkle proofs.

Policies for each role need to all be satisfied, iff a policy is not overriden for a role, the default policy must be satisfied for that role.

Yes this is probably way simpler to implement, at the cost of slight redundancy. We have to make assumption though, that a node can only have a single role. This is already not the case with validator and compute possibly coexisting. As validator is non-sgx we could ignore if runtime descriptor has a constraint for this role or possibly even reject registration of such descriptors. Can this change in the future for other (sgx-roles) in the future and possible bite us?

What would you do about Chicken and an egg problem? I.e. during initial RHP, set_quote calls set_quote_policy, however as the node is not yet registered yet, it cannot fetch the role and corresponding quote from the consensus. Of course it can ask the host about its role, like it asks for the node_id, but the host could lie about identity/role it will register for. Is this even critical, given that all rpc enclave calls, and keymagers also verify on their side the actor is respecting the constraints, so the malicious operator wouldn't benefit much here?

[kostko]

As validator is non-sgx we could ignore if runtime descriptor has a constraint for this role or possibly even reject registration of such descriptors

It is valid to have a validator with SGX support also being a compute node (eg. we should not reject such configurations). We could have a list of roles for which these policies are actually looked up (eg. observer and compute). A node that registers only as a validator cannot have any runtimes anyway (see ComputeRuntimeAllowedRoles).

however as the node is not yet registered yet, it cannot fetch the role and corresponding quote from the consensus

I think it is fine if the host specifies what roles it is registering for during the attestation flow. You could even make the runtime sign this information as part of the attestation (similar to the node ID), so if the host lied, the registration would fail. But not sure if this is critical.

[martintomazic]

It is valid to have a validator with SGX support also being a compute node (eg. we should not reject such configurations).

I meant whether we disallow adding quote policy for the validator role (sensible). Concretely should we prevent registering runtime descriptor with sgx constraints, where per-role quote policies also include validator role (which obviously doesn't make sense). This is nit, since it is expected from the runtime owner to set sensible constraints.

The point I am trying to make, unless you don't guarantee single role together with the second semantic you propose, you still don't get rid of the need for merging policies.

[kostko]

Yes, I would reject such invalid policy specifications.

[martintomazic]

Tried implementing it, so that we only support per-role policies for compute and observer roles. Anything else, as discussed above does not make sense. Moreover, explicitly validating valid roles for the per-role policies is desired, as it prevents undesired side effects a developer may accidentally introduce when adding new role to role mask.

There is still a need for merging quote policies though (and complex code) unless...:

Invariant: A node can have at most one of the runtime sgx roles at the time (compute, observer, keymanager).

Unfortunately, the protocol does not enforce such invariant (see TODO and test). In practice, with official binaries this is operationally impossible and I doubt there was ever such node descriptor registered. Still with no protocol guarantee of this invariant, it cannot be assumed.

I see two options:

1. Add this invariant under same feature flag.
   • Do a state-sync once feature flag is enabled and if no node descriptor with multiple runtime sgx roles was added in the meantime, we can safely remove respective code paths.
   • pros/cons: More constrains means simpler and easier code to reason about, but more problems later if we want to change this constraints, e.g. a new runtime sgx role that would coexist with another, both requiring different policy (unlikely).
2. Follow original second invariant:

Policies for each role need to all be satisfied, iff a policy is not overriden for a role, the default policy must be satisfied for that role.

• Here you do not get away from merging policies, since either you merge policies early and avoid changing many function signatures or you merge constraints inside internal functions. The result is the same, i.e. "this can result in strange side effects".
• If you are forced to do the merging of the policies anyways, then the only benefit of the second invariant compared to "The default policy always needs to be satisfied. Policies for each role need to all be satisfied as well." is that you don't have to answer question like "what if default policy is more restrictive then per-role" policy.