Skip to content

Multi-cloud comm optimization #5

@camelop

Description

@camelop

Migrated from: https://github.com/camelop/dds-dev/issues/17

Idea

When considering decentralized data science in a multi-cloud setting, we might be able to do some system level optimization for existing abstractions. For example, consider two parties running a protocol, party A has a core running on cloud AC with storage on cloud AS1, AS2 and protocol operators running on cloud AP1, AP2; similarly party B has a core on BC, storage backends on BS1, BS2 and protocol operators BP1, BP2.

  1. In normal settings, we have AC=AS1=AS2=AP1=AP2, BC=BS1=BS2=BP1=BP2. In that case, there isn't much need for a multi-cloud optimization. Because if AC=BC, everyone is using the same cloud. If AC!=BC, there's no chance for data/execution migration towards optimization.
  2. However, if party A is a large company that owns abundant cloud resource, it might also be the case that AS1!=AS2 and AS2=BP1 (e.g. party A has both storage on AWS and GCP), with a upgraded core that supports operating across clouds, an internal migration before execution might be helpful to improve performance, because BP1 can directly take data from AS2.
  • It might also be the case that a party C want to provide a service to help A, B to accelerate their protocol execution. Serving as something similar to a proxy, for setting 1, there might be CS1=AS1 and CS2=BS1 which provides an opportunity for tunneling.

Note

@suquark might have some better ideas on this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cl-designNew design, research ideas, thoughts, or paper plans.enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions