Refactor C2D docs. #1513

mariacarmina · 2025-05-12T18:19:13Z

Fixes #1515 .

Changes proposed in this PR:

adding sequence flow diagrams for free + paid compute
explainin the entire free and paid flow diagrams
refactor existing documentation for c2d
create documentation for escrow contract
update code example for c2d in ocean.js

…xists.

giurgiur99

Some small changes 🙏

giurgiur99 · 2025-05-23T06:14:16Z

.gitbook/assets/vscode/setup.png

Can we update with latest screenshot without paid resources? Thanks.

giurgiur99 · 2025-05-23T06:16:04Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+In the C2D workflow, the following steps are performed:
+
+1. The consumer initiates a compute-to-data job by selecting the desired data asset and algorithm, and then, the orders are validated via the dApp used.
+2. A dedicated and isolated execution pod is created for the C2D job.


Container not pod for now

giurgiur99 · 2025-05-23T06:17:11Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+3. The execution pod loads the specified algorithm into its environment.
+4. The execution pod securely loads the selected dataset for processing.
+5. The algorithm is executed on the loaded dataset within the isolated execution pod.
+6. The results and logs generated by the algorithm are securely returned to the user.


We can mention that we use a web3 auto-generated/custom PK and this is how we ensure private access to results

we can mention the signature generation that it is required on get results endpoint.

giurgiur99 · 2025-05-23T06:18:00Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+
+Now, let's delve into the inner workings of the Provider. Initially, it verifies whether the Consumer has sent the appropriate datatokens to gain access to the desired data. Once validated, the Provider interacts with the Operator-Service, a microservice responsible for coordinating the job execution. The Provider submits a request to the Operator-Service, which subsequently forwards the request to the Operator-Engine, the actual compute system in operation.
+
+The Operator-Engine, equipped with functionalities like running Kubernetes compute jobs, carries out the necessary computations as per the requirements. Throughout the computation process, the Operator-Engine informs the Operator-Service of the job's progress. Finally, when the job reaches completion, the Operator-Engine signals the Operator-Service, ensuring that the Provider receives notification of the job's successful conclusion.


No kubernetes for now

This is old stack, will remove these lines.

giurgiur99 · 2025-05-23T06:19:02Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `GetComputeEnvironments` - returns list of environments that can be selected to run the algorithm on
+- `InitializeCompute` - generates provider fees necessary for asset's ordering
+- `FreeStartCompute` - runs algorithms without necessary publish the assets on-chain (dataset and algorithm), using free resources from the selected environment
+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.


startCompute

COMPUTE_START: 'startCompute',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:19:33Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `FreeStartCompute` - runs algorithms without necessary publish the assets on-chain (dataset and algorithm), using free resources from the selected environment
+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.
+- `ComputeGetStatus` - retrieves compute job status.
+- `ComputeStop` - stops compute job execution when the job is `Running`.


COMPUTE_STOP: 'stopCompute',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:19:44Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.
+- `ComputeGetStatus` - retrieves compute job status.
+- `ComputeStop` - stops compute job execution when the job is `Running`.
+- `ComputeGetResult` - returns compute job results when job is `Finished`.


COMPUTE_GET_RESULT: 'getComputeResult',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:20:25Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+
+One of its responsibility revolves around fetching and preparing the required assets and files, ensuring a smooth and seamless execution of the job. By meticulously handling the environment configuration, the **C2D Engine** guarantees that all necessary components are in place, setting the stage for a successful job execution.
+
+1. **Fetching Dataset Assets**: It downloads the files corresponding to datasets and saves them in the location `/data/inputs/DID/`. The files are named based on their array index ranging from 0 to X, depending on the total number of files associated with the dataset.


Dataset can be did/url/arweave/ipfs

And also need to specify for the algorithm as well, good catch!

giurgiur99 · 2025-05-23T06:21:30Z

developers/compute-to-data-v2.0/free-compute-to-data-flow.md

+
+## Prerequisites
+
+The prerequisite for this flow is the algorithm code which can be input for consumers components: Ocean CLI and it is open for integration with other systems (e.g. Ocean Enterprise Marketplace).


CLI and vscode extension, might be useful to add links

There are linked in Setup , but I will link as well in the Prerequisites, thank you.

giurgiur99 · 2025-05-23T06:24:26Z

developers/vscode/README.md

New dataset here will be url/did/ipfs/arweave.

We will now use the c2d_examples not algo_dockers repo

C2D refactor for docs.

26d91a8

mariacarmina self-assigned this May 12, 2025

mariacarmina added 5 commits May 22, 2025 11:00

Updated architecture document for C2D v2. Updated flow diagrams.

df3ae44

detail free compute flow.

79f2f76

Updated flow diagrams and finished detailing free compute flow.

9decebd

Add content for paid compute detailed flow.

feb2b83

Finished paid compute flow. Added introduction for escrow contract.

56dd872

mariacarmina marked this pull request as ready for review May 22, 2025 20:28

mariacarmina added 2 commits May 22, 2025 23:49

Update escrow content. Deeleted compute file from ocean.js, already e…

46d8d8d

…xists.

Updated old compute example for ocean.js with the paid compute version.

7cbd13a

mariacarmina changed the title ~~C2D refactor for docs.~~ Refactor C2D docs. May 22, 2025

giurgiur99 reviewed May 23, 2025

View reviewed changes

mariacarmina added 7 commits May 29, 2025 07:51

Add API specification for compute stack from Ocean Node.

4790083

Updated payloads.

b972e97

Added payloads examples.

14a4c8b

Fix review.

6e6f9c5

Updated formats for ddo.

c739649

Add links.

1ab0b90

Fix tweaks.

f4e5812


		Now, let's delve into the inner workings of the Provider. Initially, it verifies whether the Consumer has sent the appropriate datatokens to gain access to the desired data. Once validated, the Provider interacts with the Operator-Service, a microservice responsible for coordinating the job execution. The Provider submits a request to the Operator-Service, which subsequently forwards the request to the Operator-Engine, the actual compute system in operation.

		The Operator-Engine, equipped with functionalities like running Kubernetes compute jobs, carries out the necessary computations as per the requirements. Throughout the computation process, the Operator-Engine informs the Operator-Service of the job's progress. Finally, when the job reaches completion, the Operator-Engine signals the Operator-Service, ensuring that the Provider receives notification of the job's successful conclusion.


		One of its responsibility revolves around fetching and preparing the required assets and files, ensuring a smooth and seamless execution of the job. By meticulously handling the environment configuration, the C2D Engine guarantees that all necessary components are in place, setting the stage for a successful job execution.

		1. Fetching Dataset Assets: It downloads the files corresponding to datasets and saves them in the location `/data/inputs/DID/`. The files are named based on their array index ranging from 0 to X, depending on the total number of files associated with the dataset.


		## Prerequisites

		The prerequisite for this flow is the algorithm code which can be input for consumers components: Ocean CLI and it is open for integration with other systems (e.g. Ocean Enterprise Marketplace).

Refactor C2D docs. #1513

Are you sure you want to change the base?

Refactor C2D docs. #1513

Conversation

mariacarmina commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giurgiur99 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mariacarmina commented May 12, 2025 •

edited

Loading