diff --git a/turret-hosts/anatomy-of-a-turret.md b/turret-hosts/anatomy-of-a-turret.md new file mode 100644 index 0000000..d6cbc5c --- /dev/null +++ b/turret-hosts/anatomy-of-a-turret.md @@ -0,0 +1,58 @@ +# Anatomy of a Turret + +Before you can host a turret, it is probably good to have an understanding of +what you're actually hosting. Here's a quick description of the important pieces +at work. + +> If you prefer a more technical explanation, or just want to follow along with +> these descriptions with real-world pieces, you can check out the [Reference +> Implementation][reference_implementation] + +## An API Application + +> If you're following along with the [reference_implementation], this is what is +> set up as the "cloudflare worker." + +The first thing you'll need is a place where `txFunctions` can be uploaded, managed, paid for, called for execution, and respond to requests. Additionally, this is where `txFunctions` are assigned a signing key and transactions can actually be signed by the turret. + +The bulk of the Stellar Turrets codebase is dedicated to this piece of the turret. This part receives and responds to API calls, + +When a user, application, or anybody wants to actually use an authored `txFunction`, this API application will take the following actions: + +1. Receives the request to run a specified `txFunction`, ensuring it includes a valid txFunction hash, has paid the turret fee for running, and is properly authenticated. + +2. Sends the `txFunction` code to the JavaScript environment (see below) to be executed in a safe, isolated sandbox, and receives back the generated XDR from the `txFunction`. + +3. Signs the transaction XDR from the previous step with the Keypair it has assigned for that particular `txFunction`. + +4. Returns to the initial requester the transaction XDR, a base64-encoded signature that can be added to the transaction, and the public key of the signer. + +## A JavaScript Environment + +> If you're following along with the [reference_implementation], this is the AWS +> lambda function that is setup through serverless.com. + +So, now we have a way to upload, interact with, and call our turret contracts. +The next thing we'll need is a place to actually execute that contract. (If +you're familiar with Ethereum smart contracts, this piece is what's known as the +Ethereum Virtual Machine (EVM): A runtime environment where the code of the +smart contract can run.) + +This environment is where the `txFunctions` will actually run and execute. It must +be able to run arbitrary JavaScript code (whatever a `txFunction` author has +written) in a safe way, and output a predictable result. + +This environment must also provide a consistent, reliable environment for +`txFunction` authors. The execution will take place in a `nodejs14.x` +environment, with the following default `node_module` packages installed: + +- [bignumber.js] +- [node-fetch] +- [stellar-sdk] +- [lodash] + +[reference_implementation]: https://github.com/tyvdh/stellar-turrets/blob/master/README.md +[bignumber.js]: https://www.npmjs.com/package/bignumber.js +[node-fetch]: https://www.npmjs.com/package/node-fetch +[stellar-sdk]: https://www.npmjs.com/package/stellar-sdk +[lodash]: https://www.npmjs.com/package/lodash diff --git a/turret-hosts/reference-implementation.md b/turret-hosts/reference-implementation.md new file mode 100644 index 0000000..308bc0d --- /dev/null +++ b/turret-hosts/reference-implementation.md @@ -0,0 +1,223 @@ +# Stellar Turrets Reference Implementation + +This Stellar Turrets reference implementation employs two serverless services. +Cloudflare workers and an AWS lambda function. The reason for this is that +txFunctions are by their nature unsafe arbitrary Javascript functions. +Cloudflare doesn't allow the execution of such functions thus we're splitting +the workload between the much more performant and affordable Cloudflare workers +and an AWS lambda function which will serve as our txFunction execution +environment. + +See below for specific instructions for setting up and running both services. + +> The Stellar Turrets Network is currently in the very early stages of +> deployment on the Stellar mainnet. Doing so entails setting up two turret +> environments for the testnet and mainnet, respectively. The process is done in +> pretty hands-on cooperation with the project maintainers at this point in +> time. For more information, please have a look at this [pull-request] on the +> `stellar-turrets` repository. + +## Wrangler (Cloudflare) + +If you haven't already go ahead and [signup for Cloudflare workers][1]. You can +attempt to run on their free tier but I highly suggest just biting the very +affordable bullet and upgrading to their $5/mo plan which will allow you to +scale much more nicely. + +Next generate the `.env` and `wrangler.toml` files from their templates. Just +run `npm run init` inside the `wrangler` subdirectory. The script will ask for a +bunch of values, which you'll need at hand in order to successfully finish the +init step. If there are default-values, they will be prefilled (e.g. +`HORIZON_URL` defaults to the test net). If you don't have the value at hand +(i.e. you can only run the wrangler cli to create KV namespaces (step 2 below) +after providing `WRANGLER_ACCOUNT_ID`) just hit enter. You can repeat this step +as often as you want to. Variables in `.env` that already have values assigned +won't be updated again by this step. + +1. For the `account_id` go to the workers page on [dash.cloudflare.com][1] and + copy your `Account ID`. +2. For `kv_namespaces` create three new kv namespaces via the wrangler cli: + + ``` + $ npx wrangler kv:namespace create "META" + $ npx wrangler kv:namespace create "TX_FUNCTIONS" + ``` + + Each of those commands will spit out the object you should use to provide in + the init-step. + +3. Finally for `vars` set `STELLAR_NETWORK` to either `TESTNET` or `PUBLIC` to + toggle this Turret between using either the Test or Public Stellar network + passphrases. For `HORIZON_URL` place in the url for the horizon service your + Turret will consume. This should match with either the Test or Public network + passphrase which you just set for the `STELLAR_NETWORK` variable. For + `TURRET_ADDRESS` just use any valid, funded, Stellar account you privately + own. This is the account into which fees will be paid as txFunctions are + uploaded and run on your Turret. Next set `TURRET_RUN_URL` to `null` for now + until we've got the Serverless AWS lambda setup with it's endpoint, at which + point you'll update this value to that url. Finally set the `XLM_FEE_MIN`, + `XLM_FEE_MAX`, `UPLOAD_DIVISOR`, and `RUN_DIVISOR` values to reasonable + defaults. + + Now that the `wrangler.toml` file has been created let's move to the + `stellar.toml` file. This file is served as your Turret's `stellar.toml` file. + Particularly note the `[TSS].TURRETS` array; this will be an array of other + Turret addresses that you trust to cohost txFunctions with in the case of + txFunction healing. For now just make sure to include your own + `TURRET_ADDRESS` which should be the first entry. There are already a few + other turrets in the file as well. You can add or remove those entries to your + liking/trust. + + Once you've got that go ahead and upload it to the `META` kv store you + instantiated earlier. + + ``` + $ npx wrangler kv:key put --binding=META "STELLAR_TOML" ./stellar.toml --path + ``` + + Make sure to run these wrangler commands from the `./wrangler` directory + + Finally to deploy the project run: + + ``` + $ npm i + $ wrangler publish --new-class TxFees + ``` + + From within the `./wrangler` directory. + + You may have to work through a few errors to get logged into your Cloudflare + account but the wrangler cli errors are typically quite helpful. Feel free to + update this README with more clear instructions as it's been ages since I + started from scratch on my first wrangler project. + +4. Once you've successfully got your project created and running upload a + `TURRET_SIGNER` Stellar secret key to your Cloudflare worker. + + ``` + $ npx wrangler secret put TURRET_SIGNER + ``` + + When the dialog asks your for a value paste in a valid Stellar **secret key**. + Most often this will be the secret key counterpart to your `TURRET_ADDRESS` + but this isn't a requirement. This key is used to authenticate requests + between your Cloudflare and Serverless services, nothing else. + +5. Whenever you need to redeploy the project in the future either run + + ``` + $ npm run deploy + ``` + + or + + ``` + $ wrangler publish + ``` + + The `--new-class TxFees` you included in the first deploy was just an + initializer argument for the `TxFees` Durable Object. Once it's run + successfully once it will fail if included again. + +## Serverless (AWS) + +Next we have the Serverless lambda endpoint which is hosted with AWS but +deployed using the far more sane [serverless.com][2] cli tool. If you haven't go +create both an [AWS console account][3] and a [serverless.com account][4]. Once +you have those setup ensure you've got the [serverless cli installed][5]. + +Now it'll be the fun task of getting: + +``` +$ yarn +$ npm run deploy +``` + +To successfully run from within the `./serverless` directory. + +You will be prompted for `SLS_ORG`, `TURRET_BASE_URL` and +`TURRET_SIGNER_ACCOUNT`. The first one must match your organization you use in +serverless to deploy the lambda. `TURRET_BASE_URL` needs to reflect the worker +base url where your wrangler service is hosted on. The `TURRET_SIGNER_ACCOUNT` +needs to be set to your Turret's `TURRET_SIGNER` **public key** that you have +set in the Wrangler setup step before. + +This connection is what secures and protects access between my Cloudflare and +Serverless APIs. Remember Cloudflare gets the **private key** and Serverless +gets the **public key**. + +Follow any errors carefully and you should be able to get successfully deployed +pretty quickly. The most probable issue will be you need to manually create an +app in the [Serverless dashboard][6] and attach some new IAM credentials to it +manually. There's a helpful UI walk through they have so you should be able to +sort it out. Again feel free to update these docs with more clear instructions +as you sort out the nuances of setting the Serverless service up. + +*If* you are going to use the github action to deploy serverless, you need to +*set the three variables from above as repo-secrets as well. + +When you finally get success on this task you'll be rewarded with an endpoint +where your function is hosted. Copy that base url and paste it as the value for +the `TURRET_FUNCTION_RUNNER_URL` `var` back in the `.env` file in the +`./wrangler` directory and run `npm run init && npm run deploy` again to update +the worker to point to your lambda. + +## Congrats! + +Assuming both `npm run deploy`'s are now firing off without a hitch you should +have a fully functional Turret ready to participate in the Stellar Turrets +network delivering decentralized smart contracting functionality to anyone and +everyone who chooses to use your Turret. Nice! + +## CI (github actions) + +There are GH actions defined to actually deploy the serverless and wrangler +parts continously. For this to work add the following secrets to your (cloned) +repo: + +| VARIABLE | Description | Default | +| --------------------------- | ------------------------------------------------------------------------------- | --------------------------------------- | +| *STELLAR_NETWORK* | the network to use | **TESTNET** | +| *HORIZON_URL* | URL of horizon server. | **https://horizon-testnet.stellar.org** | +| TURRET_ADDRESS | Existing and funded stellar address to receive fees | | +| TURRET_FUNCTION_RUNNER_URL | base URL of function runner (serverless part from above) | null | +| WRANGLER_ACCOUNT_ID | ID of cloudflare account to deploy wrangler to | | +| WRANGLER_API_TOKEN | Token for cloudflare API access | | +| *WRANGLER_WORKER_NAME* | name of your worker inside cloudflare | **tss-wrangler** | +| WRANGLER_META | KV namesapace for worker's META information | | +| WRANGLER_TX_FUNCTIONS | KV namespace for worker's functions | | +| *WRANGLER_XLM_FEE_MIN* | The minimum claimable fee balance allowed. See [turret_info] | **1** | +| *WRANGLER_XLM_FEE_MAX* | The maximum claimable fee balance allowed. See [turret_info] | **10** | +| *WRANGLER_UPLOAD_DIVISOR* | The divisor used in fee calculations for uploading functions. See [turret_info] | **1000** | +| *WRANGLER_RUN_DIVISOR* | The divisor used in fee calculations for running functions. See [turret_info] | **100000** | +| SLS_ORG | The serverless organization to deploy to | | +| SLS_TURRET_BASE_URL | Base URL of turret (wrangler URL) | | +| SLS_TURRET_SIGNER_ACCOUNT | Public key of function signer (counterpart to turret's **private** signer key) | | +| *SLS_AWS_PLAN* | Indicate if you are using a free or paid plan | **free** | +| SERVERLESS_ACCESS_KEY | Token for SLS API access | | + +optional values are *italic* (i.e. will be using defaults if not set) + +## [API Docs][7] + +## Disclaimer + +This is alpha software representing a reference implementation for the [Stellar +Turrets protocol][8]. + +For this reason I strongly suggest either: + +1. Leaving your `STELLAR_NETWORK` set to `TESTNET`, or +2. Encouraging users to leave themselves as a majority signer on any controlled + account they're attaching Turret signers to + +[pull-request]: https://github.com/tyvdh/stellar-turrets/pull/36 +[turret_info]: https://tyvdh.github.io/stellar-turrets/#operation/getTurretInfo +[1]: https://dash.cloudflare.com/ +[2]: https://serverless.com +[3]: https://aws.amazon.com/ +[4]: https://www.serverless.com/dashboard/ +[5]: https://github.com/serverless/components#quick-start +[6]: https://app.serverless.com/ +[7]: https://tyvdh.github.io/stellar-turrets/ +[8]: https://tss.stellar.org/