Skip to content

Commit

Permalink
README|CONTRIBUTING: Improved the docs
Browse files Browse the repository at this point in the history
Pointing to the right specific one and also updating some old information
  • Loading branch information
xescugc committed Jan 18, 2022
1 parent 41bf0cb commit 198fcee
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 12 deletions.
54 changes: 45 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,19 +46,55 @@ Run all the tests:
$ make test
```

## General information

### What is a SKU?

SKU means Stock Keeping Unit (SKU) and it is and identifier such as a number or a (bar-)code for a single item in a retail company’s range. The SKU enables a product to be clearly identified and is important for a retailer’s inventory management.

So each SKU it's a different product which has a related price.

### What does it mean to calculate the cost of a resource

We have to make a link between the Cloud Provider SKU and the configuration attributes of the resource, either HCL or the Plan.
This link is done manually by knowing what the Cloud Provider is pricing for and how it's declared on Terraform, on the `aws_instance` case for example:

* `instance_type`(`"t2.micro"`): Which at the end is a number of CPU+RAM of the instance
* `tenancy`(`"dedicated"`): Tenancy of the instance

We track other attributes and combination of them, for example the `root_block_device` configuration block is
mapped to a `aws_ebs_volume` so at the end we have a total price which is a composition of other blocks.

We also take sometimes assumptions that later on may be parametrized, in this case we assume that it's always using a Linux OS

### Importing

The importing process is done by reading the information for the Cloud Provider API (or file) and then reduce it down to a `product.Product` and `price.Price` which
are the main entities we store on the DB and query to get the Prices. Both of them have an `Attributes` which is a `map[string]interface{}` which is used
to store specific information that is unique for the Cloud Provider than then we can use to make dedicated queries when fetching the prices.

To minimize the import volume we always import filter by a Service and a Region/Location/Zone (each Cloud Provider has it's own names) and we also have a `{provider}.MinimalFilter` which would only import from that previous filter only the elements we know and use at the moment, this is useful to not flood the DB with unnecessary prices. This options are used when initializing the specific Importer.

## Adding a new resource

### AWS

1. Familiarize yourself with the official AWS pricing page for the service as well as the Terraform documentation for the resource you want to add. Note all factors that influence the cost.
2. Download and familiarize yourself with the pricing data CSV. This can be done by first checking the [index.json](https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json), finding the respective service under the `offers` key and downloading the file at the URL under the `currentVersionUrl` (replace `json` with `csv`).
3. Find the names of all columns that contain relevant cost factors and check that the `aws/field/field.go` file contains them - add them if this is not the case. The constant name should be a correct Go identifier, while the comment should contain the name as it appears in the CSV file.
4. Run `go generate ./...` to regenerate the field list.
5. Create a new file with the name of the Terraform resource (without the `aws` prefix), e.g. for `aws_db_instance` it would be `db_instance.go`. It should include two new structs: `Resource` (that is an intermediate struct containing only the relevant cost factors) and `resourceValues` (that directly represents the values from the Terraform resource.) Additionally, the `Resource` struct must implement the `Components` method that returns `[]query.Component`. See the other existing resources for inspiration.
6. Write tests for your resource. As before, check the other existing test files for inspiration.
7. Test and make sure that estimating your resource works.
8. Open a PR with the changes and please try to provide as much information as possible, especially: description of all the cost factors that the PR uses, links to Terraform docs and AWS pricing page, examples of a Terraform file and the resulting estimation.
To add more AWS resources first read the [documentation](docs/aws.md) we have about it.

### Google

To add more Google resources first read the [documentation](docs/google.md) we have about it.

### AzureRM

To add more AzureRM resources first read the [documentation](docs/azurerm.md) we have about it.

## Testing resources

All the providers use API to get the needed data from the prices so we can ingest it, the way we use for testing is to mock that API call by being able to initialize
the provider with a different endpoint (on Azure we need to use `"https://prices.azure.com/"`) and on the test we just create a new [`httptest.NewServer`](https://pkg.go.dev/net/http/httptest#NewServer) and mock the needed endpoints. This implementations are in `testutil/` and the data that they return on `testdata/`. With that we can mock
any API call we need and expect the same results.

## Adding a new provider/backend

**Please be aware that, at the moment, Cycloid only supports MySQL as a backend and AWS as cloud provider.** Based on this, please refrain from making contributions that add a new backend or cloud provider as we cannot guarantee they'd be merged and/or supported. To make improvements in this area, please instead open an appropriate issue so that we can discuss it and provide any necessary guidance.
**Please be aware that, at the moment, Cycloid only supports MySQL as a backend.** Based on this, please refrain from making contributions that add a new backend or cloud provider as we cannot guarantee they'd be merged and/or supported. To make improvements in this area, please instead open an appropriate issue so that we can discuss it and provide any necessary guidance.
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,17 @@ go get github.com/cycloidio/terracost

## Requirements

- Go 1.15 or newer (older may work but are not supported)
- Go 1.16 or newer
- MySQL database

## Provider support

Currently, Terracost supports only a few resources of the AWS provider, however we're actively working on adding more support. See the full list of supported resources [on this wiki page](https://github.com/cycloidio/terracost/wiki/Supported-Resources).
Currently Terracost supports the following providers and with an specific subset of resources from them:
* [AWS](./docs/aws.md#list-of-supported-resources-and-attributes)
* [Google](./docs/google.md#list-of-supported-resources-and-attributes)
* [AzureRM](./docs/azurerm.md#list-of-supported-resources-and-attributes)

### Google
### Google Credentials

To be able to ingest the pricing data from Google the credentials needed have to have access to [Compute Engine API](https://console.cloud.google.com/apis/library/compute.googleapis.com) and have also billing enabled. This is needed to be able to fetch the Machine Types, for the SKUs we would only need an normal set of credentials (or even API Key) but as we need both we have to use the more complex one. If you do not know how to activate those permissions, just use the credentials and the import will fail and on the logs will inform of what is missing and how to activate it.

Expand Down

0 comments on commit 198fcee

Please sign in to comment.