Cloud Hub

Cloud Hub is a Java library that uses Grafana and helps you monitoring your Multicloud System, providing ready-to-use and configurable Dashboards.

You don't need to have any Grafana files installed on your system. The library is responsible to deploy the Grafana server folder for you and runs it on localhost. The only thing you need to do is to add the Dashboards of the services you want to monitor.

How it works: A compressed folder of a portable Grafana server instance is embedded in the Cloud Hub package. On startup the folder is moved to a temporary directory, decompressed and finally the grafana-server executable is started. When you configure and then add a new Dashboard to a Grafana instance, the library will automatically make the provisioning for you, creating its configuration files and placing them in the server directory.

Supported Systems

Currently Cloud Hub is developed on Ubuntu 19.04, but we are planning to support other Operating Systems. It should not be difficult since it's just a matter of resolving OS's dependencies (Grafana binaries files, process management, ...)

OS	Supported
Ubuntu	✔️
Windows	🔨
Mac OS	✖️

Supported Services

This is a list of the currently supported services you can monitor through the library:

Cloud Provider	Service Name
Amazon Web Services	Billing
	EC2
	Lambda
	S3
	SQS
Microsoft Azure	Functions
	Storage Container
	Storage Queue
	VMs

You can also run a simple HTTP Web Server to monitor the local system (CPU usage and temperature, Memory usage) in real time.

Dependencies

• Zip4j - to manage (essentially extract) the archive containing the Grafana server;
• JSON-java - to better manage the data exchange with Grafana for the Local Monitoring;
• jSensors - to get the CPU temperature.

Thanks to all the creators for their awesome work!

Quick Start

In order to monitor a specific service, you need to get a Grafana instance first specifying the web server port and a flag indicating whether or not to print the console logs:

Grafana grafana = new Grafana(SERVER_PORT, FLAG);

Then specify the appropriate Datasource of the service you want to monitor, e.g. CloudWatch for Amazon Web Services:

CloudWatchDataSource datasource = new CloudWatchDataSource("ACCESS_KEY", "SECRET_KEY", "DEFAULT_REGION");

Now you have to define a Dashboard compatible with the Datasource we created earlier, e.g. AWS Lambda:

AWSLambdaDashboard dashboard = new AWSLambdaDashboard("DASHBOARD_NAME", datasource);

You finally have to add the Dashboard to the Grafana instance in order to effectively deploy its configuration:

grafana.add(dashboard);

NOTE: optionally for this particular Dashboard, in order to hide the rest of the functions listed in the account, you can filter specific Lambda functions using their names in the method:

dashboard.addFunction("FUNCTION_NAME");

either before or after adding the Dashboard to Grafana. If you want to enable the Local Monitoring, you need to specify the port where the server is going to run and the delay that will be applied for collecting local data:

grafana.enableLocalMonitoring(API_PORT, COLLECTOR_DELAY);

Documentation

• grafana.Grafana
  • Constructor
  • start
  • addDashboard
  • removeDashboard
  • enableLocalMonitoring
  • disableLocalMonitoring
  • waitFor
• grafana.datasource
  • grafana.datasource.AzureMonitorDataSource
    • Constructor
  • grafana.datasource.CloudWatchDataSource
    • Constructor
• grafana.dashboard
  • grafana.dashboard.AWSLambdaDashboard
    • Constructor
    • addFunction
  • grafana.dashboard.AWSSQSDashboard
    • Constructor
    • addQueue
  • grafana.dashboard.AWSBillingDashboard
    • Constructor
  • grafana.dashboard.AzureFunctionsDashboard
    • Constructor
    • addFunction
  • grafana.dashboard.AzureQueueDashboard
    • Constructor

Grafana

It's the core of the library as it represents the Grafana server itself and it's used to control the dashboards.

Grafana(int httpPort, boolean consoleLog)

Deploys a new Grafana server instance in a temporary system folder.

• httpPort is the port where the Grafana Web interface will listen on
• consoleLog enable/disable the Grafana console logs

void start() throws IOException

Runs the Grafana server previously deployed.

void addDashboard(Dashboard dashboard)

Adds a dashboard to Grafana by deploying its configuration file.

• dashboard is one of the supported dashboards (which extends the abstract Dashboard class)

void removeDashboard(Dashboard dashboard)

Removes a dashboard from Grafana by deleting its configuration file.

• dashboard is one of the supported dashboards (which extends the abstract Dashboard class)

void enableLocalMonitoring(int apiPort, int collectorDelay)

Enables the local monitoring dashboard along with the HTTP API server that provides data to Grafana and the thread that collects the informations about the local resources.

• apiPort is the port of the HTTP API server
• collectorDelay is the time in milliseconds between one measurement and another

void disableLocalMonitoring()

Disables the local monitoring by removing its dashboard and stopping the collector thread along with the HTTP API server.

void waitFor() throws InterruptedException

Prevents the Grafana process to close when there are no more instructions left in Java (note that this method is blocking).

AzureMonitorDatasource

Represents the Azure Monitor Datasource and is responsible to create its configuration file for provisioning.

AzureMonitorDataSource(String tenantId, String clientId, String clientSecret, String defaultSubscription, String applicationId, String apiKey)

You need 4 pieces of information from the Azure Portal:

• tenantId (Azure Active Directory -> Properties -> Directory ID)
• clientId (Azure Active Directory -> App Registrations -> Choose your app -> Application ID)
• clientSecret (Azure Active Directory -> App Registrations -> Choose your app -> Keys)
• defaultSubscription (Subscriptions -> Choose subscription -> Overview -> Subscription ID)

You need 2 more pieces of information from the Azure Portal:

• applicationId
• apiKey

Please refer to this link to know how to get them. If you are still having problems, please refer to the Grafana documentation here.

CloudWatchDatasource

Represents the CloudWatch Datasource from AWS and is responsible to create its configuration file for provisioning.

CloudWatchDatasource(String accessKey, String secretKey, String defaultRegion)

Grafana needs permissions granted via IAM to be able to read CloudWatch metrics. So after creating the role needed you have to provide:

• accessKey
• secretKey

You can find more about that on the Grafana documentation here. If you need a quick guide have a look at this.

You also need to specify:

• defaultRegion (e.g. us-east-1)

AWSLambdaDashboard

Represents the Dashboard used to monitor the AWS Lambda functions.

AWSLambdaDashboard(String name, CloudWatchDataSource cloudWatchDataSource)

• name is a unique name used to identify the Dashboard. It is also its title in Grafana
• cloudWatchDataSource is the Datasource where to get the informations from

void addFunction(String functionName)

Adds a function to be monitored from the Dashboard.

• functionName is the name of the function you want to monitor

AWSSQSDashboard

Represents the Dashboard used to monitor the AWS SQS service.

AWSSQSDashboard(String name, CloudWatchDataSource cloudWatchDataSource)

• name is a unique name used to identify the Dashboard. It is also its title in Grafana
• cloudWatchDataSource is the Datasource where to get the informations from

void addQueue(String queueName)

Adds a queue to be monitored from the Dashboard.

• queueName is the name of the queue you want to monitor

AWSBillingDashboard

Represents the Dashboard used to monitor the AWS Billing service.

AWSBillingDashboard(String name, CloudWatchDataSource cloudWatchDataSource)

• name is a unique name used to identify the Dashboard. It is also its title in Grafana
• cloudWatchDataSource is the Datasource where to get the informations from

AzureFunctionsDashboard

Represents the Dashboard used to monitor the Azure Functions service.

AzureFunctionsDashboard(String name, AzureMonitorDataSource azureMonitorDataSource)

• name is a unique name used to identify the Dashboard. It is also its title in Grafana
• azureMonitorDataSource is the Datasource where to get the informations from

void addFunction(String functionName)

Adds a function to be monitored from the Dashboard.

• functionName is the name of the function you want to monitor

AzureQueueDashboard

Represents the Dashboard used to monitor the Azure Queue Storage service.

AzureQueueDashboard(String name, String resourceGroup, String resourceName, AzureMonitorDataSource azureMonitorDataSource)

• name is a unique name used to identify the Dashboard. It is also its title in Grafana
• resourceGroup is the name of the resource group where the storage account is located
• resourceName is the name of the storage account
• azureMonitorDataSource is the Datasource where to get the informations from

The only Azure Monitor limitation is that you can't filter the queue you want to monitor, but you will get all of them together. In order to overcome this limitation there is the Azure Service Bus which is not implemented yet.