-
Notifications
You must be signed in to change notification settings - Fork 1
OECC demo #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
OECC demo #3
Changes from all commits
51e722a
46276ba
fa2bc32
47fbfeb
87cf439
34760a7
7a73510
fe4b905
37998c5
84a82fc
3f5a496
6261fa7
58f7346
7edc1b7
35d880d
e81e3a4
f71e8dc
a6ced45
def3c44
d224b9b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| # Kafka Consumer/Producer # | ||
|
|
||
| To run `./comsume.ch` | ||
| To run `./comsume.sh` | ||
|
|
||
| To run `./producer.sh` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| # OpenEdge Command Center # | ||
|
|
||
| Demo scripts to setup an OpenEdge Command Center environment with support for OpenTelemetry metrics. | ||
|
|
||
| The scripts are an updated version of the scripts used for the following blog post: | ||
| * https://www.progress.com/blogs/using-opentelemetry-metrics-support-openedge-azure | ||
|
|
||
| In the new version, the scripts can be used on an Ubuntu VM running on various virtualization options such as AWS, Azure, VirtualBox or VMWare Fusion. | ||
|
|
||
| Scripts run commands to detect the IP address of the machine. | ||
|
|
||
| ### Requirements | ||
| * Ubuntu Linux 22.04 | ||
| * user with sudo access | ||
| * OpenEdge media: | ||
| * PROGRESS_OE_12.8.3_LNX_64.tar.gz | ||
| * PROGRESS_OECC_SERVER_1.3.0_LNX_64.tar.gz | ||
edselg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * PROGRESS_OECC_AGENT_1.3.0_LNX_64.bin | ||
| * Reponse file for OpenEdge installation: | ||
| * response_12.8.3.ini | ||
| * Products: | ||
| * OE Adv. Ent. RDBMS | ||
| * 4GL Development System | ||
| * Progress ProdAS for OE | ||
|
|
||
| **Notes:** | ||
| * The OpenEdge media for both OpenEdge and OpenEdge Command Center can be downloaded from the Progress OpenEdge 12.8.x section in the ESD site. | ||
| * You can use the `OpenEdge-Samples/examples/OECC/config/response_12.8.3.ini` file as a template and specify the control codes in the file. | ||
| * The demo scripts install the components using the media and the response .ini file. | ||
| * The demo scripts install OpenJDK to `/usr/lib/jvm/jdk` and OpenEdge to `/psc/dlc`. | ||
| * The demo scripts install Docker Engine on Ubuntu using the instructions at https://docs.docker.com/engine/install/ubuntu/. | ||
|
|
||
| ### Components | ||
| The infrastructure uses multiple components. To simplify the deployment, it uses a single virtual machine with some components running at the OS level and others running as a container using Docker. | ||
|
|
||
| * Ubuntu Virtual Machine latest 22.04 (tested 22.04.4) | ||
| * OpenEdge Database 12.8.3 | ||
| * PAS for OpenEdge 12.8.3 | ||
| * OpenEdge Command Center agent 1.3.0 | ||
| * Docker (latest, version tested: 26.1.4) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How should a user install Docker for Ubuntu? Could we add a link to something like this page or embed some instructions? https://docs.docker.com/engine/install/ubuntu/ Asking as my Ubuntu installation came with Docker 20.0.5 but following those instructions got me up to version 27.0.2 so it may be worth stating that as a reference for those people unfamiliar with Docker.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Calling ./scripts/setup.sh would call ../common/scripts/install_docker.sh which performs the installation of Docker on Ubuntu.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was honestly not aware of that, and apparently did not look that closely at the setup.sh script to realize. This is why I've been incredibly pedantic during the code review and this is exactly what has stopped me from proceeding further than I would have liked. There are a LOT of assumptions made about the skillset of the user, familiarity with the technology, and perhaps some details missed because you are writing as the subject matter expert. It's easy to miss things or take for granted what you already know. I would ask that you take another look at the opening of the main README file, specifically the Requirements and Components sections to add notes about where each should come from. For example, for the OpenEdge media, state those should come from the ESD site, and where they might be found. For the components such as Docker, Grafana, etc. state that those are installed by the scripts if that's the case--otherwise, explain how to manually install them (apt-get, etc.) so users know they need to have them installed before proceeding.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The intention of the demo scripts is to provide something that reduces the complexity of the installation process of OpenEdge Command Center and also gives the user the option to try the OpenTelemetry metrics support. I am hoping to write a new version of the blog post. Please try running ./scripts/setup.sh. Thanks.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As someone who has had little exposure to any of this, I'm now curious how much is OECC and how much is for OTel? Would this have been better to demonstrate as 2 separate examples? OECC should just be OECC, OTel just OTel. By mixing components it's difficult to know which is required or optional for their respective technology. We already have a separate Kafka folder that demonstrates just that technology--why mix these concepts under OECC?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If feel that we can go either way on this. The idea of having both together was because, at the time that the initial scripts were written and perhaps still today, OpenTelemetry is one of the most important features when using OpenEdge Command Center. It might be likely that users exploring OpenEdge Command Center (server) would also be interested on taking a look at the OpenTelemetry support. A demo for OpenTelemetry only needs the configuration of OpenEdge Command Center agent. Thanks.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If I were a new looking at this folder as just "OECC" I would assume that these are all required and normal components. Having to install several extra tools to get OECC running without differentiation makes this feel more complicated than it should be. You already have a "common" scripts folder, so why not make these subtopics which isolates learning?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where are you expecting these subtopics? The current README.md file has a "Creating a Dashboard in Grafana" as a separate section on the demo. I can add an optional section to the README.md on "Using OpenTelemetry Support in OpenEdge" and leave the core of the README.md to be OpenEdge Command Center. Please confirm. Thanks.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where did this work originate? Do we have a ticket that explains the purpose of the work, acceptance criteria, etc? Or is this all just off-the-cuff? We should really be tracking the work and should have discussed what goes into these examples as a matter of agreeing on the content. The more I'm diving into this the more confused I am over why I would need Docker and all these other tools just to install something that already has installer files. The OECC itself would be on a central server/location while the agent would be installed on multiple client machines, so it's hard to visualize using this in a distributed manner. If the need for Docker is so the instance can be easily encapsulated, then there should be a Docker container for the OECC (server) component while a separate container with a PASOE instance and the agent would be installed together. That way it could demonstrate how to set up each side of the solution. This could all be a single document that is the OECC example itself--nothing more. The addition of OTel, Grafana, etc. just seem extraneous and should be it's own topic for after you've successfully set up OECC to monitor a PAS instance. That way the user could deploy the things needed to each side--either the server side of the components or the agent-monitored instances. |
||
| * OpenTelemetry Collector 0.53.0 | ||
| * Prometheus (latest, version tested: v2.53.0) | ||
| * Grafana (latest, version tested: 9.5.20) | ||
| * MongoDB 4.4-rc-focal | ||
| * OpenEdge Command Center server 1.3.0 | ||
|
|
||
| The OpenEdge Database, PAS for OpenEdge and the OpenEdge Command Center agent are installed at the OS level. | ||
| The OpenTelemetry Collector, Prometheus, Grafana, MongoDB, and the OpenEdge Command Center server are deployed using Docker. | ||
|
|
||
| The following diagram shows the components use for OpenTelemetry support. | ||
|  | ||
|
|
||
| See related diagram at | ||
| https://www.progress.com/openedge/features/command-center | ||
|
|
||
| ### Steps | ||
|
|
||
| #### Preparing the installation with the prerequisites | ||
| 1. Create `/install` directory used for the installation files. | ||
| ~~~ | ||
| sudo mkdir -p /install | ||
| sudo chown $USER /install | ||
| ~~~ | ||
|
|
||
| 2. Copy the OpenEdge media files and the response file to `/install`. | ||
|
|
||
| #### Setting up the environment | ||
|
|
||
| 1. Clone the OpenEdge-Samples repo: | ||
| ~~~ | ||
| cd | ||
| git clone https://github.com/progress/OpenEdge-Samples.git | ||
| ~~~ | ||
|
|
||
| 2. Run `setup.sh` in the `OECC/scripts` directory. Script uses `sudo` to perform administrator tasks. | ||
| ~~~ | ||
| cd ~/OpenEdge-Samples/examples/OECC | ||
| ./scripts/setup.sh | ||
| ~~~ | ||
|
|
||
| #### Testing the environment | ||
|
|
||
| 1. Test the configuration. | ||
| ~~~ | ||
| ./scripts/test.sh | ||
| ~~~ | ||
|
|
||
| 2. Access OpenEdge Command Center via a web browser: | ||
| * http://<host-machine>:8000 | ||
| * Use admin/admin to login | ||
|
|
||
| 3. Access the PAS for OpenEdge instance via a web browser: | ||
| * http://<host-machine>:8810 | ||
| * http://<host-machine>:8810/web/customer.p | ||
|
|
||
| 4. Access Prometheus via a web browser: | ||
| * http://<host-machine>:9090 | ||
| * Use DB_ in the Expression field to query for metrics for the OpenEdge database | ||
| * Use PASOE_ in the Expression field to query for metrics for PAS for OpenEdge | ||
|
|
||
| 5. Access Grafana via a web browser: | ||
| * http://<host-machine>:3000 | ||
| * Use admin/admin to login | ||
|
|
||
| **Notes:** | ||
| * The setup script, adds `/etc/rc.local` to start the configuration on startup of the machine. You can also use `./scripts/start.sh` and `./scripts/stop.sh` to manually start and stop the configuration. | ||
|
|
||
| ### Creating a Dashboard in Grafana | ||
|
|
||
| **Steps** | ||
|
|
||
| 1. Click on DATA SOURCES then on Prometheus to create a data source with the following values: | ||
| * Prometheus server URL: http://prometheus:9090 | ||
| * Prometheus is a reference to a service in the `docker-compose.yaml` file and can be used as a hostname. | ||
| * Scroll down and click on Save & Test | ||
|
|
||
| 2. Click on Home then on DASHBOARDS to create a dashboard. | ||
| 3. Click on Add visualization to add a visualization for an OpenEdge database metric. | ||
| * Select Prometheus as the data source. | ||
| * Click on "Select metric" and click on Open. | ||
|
|
||
|  | ||
|
|
||
| * Search for "DB_" and select the BufferHits metric from the list. | ||
|
|
||
|  | ||
|
|
||
| * Click on Run queries. | ||
| * Click on Time series to change the visualization to use a Gauge. | ||
| * Change the Panel Title to "Buffer Hits" | ||
| * Scroll down and change the Unit in "Standard options" to Misc / Pecent (0-100). | ||
| * Scroll down and click on the Red dot for the Threshold to use Green. | ||
| * Click on the Green dot for Base to use Orange. | ||
| * Click on Apply. | ||
|
|
||
|  | ||
|
|
||
| 4. Click on Add / Visualization (on toolbar) to add a visualization for a PASOE metric. | ||
| * Click on "Select metric" and click on Open. | ||
| * Search for "getrequests" and select the getRequests metric from the list. | ||
| * Click on Run queries. | ||
| * Change the Panel Title to "Get Requests". | ||
| * Click on Apply. | ||
| 5. Click on the "Save dashboard" button (floppy disk icon on toolbar) to save the dashboard. | ||
|
|
||
|  | ||
|
|
||
| Notes: | ||
| * Examples of Grafana dashboards can be found at https://grafana.com/grafana/dashboards/ | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| /** ------------------------------------------------------------------------ | ||
| File : customer.p | ||
| Purpose : Return records from the Customer table as JSON. | ||
| Syntax : | ||
| Description : | ||
| @author egarcia | ||
| Modified : Thu Jun 27 15:32:00 EST 2024 | ||
| Notes : | ||
| * Deploy to openedge/ directory in PAS instance | ||
| using OpenEdge.Web.CompatibilityHandler | ||
| * Access using http://<host-machine>:8810/web/customer.p | ||
| ---------------------------------------------------------------------- */ | ||
| USING Progress.Json.ObjectModel.JsonObject. | ||
edselg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| CREATE WIDGET-POOL. | ||
|
|
||
| {src/web2/wrap-cgi.i} | ||
|
|
||
| DEFINE TEMP-TABLE ttCustomer LIKE Customer. | ||
| DEFINE DATASET dsCustomer FOR ttCustomer. | ||
|
|
||
| RUN process-web-request. | ||
|
|
||
| PROCEDURE outputHeader : | ||
| output-content-type ("application/json":U). | ||
| END PROCEDURE. | ||
|
|
||
| PROCEDURE process-web-request : | ||
| DEFINE VARIABLE oJsonObject AS Progress.Json.ObjectModel.JsonObject NO-UNDO. | ||
| DEFINE VARIABLE lChar AS LONGCHAR NO-UNDO. | ||
| RUN outputHeader. | ||
|
|
||
| EMPTY TEMP-TABLE ttCustomer. | ||
| FOR EACH Customer WHERE Customer.CustNum < 3110 NO-LOCK: | ||
| CREATE ttCustomer. | ||
| BUFFER-COPY Customer TO ttCustomer. | ||
| END. | ||
|
|
||
| oJsonObject = NEW JsonObject(). | ||
| oJsonObject:READ(DATASET dsCustomer:HANDLE). | ||
|
|
||
| lChar = oJsonObject:GetJsonText(). | ||
| {&OUT-LONG} lChar. | ||
|
|
||
| END PROCEDURE. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| [AppServer.SessMgr] | ||
| agentStartupParam=-T "${catalina.base}/temp" -db sports2020 -S 20000 | ||
|
|
||
| [AppServer.Agent] | ||
| numInitialSessions=2 | ||
|
|
||
| [oepas1.ROOT.WEB] | ||
| adapterEnabled=1 | ||
| defaultHandler=OpenEdge.Web.CompatibilityHandler |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| exporter: | ||
| name: "otlp" | ||
| endpoint: "http://192.168.56.215:4317" | ||
| protocol: "grpc" | ||
| connectionretry: 20 | ||
| timeout: 10 | ||
|
|
||
|
|
||
| oedbInstances: | ||
| - dbname: sports2020 | ||
| host: localhost | ||
| port: 20000 | ||
| user: dba | ||
| password: dba | ||
| metricsregex: | ||
| otherdbconnparams: | ||
| dbconnectionretry: 5 | ||
| dbschedule: 30 | ||
| dbduration: seconds |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| exporter: | ||
| name: "otlp" | ||
| endpoint: "http://192.168.56.215:4317" | ||
| protocol: "grpc" | ||
| connectionretry: 20 | ||
| timeout: 10 | ||
|
|
||
| pasInstances: | ||
| - pasdir: "/psc/wrk/oepas1" | ||
| passchedule: 30 | ||
| pasconnectionretry: 5 | ||
| pasduration: seconds | ||
| metricsregex: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/sh -e | ||
edselg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # | ||
|
|
||
| export DLC=/psc/dlc | ||
edselg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| export PATH=$DLC/bin:$PATH | ||
|
|
||
| cd /psc/wrk | ||
| proserve sports2020 -S 20000 -n 30 | ||
| /psc/wrk/oepas1/bin/tcman.sh start | ||
|
|
||
| exit 0 | ||
Uh oh!
There was an error while loading. Please reload this page.