Skip to content

Commit c096092

Browse files
AuditHawkssatishc1
AuditHawks
authored and
satishc1
committed
Remote dashboard creation and audit api (#1576)
* Add api-audit * First commmit to support dashboard creation via API. * Made collector item property strings accessible to collectors. Added required and optional fields to collector. * Added code to save required and optional fields for each collector. * Added code to save unique and all fields for each collector. * Add DashboardRemoteService to test configs. * Changed Collector options Map types. * Optimize Imports. * Modified dashboard delete logic: disable collector item if needed. Added tests. * PMD fixes. * Bug fixes. * Changed CMDB lookup type parameter * New endpoint to update existing dashboards. * Bug fixes. * Added code to insert all fields and unique fields to collector via "webhook"s such as Github, Jenkins * Add test for remote dashboard creation. * Add test for remote dashboard creation. * Add test for remote dashboard creation. * Add test for remote dashboard creation. * Add test for remote dashboard creation. * Major refactor of Github collector - mainly around collecting pull requests and issues. * PMD fixes. * Add "remote created" flag to dashboard. * Add audit functionality for build job config history.
1 parent 53a4273 commit c096092

File tree

91 files changed

+51828
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

91 files changed

+51828
-0
lines changed

‎.gitignore

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Mac artifacts
2+
.DS_Store
3+
4+
# Eclipse artifacts
5+
.project
6+
.settings
7+
.classpath
8+
9+
# Logs
10+
logs
11+
log
12+
*.log
13+
14+
# Runtime data
15+
pids
16+
*.pid
17+
*.seed
18+
*.iml
19+
/target
20+
/.idea
21+
22+
# Distributable artifacts
23+
target
24+
25+
# Specific project files
26+
jir-client.properties
27+
/bin/
28+
application.properties

‎AuditRules.md

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Peer Review Audit API:
2+
3+
### Rules:
4+
##### 1. Source code is version controlled (Github)
5+
Logic:
6+
##### 2. No direct commit to release branch
7+
Logic:
8+
##### 3. All changes in the release branch are via Pull Requests
9+
Logic:
10+
##### 4. All Pull Requests are reviewed by someone other than the authors
11+
Logic:
12+
##### 5. All Pull Requests are merged by someone other than the authors
13+
Logic:
14+
15+
# Quality Audit API:
16+
17+
### Rules:
18+
##### 1. Static code analysis run on the version of artifact to be released meets threshold.
19+
Logic:
20+
##### 2. Static code analysis run profile change is reviewed by someone other than the person who made changes
21+
Logic:
22+

‎README.md

+126
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
[![Docker Stars](https://img.shields.io/docker/stars/capitalone/hygieia-api.svg)](https://hub.docker.com/r/capitalone/hygieia-api/)
2+
[![Docker Stars](https://img.shields.io/docker/pulls/capitalone/hygieia-api.svg)](https://hub.docker.com/r/capitalone/hygieia-api/)
3+
4+
# Hygieia℠ Audit API
5+
6+
7+
8+
This project uses Spring Boot to package the api as an executable JAR with dependencies.
9+
10+
11+
## Building
12+
13+
Run `mvn install` to package the collector into an executable JAR file.
14+
15+
16+
## API Properties file
17+
18+
The API layer needs a property file in following format:
19+
20+
```properties
21+
# dashboard.properties
22+
dbname=dashboard
23+
dbusername=[MogoDb Database Username, defaults to empty]
24+
dbpassword=[MongoDb Database Password, defaults to empty]
25+
dbhost=[Host on which MongoDb is running, defaults to localhost]
26+
dbport=[Port on which MongoDb is listening, defaults to 27017]
27+
dbreplicaset=[false if you are not using MongoDB replicaset]
28+
dbhostport=[host1:port1,host2:port2,host3:port3]
29+
server.contextPath=[Web Context path if any]
30+
server.port=[Web server port - default is 8080]
31+
logRequest=false
32+
logSplunkRequest=false
33+
34+
```
35+
36+
All the above values are optional. Even without the property file you must be able to run the api (assuming you have mongodb installed with no authorization).
37+
38+
**Note:** When `dbusername` is not present or the value is empty then it skips the mongodb authorization part.
39+
40+
41+
## Run the API
42+
43+
After you have build your project, from the target folder run the below command,
44+
45+
```bash
46+
java -jar api-audit.jar --spring.config.location=api-audit.properties
47+
```
48+
49+
By default the server starts at port `8080` and uses the context path `/api`.
50+
These values are configurable by using the below 2 properties in `dashboard.properties`.
51+
The jasypt.encryptor.password system property is used to decrypt the database password. For more information, refer to encrypted properties.
52+
53+
```properties
54+
server.contextPath=/api-audit
55+
server.port=8090
56+
```
57+
58+
For more information about the server configuration, see the Spring Boot [documentation](http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-external-config-application-property-files).
59+
60+
## Docker image
61+
62+
63+
### Create
64+
65+
```bash
66+
# from top-level project
67+
mvn clean package -pl api docker:build
68+
```
69+
70+
### Run
71+
72+
First start Mongodb
73+
74+
For example:
75+
```
76+
docker run -d -p 27017:27017 --name mongodb -v ./mongo:/data/db mongo:latest mongod --smallfiles
77+
```
78+
79+
80+
Create User:
81+
```
82+
use db
83+
db.createUser({user: "db", pwd: "dbpass", roles: [{role: "readWrite", db: "dashboard"}]})
84+
```
85+
or from CLI:
86+
```bash
87+
mongo 192.168.64.2/admin --eval 'db.getSiblingDB("db").createUser({user: "db", pwd: "dbpass", roles: [{role: "readWrite", db: "dashboard"}]})'
88+
```
89+
90+
More details: <https://hub.docker.com/r/library/mongo/>
91+
92+
93+
Then running the API from docker is easy:
94+
95+
```
96+
docker run -t -p 8080:8080 --link mongodb:mongo -v ./logs:/hygieia/logs -i hygieia-api:latest
97+
```
98+
99+
### Environment variables
100+
101+
Environment variables for dashboard properties can be specified like:
102+
103+
```
104+
docker run -t -p 8080:8080 -v ./logs:/hygieia/logs -e "SPRING_DATA_MONGODB_HOST=127.0.0.1" -i hygieia-api:latest
105+
```
106+
107+
For more properties see the [Dockerfile](Dockerfile)
108+
109+
### List containers
110+
111+
View port by running
112+
```bash
113+
docker ps
114+
```
115+
116+
### API Access
117+
118+
Take the port mapping and the IP for your docker-machine <env> ip and verify by ```http://<docker-machine env ip>:<docker port for hygieia_api>/api/dashboard```
119+
120+
121+
## Create new API
122+
1. Create a new rest controller or add to an existing controller
123+
2. Create a new service interface and new service implementation.
124+
3. Add new request and response classes
125+
126+
Note: For common data models used in the audit api's, look into core module's model package.

‎docker/Dockerfile

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
2+
FROM docker.io/java:openjdk-8-jdk
3+
4+
MAINTAINER [email protected]
5+
6+
7+
ENV SPRING_DATA_MONGODB_DATABASE=dashboard
8+
ENV SPRING_DATA_MONGODB_HOST=10.0.1.1
9+
ENV SPRING_DATA_MONGODB_PORT=9999
10+
ENV SPRING_DATA_MONGODB_USERNAME=db
11+
ENV SPRING_DATA_MONGODB_PASSWORD=ENC(aSPTk36yA/ZklUg75CrZ8w==)
12+
13+
14+
RUN \
15+
mkdir /hygieia
16+
17+
COPY hygieia /hygieia
18+
COPY properties-builder.sh /hygieia/
19+
20+
21+
WORKDIR /hygieia
22+
23+
VOLUME ["/hygieia/logs"]
24+
25+
26+
EXPOSE 8080
27+
CMD ./properties-builder.sh &&\
28+
java -Djava.security.egd=file:/dev/./urandom -jar api.jar --spring.config.location=/hygieia/dashboard.properties

‎docker/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

‎docker/properties-builder.sh

+70
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
#!/bin/bash
2+
3+
# if we are linked, use that info
4+
if [ "$MONGO_PORT" != "" ]; then
5+
# Sample: MONGO_PORT=tcp://172.17.0.20:27017
6+
export SPRING_DATA_MONGODB_HOST=`echo $MONGO_PORT|sed 's;.*://\([^:]*\):\(.*\);\1;'`
7+
export SPRING_DATA_MONGODB_PORT=`echo $MONGO_PORT|sed 's;.*://\([^:]*\):\(.*\);\2;'`
8+
fi
9+
10+
echo "SPRING_DATA_MONGODB_HOST: $SPRING_DATA_MONGODB_HOST"
11+
echo "SPRING_DATA_MONGODB_PORT: $SPRING_DATA_MONGODB_PORT"
12+
13+
14+
cat > dashboard.properties <<EOF
15+
#Database Name - default is test
16+
dbname=${SPRING_DATA_MONGODB_DATABASE:-dashboard}
17+
18+
#Database HostName - default is localhost
19+
dbhost=${SPRING_DATA_MONGODB_HOST:-10.0.1.1}
20+
21+
#Database Port - default is 27017
22+
dbport=${SPRING_DATA_MONGODB_PORT:-9999}
23+
24+
#Database Username - default is blank
25+
dbusername=${SPRING_DATA_MONGODB_USERNAME:-db}
26+
27+
#Database Password - default is blank
28+
dbpassword=${SPRING_DATA_MONGODB_PASSWORD:-dbpass}
29+
30+
logRequest=${LOG_REQUEST:-false}
31+
logSplunkRequest=${LOG_SPLUNK_REQUEST:-false}
32+
33+
corsEnabled=${CORS_ENABLED:-false}
34+
35+
corsWhitelist=${CORS_WHITELIST:-http://domain1.com:port,http://domain2.com:port}
36+
37+
feature.dynamicPipeline=${FEATURE_DYNAMIC_PIPELINE:-disabled}
38+
39+
#Authentication Settings
40+
# JWT expiration time in milliseconds
41+
auth.expirationTime=${AUTH_EXPIRATION_TIME:-}
42+
# Secret Key used to validate the JWT tokens
43+
auth.secret=${AUTH_SECRET:-}
44+
auth.authenticationProviders=${AUTH_AUTHENTICATION_PROVIDERS:-}
45+
46+
# LDAP Server Url, including port of your LDAP server
47+
auth.ldapServerUrl=${AUTH_LDAP_SERVER_URL:-}
48+
49+
# If using standard ldap
50+
# LDAP User Dn Pattern, where the username is replaced with '{0}'
51+
auth.ldapUserDnPattern=${AUTH_LDAP_USER_DN_PATTERN:-}
52+
53+
# If using ActiveDirectory
54+
# This will be the domain part of your userPrincipalName
55+
auth.adDomain=${AUTH_AD_DOMAIN:-}
56+
# This will be your root dn
57+
auth.adRootDn=${AUTH_AD_ROOT_DN:-}
58+
# This is your active directory url
59+
auth.adUrl=${AUTH_AD_URL:-}
60+
61+
#Monitor Widget proxy credentials
62+
monitor.proxy.username=${MONITOR_PROXY_USERNAME:-}
63+
monitor.proxy.password=${MONITOR_PROXY_PASSWORD:-}
64+
65+
#Monitor Widget proxy information
66+
monitor.proxy.type=${MONITOR_PROXY_TYPE:-http}
67+
monitor.proxy.host=${MONITOR_PROXY_HOST:-}
68+
monitor.proxy.port=${MONITOR_PROXY_PORT:-80}
69+
70+
EOF

0 commit comments

Comments
 (0)