Testing and logging related policies

Technical-Steering-Committee/guidelines/README.md

@@ -0,0 +1,3 @@
+# Guidelines
+
+The files in this repository represents recommendation and guidelines not a strict policies that the squads has to follow. 

Technical-Steering-Committee/guidelines/branching-strategy.md

@@ -1,6 +1,6 @@
 # Branches Guideline
 
-This documentation introduces guideline for using branches in Zowe project, including all component projects. We try to keep consistent and apply the same branching model to all source code repositories. Subproject may use a different branch model to better suit its needs, but it should be properly documented. For example, Zowe CLI used a different [Versioning](https://github.com/zowe/zowe-cli/blob/master/docs/MaintainerVersioning.md) to better work with NPM package tags.
+This documentation introduces guideline for using branches in Zowe project, including all sub-projects. We try to keep consistent and apply the same branching model to all source code repositories. Subproject may use a different branch model to better suit its needs, but it should be properly documented. For example, Zowe CLI used a different [Versioning](https://github.com/zowe/zowe-cli/blob/master/docs/MaintainerVersioning.md) to better work with NPM package tags.
 
 Zowe project follows [Semantic Versioning](https://semver.org). So all versions mentioned in this documentation follows `[major].[minor].[patch]` version pattern.
 
@@ -12,7 +12,7 @@ Zowe project follows [Semantic Versioning](https://semver.org). So all versions
 
   Branches listed below are mandatory and should exist all the time. These branches should be **protected** from updating. So any changes go into these branches should be through **Pull Request** and reviewed by other committers.
 
-  * **master** - this is the main release branch. It contains the most recent stable formal release. The release tag is created on this branch. If the component has dependencies, only static version of dependent artifacts should be listed. For example, a dependency on another sub-project version `~5.2.0` (which means latest patch release of `5.2.*`) is not acceptable. This _master_ branch should not be merged back into _staging_ branch. In certain circumstances, we will _cherry-pick_ hot fixes back into _staging_ branch.
+  * **main** - this is the main release branch. It contains the most recent stable formal release. The release tag is created on this branch. If the component has dependencies, only static version of dependent artifacts should be listed. For example, a dependency on another sub-project version `~5.2.0` (which means latest patch release of `5.2.*`) is not acceptable. This _master_ branch should not be merged back into _staging_ branch. In certain circumstances, we will _cherry-pick_ hot fixes back into _staging_ branch.
   * **staging** - this is the main development thread. It contains the most recent code for **next** (the coming) release. If the component has dependencies, dynamic patch-level version of dependent artifacts is recommended and acceptable. For example, `~5.2.0` is acceptable because it means `5.2.*`. But `^5.2.0` is not acceptable because it means `5.*`. This branch will be merged into _master_ branch during release process.
 
 - Optional Ephemeral Branches
@@ -38,7 +38,7 @@ With an example of main thread are ready for v2.x release, then v1.x became LTS
 
 - Long-Lived Branches
 
-  * **v1.x/master** - branched out from _master_ branch.
+  * **v1.x/main** - branched out from _main_ branch.
   * **v1.x/staging** - branched out from _staging_ branch.
 
 - Optional Ephemeral Branches

Technical-Steering-Committee/logging.md

@@ -0,0 +1,56 @@
+# Message Management
+
+## How do we log
+
+### Message ID Components (message prefixes)
+
+ZWE - Zowe, This should be used as default for all Zowe messages. 
+
+### Zowe Projects
+
+There are server and client side components that are logging separately as such we may have duplication of logging for Server side and Client side. 
+
+#### Server Side
+
+A - API Mediation Layer
+D - Zowe App Framework
+S - Zowe System Services
+
+#### Client Side 
+
+C - Zowe CLI
+E - Zowe VSCode Explorer
+H - Zowe Chat
+I - Zowe Intellij Explorer
+J - Client Java SDK
+K - Client Kotlin SDK
+N - Client Node.js SDK
+P - Client Python SDK
+
+### Message Formats
+
+ZWEcnnnnt, ZWEcsnnnt or ZWEcsxnnt {Actual message} where:
+
+**ZWE** - static for Zowe messages.
+
+**c** - one of the registered Components ... (see above).  Components need the TSC to approve the use of the project letter to ensure consistency across all of the projects.
+
+**s** - potentially a sub component within the component space. An example from API Mediation Layer would be G for Gateway. 
+
+**x** - potentially a space for extensions. For the extensions the expectation is that there will be X as sub component. An example would be ZWEDXA11E
+
+**nnn** - is a linear sequence starting from 001 to 999.
+
+**nnnn** - is a linear sequence starting from 0001 to 9999.
+
+**t** - is a Type.  One of (I, E, A, W)  General conventions by IBM are **I**nform, **E**ventual Action (often construed as error), **A**ction, **W**arning or **S**evere error
+
+Messages should be sent to the doc-team and follow the general IBM format as described in Messages and Codes. All the messages needs to have associated explanatory message in the Zowe docs. 
+
+An example of a message is: *ZWEAG123E An error with z/OSMF authentication* 
+
+## What do we log
+
+In the default configuration we should be logging only the errors and warnings. Errors should be problems that prevents usage of either whole Zowe or some part of Zowe. Warnings should represent potential problems that may have an impact on functionality but won't prevent usage. 
+
+Apart from that all the components should log their startup and turndown when it happens as an Information. 

Technical-Steering-Committee/support.md

@@ -0,0 +1,14 @@
+# Zowe Community statement on Support
+
+Zowe is an Open Source project that's being driven by the community. As such any effort provided by the community in on best-effort basis.
+If you need a support with SLA's please consult our list of [Conformant Support Providers](https://openmainframeproject.org/our-projects/zowe-conformant-support-provider-program/)
+
+## Validating pre-requisities
+
+The Zowe community has access only to limited set of hardware. This limits our capacity to run automated test suites. We keep the list of the pre-requisites that
+Zowe is tested on in [Zowe docs](https://github.com/zowe/docs-site/blob/tsc/support/matrix/docs/user-guide/systemrequirements-zos.md)
+
+## Out of support prerequisites
+
+For the pre-requisites that end their support Zowe doesn't guarantee any support for running. E.g. Zowe doesn't have to support z/OS 2.2 as it's already out of support. 
+As such changes that break support for such version aren't considered as breaking change. 

Technical-Steering-Committee/testing.md

@@ -0,0 +1,94 @@
+TODO: Is there any way to showcase how we follow the guidelines?
+
+The first stage of the testing and in general of quality assurance lies with the developers developing the code. They are responsible for the quality of the code. There are tools helping them ensure high quality of the code and preventing unnecessary issues with the code. What are the tools that we expect the developers to work with and integrate to their pipelines. There are two wider categories of the tools:
+
+1) Static analysis - Going through the created artifacts such as the code and looking for potential risks
+2) Dynamic analysis - Testing against the running applications what is the behavior for certain known and common vulnerabilities. The typical tests written by developers and executed on every run belong to this category. 
+
+#### Static analysis
+
+The tools in this category goes through the code and/or artifacts as they are written and look for known problems and risks. Some of them are easy to integrate in a local workflow, some of them are better integrated via the workflow in Github Actions. These tools needs to be run at least on a per release basis and the output neds to be either cleansed or approved by either the squad or by security workgroup in case of security issues. 
+
+- Compilers
+  - In case the language uses compilers, the first quality control is to fix all the warnings and above unless approved by the squad on the squad meeting. Every squad should keep list of the warnings approved by the squad. 
+- Quality scanners (linters)
+  - Available for every language used in our code. 
+    - Java - [checkstyle](https://checkstyle.org/)
+    - Javascript - [ESLint](https://eslint.org/) or JSLint (https://www.jslint.com/)
+    - Typescript - [typescript-eslint](https://typescript-eslint.io/)
+    - C - lint, gcc
+    - Kotlin - [KTLint](https://github.com/pinterest/ktlint)
+    - Python - [PyLint](https://pylint.org/)
+- Security Code Scanners (Static Application Security Testing (SAST) Tools)
+  - [SonarCloud](https://sonarcloud.io/organizations/zowe/projects) - is also a quality tool. 
+    TODO: What is the threshold over which we shouldn't release the projects?
+  - CodeQL - Tool integrated to the GitHub, the details aren't shown anywhere
+    TODO: Security needs to be fixed or agreed upon by Security Workgroup 
+    TODO: Where and how the security workgroup keep this information.
+- Secret Scanning Tools
+  - TODO: GitHub does some work in the area
+- Software Component Analysis (SCA)/Dependency Analysis tools 
+  - BlackDuck 
+    - The information are shared by one of the vendor running the tool. Overall this is imperfect solution. 
+  - LFX Security - Work in progress as the current information keeps information from previous branches and as such doesn't represent the current situation. 
+
+TODO: Where and how do I find the results?
+
+#### Dynamic analysis
+
+- Traditional testing for security (Unit testing, Integration testing)
+  - Unit testing
+  - Functional testing 
+  - System Testing - Integration and testing of the interaction between components. 
+  - Statement and branch coverage - Provided to us by Sonarcloud
+- Fuzzers
+  - Explore https://github.com/google/oss-fuzz 
+- Web Application Scanner - AppScan
+
+#####
+
+How do we improve our capability to catch the security related issues? 
+Is there any decent way to introduce the other companies into this?
+
+It would be great to be able to think more like and adversary. 
+
+### Penetration Testing
+
+Probably another class of tests that focuses on the security
+
+The current situation is that the companies are pen testing Zowe internally and share the results. This basically means that we are triplicating the effort. I believe we should be able to do this out in the open. 
+
+To make sure that open means that only trusted actors will be involved and notified of the risks, I propose we keep the information within the security-reports repository and within the security workgroup. This should give us enough transparency without risking security for users. 
+
+#### Internal testing
+
+##### Red Team Pen Testing 
+
+This is a more complex issue as there are more ways to approach the topic. In general the name comes from either an internal or external team that acts as the adversary and tries to get access to the systems, steal the data, make the services inaccessible and so on. 
+
+There is also a second team known as Blue team which tries to prevent the success of the red team. 
+
+The GitLab explains how they use the red team in their security operations here: https://about.gitlab.com/handbook/engineering/security/security-operations/red-team/. Among others advocating the approach is the UK government (https://www.gov.uk/government/publications/a-guide-to-red-teaming and https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1027158/20210625-Red_Teaming_Handbook.pdf). 
+
+Keeping the whole red team would be quite expensive and complex in an open source environment and therefore what I recommend here is an one day exercise as a part of innovation week where Zowe splits to two teams. One trying to gain access to the system via Zowe and second trying to prevent this as well proving that such effort will be shown in the systems. 
+
+If we find this exercise valuable, then we should figure out a way to do it regularly e.g. once a PI or at least once a year. 
+
+#### External guarantees (External Audit)
+
+This section covers the actors without direct stake in the Zowe and that will help the community and the users trust the delivered solution. 
+
+##### Bug Bounties
+
+The Bug Bounty is a concept where the external parties to the project gets money from uncovering vulnerabilities within the software and reporting them via the standard security process to the community. 
+
+The aim is to persuade more white hat hackers to explore the code as there is tangible benefit for them to find and share the bugs. 
+
+It’s used for example by Google. Here: https://bughunters.google.com/ you can learn more about the community that was built by Google for bug hunting. 
+
+##### External Pen Testing Companies
+
+At the moment the major vendors involved within Zowe such as IBM, Rocket and Broadcom each conduct the penetration tests internally. The results are shared with the Security workgroup. 
+
+The possibility here would be to pool the resources used to do the testing by the companies to either hire better companies or to change the scope and look for more potential issues.  
+