-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fbshipit-source-id: 7c98b4413a035219f8a2949626de33424f6182e5
- Loading branch information
0 parents
commit 719915b
Showing
1,225 changed files
with
659,196 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
CircleCi integration is controlled by the `./circleci/config.yml` file. Our | ||
config currently contains two workflows. One is triggered on every pull request update. | ||
The other workflow runs nightly to verify our compatibility with prestodb internal protocol. | ||
|
||
The PR workflow is named `dist-compile` and has 4 jobs, 2 to build and run unit tests on linux and macos | ||
and 2 to check code formatting and license headers: | ||
* linux-build | ||
* macos-build | ||
* format-check | ||
* header-check | ||
|
||
## Running locally | ||
|
||
The linux container based jobs can be run locally using the `circleci` cli: | ||
|
||
``` | ||
circleci local execute --job JOB_NAME | ||
``` | ||
|
||
For example to run unit tests use: | ||
|
||
``` | ||
circleci local execute --job linux-build | ||
``` | ||
|
||
A Nightly build with prestodb/master sync checks that the presto_protocol library | ||
remains in sync with Presto Java. | ||
|
||
Run the nightly sync job locally: | ||
``` | ||
circleci local execute --job presto-sync | ||
``` | ||
|
||
## Install CircleCi cli | ||
``` | ||
curl -fLSs https://circle.ci/cli | bash | ||
``` | ||
|
||
To use containers Docker must be installed. Here are instructions to [Install | ||
Docker on macos](https://docs.docker.com/docker-for-mac/install/). Docker deamon | ||
must be running before issuing the `circleci` commands. | ||
|
||
### Macos testing | ||
|
||
Macos testing is done by using the CircleCi macos executor and installing | ||
dependencies each time the job is run. This executor cannot be run locally. | ||
The script `scripts/setup-macos.sh` contains commands that are run as part of | ||
this job to install these dependencies. | ||
|
||
### Linux testing | ||
|
||
Linux testing uses a Docker container. The container build depends on the Velox CircleCi container. Check | ||
f4d/.circleci/config.yml to see that the base container in circleci-container.dockfile is using the latest. | ||
The container build uses Docker and should be run on your macos or linux laptop with Docker installed and | ||
running. | ||
|
||
#### Build the base container: | ||
|
||
* In an up-to-date clone of f4d (maybe you have one?) | ||
|
||
``` | ||
git clone [email protected]:facebookexternal/f4d.git | ||
cd f4d | ||
make base-container | ||
``` | ||
* Wait - This step takes rather a long time. It is building clang-format v8 to be compatible with fbcode | ||
* When the base container is finished the new container name will be printed on the console. | ||
* Push the container to DockerHub | ||
``` | ||
docker push prestocpp/base-container:$USER-YYYYMMDD | ||
``` | ||
* After the push, update `scripts/velox-container.dockfile` with the newly build base container name | ||
|
||
#### Build the dependencies container | ||
|
||
* If you have a new base-container update scripts/velox-container.dockfile to refer to it | ||
* Build the velox container | ||
``` | ||
make velox-container.dockfile | ||
``` | ||
* Wait - This takes a few minutes, but not nearly as long as the base container. | ||
* When the velox container is finished the new container name will be printed on the console. | ||
* Push the container to DockerHub | ||
``` | ||
docker push prestocpp/velox-container:$USER-YYYYMMDD | ||
``` | ||
* Update `.circleci/config.yml` with the newly built circleci container name. | ||
There are two places in the config.yml file that refer to the container, update | ||
both. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
version: 2.1 | ||
workflows: | ||
version: 2 | ||
dist-compile: | ||
jobs: | ||
- linux-build | ||
- macos-build | ||
- format-check | ||
- header-check | ||
|
||
executors: | ||
build: | ||
docker: | ||
- image : prestocpp/velox-circleci:mikesh-20210610 | ||
resource_class: xlarge | ||
environment: | ||
CC: /opt/rh/gcc-toolset-9/root/bin/gcc | ||
CXX: /opt/rh/gcc-toolset-9/root/bin/g++ | ||
check: | ||
docker: | ||
- image : prestocpp/velox-check:mikesh-20210609 | ||
|
||
jobs: | ||
macos-build: | ||
macos: | ||
xcode: "11.7.0" | ||
steps: | ||
- checkout | ||
- restore_cache: | ||
name: "Restore Dependency Cache" | ||
key: velox-circleci-macos-deps-{{ checksum ".circleci/config.yml" }}-{{ checksum "scripts/setup-macos.sh" }} | ||
- run: | ||
name: "Install dependencies" | ||
command: | | ||
set -xu | ||
if [[ ! -e ~/deps ]]; then | ||
mkdir ~/deps ~/deps-src | ||
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C ~/deps | ||
PATH=~/deps/bin:${PATH} DEPENDENCY_DIR=~/deps-src INSTALL_PREFIX=~/deps PROMPT_ALWAYS_RESPOND=n ./scripts/setup-macos.sh | ||
rm -rf ~/deps/.git ~/deps/Library/Taps/ # Reduce cache size by 70%. | ||
fi | ||
- save_cache: | ||
name: "Save Dependency Cache" | ||
key: velox-circleci-macos-deps-{{ checksum ".circleci/config.yml" }}-{{ checksum "scripts/setup-macos.sh" }} | ||
paths: | ||
- ~/deps | ||
- run: | ||
name: "Calculate merge-base for CCache" | ||
command: git merge-base origin/master HEAD > merge-base | ||
- restore_cache: | ||
name: "Restore CCache cache" | ||
keys: | ||
- velox-ccache-{{ arch }}-{{ checksum "merge-base" }} | ||
- run: | ||
name: "Build on MacOS" | ||
command: | | ||
export PATH=~/deps/bin:${PATH} | ||
mkdir -p .ccache | ||
export CCACHE_DIR=$(pwd)/.ccache | ||
ccache -sz -M 5Gi | ||
cmake -B _build/debug -GNinja -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=~/deps -DCMAKE_CXX_COMPILER_LAUNCHER=ccache | ||
ninja -C _build/debug | ||
ccache -s | ||
no_output_timeout: 1h | ||
- save_cache: | ||
name: "Save CCache cache" | ||
key: velox-ccache-{{ arch }}-{{ checksum "merge-base" }} | ||
paths: | ||
- .ccache/ | ||
|
||
linux-build: | ||
executor: build | ||
steps: | ||
- checkout | ||
- run: | ||
name: "Calculate merge-base for CCache" | ||
command: git merge-base origin/master HEAD > merge-base | ||
- restore_cache: | ||
name: "Restore CCache cache" | ||
keys: | ||
- velox-ccache-{{ arch }}-{{ checksum "merge-base" }} | ||
- run: | ||
name: Build | ||
command: | | ||
mkdir -p .ccache | ||
export CCACHE_DIR=$(realpath .ccache) | ||
ccache -sz -M 5Gi | ||
source /opt/rh/gcc-toolset-9/enable | ||
make debug NUM_THREADS=8 MAX_HIGH_MEM_JOBS=3 MAX_LINK_JOBS=4 | ||
ccache -s | ||
no_output_timeout: 1h | ||
- store_artifacts: | ||
path: '_build/debug/.ninja_log' | ||
- run: | ||
name: Run Unit Tests | ||
command: | | ||
make unittest NUM_THREADS=8 | ||
no_output_timeout: 1h | ||
- save_cache: | ||
name: "Save CCache cache" | ||
key: velox-ccache-{{ arch }}-{{ checksum "merge-base" }} | ||
paths: | ||
- .ccache/ | ||
|
||
format-check: | ||
executor: check | ||
steps: | ||
- checkout | ||
- run: | ||
name: Check formatting | ||
command: 'make format-check' | ||
|
||
header-check: | ||
executor: check | ||
steps: | ||
- checkout | ||
- run: | ||
name: Check license headers | ||
command: 'make header-check' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
--- | ||
AccessModifierOffset: -1 | ||
AlignAfterOpenBracket: AlwaysBreak | ||
AlignConsecutiveAssignments: false | ||
AlignConsecutiveDeclarations: false | ||
AlignEscapedNewlinesLeft: true | ||
AlignOperands: false | ||
AlignTrailingComments: false | ||
AllowAllParametersOfDeclarationOnNextLine: false | ||
AllowShortBlocksOnASingleLine: false | ||
AllowShortCaseLabelsOnASingleLine: false | ||
AllowShortFunctionsOnASingleLine: Empty | ||
AllowShortIfStatementsOnASingleLine: false | ||
AllowShortLoopsOnASingleLine: false | ||
AlwaysBreakAfterReturnType: None | ||
AlwaysBreakBeforeMultilineStrings: true | ||
AlwaysBreakTemplateDeclarations: true | ||
BinPackArguments: false | ||
BinPackParameters: false | ||
BraceWrapping: | ||
AfterClass: false | ||
AfterControlStatement: false | ||
AfterEnum: false | ||
AfterFunction: false | ||
AfterNamespace: false | ||
AfterObjCDeclaration: false | ||
AfterStruct: false | ||
AfterUnion: false | ||
BeforeCatch: false | ||
BeforeElse: false | ||
IndentBraces: false | ||
BreakBeforeBinaryOperators: None | ||
BreakBeforeBraces: Attach | ||
BreakBeforeTernaryOperators: true | ||
BreakConstructorInitializersBeforeComma: false | ||
BreakAfterJavaFieldAnnotations: false | ||
BreakStringLiterals: false | ||
ColumnLimit: 80 | ||
CommentPragmas: '^ IWYU pragma:' | ||
ConstructorInitializerAllOnOneLineOrOnePerLine: true | ||
ConstructorInitializerIndentWidth: 4 | ||
ContinuationIndentWidth: 4 | ||
Cpp11BracedListStyle: true | ||
DerivePointerAlignment: false | ||
DisableFormat: false | ||
ForEachMacros: [ FOR_EACH, FOR_EACH_R, FOR_EACH_RANGE, ] | ||
IncludeCategories: | ||
- Regex: '^<.*\.h(pp)?>' | ||
Priority: 1 | ||
- Regex: '^<.*' | ||
Priority: 2 | ||
- Regex: '.*' | ||
Priority: 3 | ||
IndentCaseLabels: true | ||
IndentWidth: 2 | ||
IndentWrappedFunctionNames: false | ||
KeepEmptyLinesAtTheStartOfBlocks: false | ||
MacroBlockBegin: '' | ||
MacroBlockEnd: '' | ||
MaxEmptyLinesToKeep: 1 | ||
NamespaceIndentation: None | ||
ObjCBlockIndentWidth: 2 | ||
ObjCSpaceAfterProperty: false | ||
ObjCSpaceBeforeProtocolList: false | ||
PenaltyBreakBeforeFirstCallParameter: 1 | ||
PenaltyBreakComment: 300 | ||
PenaltyBreakFirstLessLess: 120 | ||
PenaltyBreakString: 1000 | ||
PenaltyExcessCharacter: 1000000 | ||
PenaltyReturnTypeOnItsOwnLine: 200 | ||
PointerAlignment: Left | ||
ReflowComments: true | ||
SortIncludes: true | ||
SpaceAfterCStyleCast: false | ||
SpaceBeforeAssignmentOperators: true | ||
SpaceBeforeParens: ControlStatements | ||
SpaceInEmptyParentheses: false | ||
SpacesBeforeTrailingComments: 1 | ||
SpacesInAngles: false | ||
SpacesInContainerLiterals: true | ||
SpacesInCStyleCastParentheses: false | ||
SpacesInParentheses: false | ||
SpacesInSquareBrackets: false | ||
Standard: Cpp11 | ||
TabWidth: 8 | ||
UseTab: Never | ||
... |
Oops, something went wrong.