Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
fbshipit-source-id: 7c98b4413a035219f8a2949626de33424f6182e5
  • Loading branch information
facebook-github-bot committed Aug 9, 2021
0 parents commit 719915b
Show file tree
Hide file tree
Showing 1,225 changed files with 659,196 additions and 0 deletions.
89 changes: 89 additions & 0 deletions .circleci/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
CircleCi integration is controlled by the `./circleci/config.yml` file. Our
config currently contains two workflows. One is triggered on every pull request update.
The other workflow runs nightly to verify our compatibility with prestodb internal protocol.

The PR workflow is named `dist-compile` and has 4 jobs, 2 to build and run unit tests on linux and macos
and 2 to check code formatting and license headers:
* linux-build
* macos-build
* format-check
* header-check

## Running locally

The linux container based jobs can be run locally using the `circleci` cli:

```
circleci local execute --job JOB_NAME
```

For example to run unit tests use:

```
circleci local execute --job linux-build
```

A Nightly build with prestodb/master sync checks that the presto_protocol library
remains in sync with Presto Java.

Run the nightly sync job locally:
```
circleci local execute --job presto-sync
```

## Install CircleCi cli
```
curl -fLSs https://circle.ci/cli | bash
```

To use containers Docker must be installed. Here are instructions to [Install
Docker on macos](https://docs.docker.com/docker-for-mac/install/). Docker deamon
must be running before issuing the `circleci` commands.

### Macos testing

Macos testing is done by using the CircleCi macos executor and installing
dependencies each time the job is run. This executor cannot be run locally.
The script `scripts/setup-macos.sh` contains commands that are run as part of
this job to install these dependencies.

### Linux testing

Linux testing uses a Docker container. The container build depends on the Velox CircleCi container. Check
f4d/.circleci/config.yml to see that the base container in circleci-container.dockfile is using the latest.
The container build uses Docker and should be run on your macos or linux laptop with Docker installed and
running.

#### Build the base container:

* In an up-to-date clone of f4d (maybe you have one?)

```
git clone [email protected]:facebookexternal/f4d.git
cd f4d
make base-container
```
* Wait - This step takes rather a long time. It is building clang-format v8 to be compatible with fbcode
* When the base container is finished the new container name will be printed on the console.
* Push the container to DockerHub
```
docker push prestocpp/base-container:$USER-YYYYMMDD
```
* After the push, update `scripts/velox-container.dockfile` with the newly build base container name

#### Build the dependencies container

* If you have a new base-container update scripts/velox-container.dockfile to refer to it
* Build the velox container
```
make velox-container.dockfile
```
* Wait - This takes a few minutes, but not nearly as long as the base container.
* When the velox container is finished the new container name will be printed on the console.
* Push the container to DockerHub
```
docker push prestocpp/velox-container:$USER-YYYYMMDD
```
* Update `.circleci/config.yml` with the newly built circleci container name.
There are two places in the config.yml file that refer to the container, update
both.
130 changes: 130 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: 2.1
workflows:
version: 2
dist-compile:
jobs:
- linux-build
- macos-build
- format-check
- header-check

executors:
build:
docker:
- image : prestocpp/velox-circleci:mikesh-20210610
resource_class: xlarge
environment:
CC: /opt/rh/gcc-toolset-9/root/bin/gcc
CXX: /opt/rh/gcc-toolset-9/root/bin/g++
check:
docker:
- image : prestocpp/velox-check:mikesh-20210609

jobs:
macos-build:
macos:
xcode: "11.7.0"
steps:
- checkout
- restore_cache:
name: "Restore Dependency Cache"
key: velox-circleci-macos-deps-{{ checksum ".circleci/config.yml" }}-{{ checksum "scripts/setup-macos.sh" }}
- run:
name: "Install dependencies"
command: |
set -xu
if [[ ! -e ~/deps ]]; then
mkdir ~/deps ~/deps-src
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C ~/deps
PATH=~/deps/bin:${PATH} DEPENDENCY_DIR=~/deps-src INSTALL_PREFIX=~/deps PROMPT_ALWAYS_RESPOND=n ./scripts/setup-macos.sh
rm -rf ~/deps/.git ~/deps/Library/Taps/ # Reduce cache size by 70%.
fi
- save_cache:
name: "Save Dependency Cache"
key: velox-circleci-macos-deps-{{ checksum ".circleci/config.yml" }}-{{ checksum "scripts/setup-macos.sh" }}
paths:
- ~/deps
- run:
name: "Calculate merge-base for CCache"
command: git merge-base origin/master HEAD > merge-base
- restore_cache:
name: "Restore CCache cache"
keys:
- velox-ccache-{{ arch }}-{{ checksum "merge-base" }}
- run:
name: "Build on MacOS"
command: |
export PATH=~/deps/bin:${PATH}
mkdir -p .ccache
export CCACHE_DIR=$(pwd)/.ccache
ccache -sz -M 5Gi
cmake -B _build/debug -GNinja -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=~/deps -DCMAKE_CXX_COMPILER_LAUNCHER=ccache
ninja -C _build/debug
ccache -s
no_output_timeout: 1h
- save_cache:
name: "Save CCache cache"
key: velox-ccache-{{ arch }}-{{ checksum "merge-base" }}
paths:
- .ccache/

linux-build:
executor: build
steps:
- checkout
- run:
name: "Calculate merge-base for CCache"
command: git merge-base origin/master HEAD > merge-base
- restore_cache:
name: "Restore CCache cache"
keys:
- velox-ccache-{{ arch }}-{{ checksum "merge-base" }}
- run:
name: Build
command: |
mkdir -p .ccache
export CCACHE_DIR=$(realpath .ccache)
ccache -sz -M 5Gi
source /opt/rh/gcc-toolset-9/enable
make debug NUM_THREADS=8 MAX_HIGH_MEM_JOBS=3 MAX_LINK_JOBS=4
ccache -s
no_output_timeout: 1h
- store_artifacts:
path: '_build/debug/.ninja_log'
- run:
name: Run Unit Tests
command: |
make unittest NUM_THREADS=8
no_output_timeout: 1h
- save_cache:
name: "Save CCache cache"
key: velox-ccache-{{ arch }}-{{ checksum "merge-base" }}
paths:
- .ccache/

format-check:
executor: check
steps:
- checkout
- run:
name: Check formatting
command: 'make format-check'

header-check:
executor: check
steps:
- checkout
- run:
name: Check license headers
command: 'make header-check'
87 changes: 87 additions & 0 deletions .clang-format
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
AccessModifierOffset: -1
AlignAfterOpenBracket: AlwaysBreak
AlignConsecutiveAssignments: false
AlignConsecutiveDeclarations: false
AlignEscapedNewlinesLeft: true
AlignOperands: false
AlignTrailingComments: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortBlocksOnASingleLine: false
AllowShortCaseLabelsOnASingleLine: false
AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
BinPackArguments: false
BinPackParameters: false
BraceWrapping:
AfterClass: false
AfterControlStatement: false
AfterEnum: false
AfterFunction: false
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Attach
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BreakAfterJavaFieldAnnotations: false
BreakStringLiterals: false
ColumnLimit: 80
CommentPragmas: '^ IWYU pragma:'
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ConstructorInitializerIndentWidth: 4
ContinuationIndentWidth: 4
Cpp11BracedListStyle: true
DerivePointerAlignment: false
DisableFormat: false
ForEachMacros: [ FOR_EACH, FOR_EACH_R, FOR_EACH_RANGE, ]
IncludeCategories:
- Regex: '^<.*\.h(pp)?>'
Priority: 1
- Regex: '^<.*'
Priority: 2
- Regex: '.*'
Priority: 3
IndentCaseLabels: true
IndentWidth: 2
IndentWrappedFunctionNames: false
KeepEmptyLinesAtTheStartOfBlocks: false
MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 1
NamespaceIndentation: None
ObjCBlockIndentWidth: 2
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: false
PenaltyBreakBeforeFirstCallParameter: 1
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
PointerAlignment: Left
ReflowComments: true
SortIncludes: true
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 1
SpacesInAngles: false
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
Standard: Cpp11
TabWidth: 8
UseTab: Never
...
Loading

0 comments on commit 719915b

Please sign in to comment.