Skip to content

readme & install files modifications #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

readme & install files modifications #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 62 additions & 27 deletions INSTALL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,62 @@
Assuming $VFLOW_ROOT is the base installation directory. The CMAKE_BUILD_TYPE can be changed to Debug or Release.
The following dependencies have to be installed first:

#veccore
cd $VFLOW_ROOT
git clone https://github.com/root-project/veccore.git
mkdir $VFLOW_ROOT/veccore_build && cd $VFLOW_ROOT/veccore_build
cmake $GEANT_ROOT/veccore -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/veccore_install -DBUILD_VC=ON -DBUILD_UMESIMD=ON
make -j install

#vecmath
cd $VFLOW_ROOT
git clone https://github.com/root-project/vecmath.git
mkdir $VFLOW_ROOT/vecmath_build && cd $VFLOW_ROOT/vecmath_build
cmake $VFLOW_ROOT/vecmath -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vecmath_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DVecCore_DIR=$VFLOW_ROOT/veccore_install/share/VecCore/cmake

#vecgeom
cd $VFLOW_ROOT
git clone https://gitlab.cern.ch/VecGeom/VecGeom.git
mkdir $VFLOW_ROOT/vecgeom_build && cd $VFLOW_ROOT/vecgeom_build
cmake $VFLOW_ROOT/VecGeom -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vecgeom_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=$VFLOW_ROOT/veccore_install -DBACKEND=Vc

The package can be installed using:

cd $VFLOW_ROOT
mkdir $VFLOW_ROOT/vectorflow_build && cd $VFLOW_ROOT/vectorflow_build
cmake $VFLOW_ROOT/vectorflow -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vectorflow_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=$VFLOW_ROOT/veccore_install -DVecCore_DIR=$VFLOW_ROOT/veccore_install/share/VecCore/cmake -DVecMath_DIR=$VFLOW_ROOT/vecmath_install/lib/cmake/VecMath -DVecGeom_DIR=$VFLOW_ROOT/vecgeom_install/lib/cmake/VecGeom
**VectorFlow** has package dependencies that need to be installed first, those are VECCORE, VECMATH, and VECGEOM. Also, these packages has other dependencies that need to be addressed too. Here is a list:

## Tools needed for multiple purposes

* sudo apt-get install git dpkg-dev cmake g++ gcc binutils libx11-dev libxpm-dev libxft-dev libxext-dev libgd-dev doxygen curl graphviz

## ROOT (VecGeom dependency)

For Ubuntu OS users, the binaries can be found:

* wget https://root.cern.ch/download/root_v6.16.00.Linux-ubuntu18-x86_64-gcc7.3.tar.gz
* tar -zxf root_v6.16.00.Linux-ubuntu18-x86_64-gcc7.3.tar.gz

**NOTE 1:** To use ROOT, add the libraries to your LD_LIBRARY_PATH:

* export LD_LIBRARY_PATH=$LIB_PATH:$VFLOW_ROOT/root/lib

**NOTE 2:** This is a version of ROOT, for more information and/or different versions, please visit:

* https://root.cern.ch/content/release-61600
* https://root.cern.ch/build-prerequisites

# BUILDING THE PROJECT

Assuming $VFLOW_ROOT is the base installation directory. The CMAKE_BUILD_TYPE can be changed to Debug or Release. The following dependencies have to be installed first:

## VECCORE
* cd $VFLOW_ROOT
* git clone https://github.com/root-project/veccore.git
* mkdir $VFLOW_ROOT/veccore_build
* cd $VFLOW_ROOT/veccore_build
* cmake $VFLOW_ROOT/veccore -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/veccore_install -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-O2 -DBUILD_VC=ON -DBUILD_UMESIMD=ON
* make -j install

**NOTE:** To build VecCore benchmarks, add to cmake the option -DBUILD_BENCHMARKS=ON

## VECMATH
* cd $VFLOW_ROOT
* git clone https://github.com/root-project/vecmath.git
* mkdir $VFLOW_ROOT/vecmath_build
* cd $VFLOW_ROOT/vecmath_build
* cmake $VFLOW_ROOT/vecmath -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vecmath_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DVecCore_DIR=$VFLOW_ROOT/veccore_install/lib/cmake/VecCore
* make -j install

## VECGEOM
* cd $VFLOW_ROOT
* git clone https://gitlab.cern.ch/VecGeom/VecGeom.git
* mkdir $VFLOW_ROOT/vecgeom_build
* cd $VFLOW_ROOT/vecgeom_build
* cmake $VFLOW_ROOT/VecGeom -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vecgeom_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=$VFLOW_ROOT/veccore_install -DBACKEND=Vc -DROOT=ON -DROOT_DIR=$VFLOW_ROOT/root/cmake -DVECGEOM_VECTOR=avx2
* make -j install

## VECTORFLOW
* cd $VFLOW_ROOT
* mkdir $VFLOW_ROOT/vectorflow_build
* cd $VFLOW_ROOT/vectorflow_build
* cmake $VFLOW_ROOT/vectorflow -DCMAKE_INSTALL_PREFIX=$VFLOW_ROOT/vectorflow_install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=/usr/local -DVecCore_DIR=$VFLOW_ROOT/veccore_install/lib/cmake/VecCore -DVc_DIR=$VFLOW_ROOT/veccore_install/lib/cmake/Vc -DVecMath_DIR=$VFLOW_ROOT/vecmath_install/lib/cmake/VecMath -DVecGeom_DIR=$VFLOW_ROOT/vecgeom_install/lib/cmake/VecGeom -DROOT_DIR=$VFLOW_ROOT/root/cmake -DROOT=ON
* make -j install

**NOTE 1:** To build VectorFlow examples and benchmarks, add to cmake the options -DBUILD_TESTING=ON &&-DBUILD_EXAMPLES=ON

**NOTE 2:** To enable GPERFTOOLS for profiling, add to cmake the option -DGPERFTOOLS=ON (see howto_gperftools.md file for instructions in how to install it)
19 changes: 8 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
# vectorflow
C++ adapter API for integrating vectorized components in a scalar workflow.
# VECTORFLOW
C++ open source adapter that integrates SIMD vectorized components in a scalar workflow.

Many FLOP-intensive algorithms may profit from the vector pipelines of modern processors; they don’t because they don’t have vectorizable inner loops. The project idea is to implement and benchmark a generic vector flow service that can non-intrusively integrate with arbitrary data processing frameworks and can expose algorithms to the higher-level event loop of these frameworks.
Many FLOP-intensive high-energy physics algorithms may profit from the vector pipelines of modern processors, they don’t because they don’t have vectorizable inner loops. The API provides a generic *vector flow* service that can non-intrusively integrate with arbitrary data processing frameworks.

* The service allows to filter from the main data stream to extract data of interest
* Scalar data of arbitrary type can be accumulated in in vectors of a given size
* Once the vectors are filled to the requested size, the scalar data can be transformed in form of a structure of arrays
* The transformed data can be directly fed into a vector-aware implementation of the given algorithm. There is no constraint by the framework on how to implement vectorization: the . Vectorization will happen on elements of the SOA. Vectorization can be compiler driven, but the framework integrates with [VecCore](https://github.com/root-project/veccore) services.
* The output data can be scattered in scalar form and re-integrated in the framework data flow.
* The service allows the user to work only on data of interest. This is, a *FILTER* step to avoid unnecessary computations.
* The scalar data of arbitrary type can be accumulated in vectors of a given size (defined by the computer architecture). This is, a *GATHER* step to prepare the data for SIMD processing.
* The transformed data, then, can be directly fed into a vector-aware implementation of the given algorithm. This is, a *VECTORIZATION* step, which integrates with [VecCore](https://github.com/root-project/veccore) services.
* At the end, the output data can be pushed back to the initial scalar form. This is, a *SCATTER* step to re-integrate the output into the framework data flow.

Depending on the intrinsic algorithm gain from SIMD vectorization and better data caching, the overheads introduced by the extra data transformations can be much smaller than the benefits.

The library aims to provide an API and a set of examples demonstrating the transformation of the scalar algorithm in a vectorized one.
Depending on the intrinsic algorithm gain from the SIMD vectorization and better instruction caching, the inherent overheads introduced by the extra data transformations can be much smaller than the benefits. The library aims to provide an API and a set of benchmarked examples demonstrating the transformation of the scalar algorithm into a vectorized one.