README.txt

============================= Usage ========================================

$ # Run through "Installation" steps below, then:

$ git clone https://github.com/brk/foster foster
$ cd foster
$ mkdir _obj        # Separate directory for compilation results and such
$ cd _obj           # The name can be whatever you'd like, but it should
                    # be located outside of the foster source directory.
$ ccmake .. -G "Unix Makefiles"
        # Press 'c' to configure; if CMake cannot find the listed paths,
        #                         please set them by hand.
        # Press 'g' to generate Makefiles and exit once paths are OK

Once the first-time steps are done:

$ make            # Recompile any updated files.
$ make regress    # Run regression tests

After running regression tests, you can look for trace messages
from the middle-end with a command like this:

$ cat test-tmpdir/*/compile.log.txt | grep '...' | sort | uniq

Other commands of interest, from _obj:

$ make hs_clean    # Delete just the Haskell-generated .o files
$ make list_regression_tests

From project root:

$ scripts/countlines.sh  # Give an overview of where code is, volume-wise.

============================= Installation =================================
Paths:
  You'll want to append $PATH_TO_FOSTER/scripts to $PATH and $PYTHONPATH,
  probably in .bashrc or equivalent.

Dependencies (non-exhaustive):
  CMake, ANTLR + ANTLR3 C runtime, LLVM, Protocol Buffers, Z3
  Python (and several 3rd party Python libraries)
  Pin (optional, for benchmarking analysis)

Ubuntu 17.10 (likewise for 18.04):
  See   scripts/ubuntu-17.10.sh

  The lines in the above script related to installing ANTLR
  also apply to Mac OS X.

Mac OS X:
  You'll want to install the GNU time binary:
     brew install gnu-time       # or
     sudo port install gtime

  Also, SWIG for building LLDB:
     sudo port install swig swig-python

  # For CSmith:
  git clone https://github.com/csmith-project/csmith ~/sw/local/csmith.git
  cd ~/sw/local/csmith.git
  ./configure --prefix=$HOME/sw/local
  make


  # For Ott:
  #         sudo apt-get install  coq texlive-latex-base
  #         wget http://www.cl.cam.ac.uk/~pes20/ott/ott_distro_0.25.tar.gz
  #         tar xf ott_distro_0.25.tar.gz
  #         ...etc...

  # For emscripten:
  #         sudo apt-get install python-software-properties
  #         sudo apt-get update
  #         sudo add-apt-repository ppa:chris-lea/node.js
  #         sudo apt-get install nodejs npm

  # For CVC4, add the following lines to /etc/apt/sources.list::
  #
  #     deb http://cvc4.cs.nyu.edu/debian/ unstable/
  #     deb-src http://cvc4.cs.nyu.edu/debian/ unstable/
  #
  # Then::
  #
  #     $ sudo apt-get update
  #     $ sudo apt-get install cvc4 libcvc4-dev



Other libraries/tools:
        gperftools: https://code.google.com/p/gperftools/
        	If you install via deb, you'll need to install both...
        kcachegrind: http://kcachegrind.sourceforge.net/html/Home.html
        valgrind
        pin:        http://software.intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool
        leveldb:    https://code.google.com/p/leveldb/
        leveldb-go: https://code.google.com/p/leveldb-go/
        snappy:     https://code.google.com/p/snappy/
        cityhash:   https://code.google.com/p/cityhash/
        ocaml-core: https://bitbucket.org/janestreet/core

        hprotoc:    http://hackage.haskell.org/package/hprotoc


Networking
==========

On Linux:

        sudo ip tuntap add tun8 mode tun  # For IP-format frames
        sudo ip link   set tun8 up
        sudo ip addr   add 192.168.6.6/24 dev tun8
        Process open()s /dev/net/tun, then calls ioctl() with TUNSETIFF
                to get a fd for tun8
        (once a process is blocked reading from tun8:)
        ping 192.168.6.42 # if we ping 6.6 the kernel delivers packets to lo

On Mac OS X, first install tuntaposx <http://tuntaposx.sourceforge.net/>, then:

        Process open()s /dev/tun8
        Process calls system(sudo ifconfig tun8 192.168.6.41 192.168.6.42)
        ping 192.168.6.42

Then, once the appropriate network interfaces are up,
run (from the _obj) directory:

   $ fosterc ../test/speed/foster-posix/foster-net/foster-net.foster
   $ ../test/speed/foster-posix/foster-net/foster-net.foster.exe

   (and ping as above, from a separate terminal window)


Benchmarks
==========

Make sure that the foster/ directory is in your $PYTHONPATH,
then execute scripts/bench-all.py. The script will churn through
a few benchmarks (of both Foster code and reference non-Foster implementations).
After a few minutes, the script will spit out a path like
        data/2017-06-01@16.37.12/all_timings.json
which we'll call $ALLTIMINGS.

The bench-ize.py script will munge this data for display, in various ways.

        $ bench-ize.py --overview $ALLTIMINGS
        ...
        third_party/shootout/fannkuchredux [LLVMopt=O0,sse=no,lang=other,date=2017-06-01@16.37.12] {'inputs': ['10']}
                                        [LLVMopt=O2,sse=no,lang=other,date=2017-06-01@16.37.12] {'inputs': ['10']}
                                        [LLVMopt=O3,sse=yes,lang=other,date=2017-06-01@16.37.12] {'inputs': ['10']}
        speed/shootout/fannkuchredux [inline=yes,LLVMopt=O2,abc=unsafe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['10']}
                                [inline=no,LLVMopt=O0,abc=safe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['10']}
                                [inline=yes,LLVMopt=O2,abc=safe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['10']}
        speed/shootout/nbody [inline=yes,LLVMopt=O2,abc=unsafe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['350000']}
                        [inline=no,LLVMopt=O0,abc=safe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['350000']}
                        [inline=yes,LLVMopt=O2,abc=safe,donate=yes,lang=foster,date=2017-06-01@16.37.12] {'inputs': ['350000']}
        ...


        $ bench-ize.py --test spectralnorm $ALLTIMINGS
        ...
        Output written to 'bench-ized.html'



        $ bench-ize.py --test spectralnorm --tags LLVMopt=O2 $ALLTIMINGS



CSmith
======

We have a tool to translate a subset of C code into Foster.

The scripts/ directory includes a few wrappers to generate random C programs
using CSmith, translate the resulting code into Foster, and compare whether
the compiled C and Foster code produce the same output.

Running  csmith2foster.py  will generate twenty random C programs in the
current directory, translate and compile each one, and report the diff status.

Most of the time you'll see output like

        --------------------------------------------------
        Attempting test csf_003.c
        Test case csf_003.c skipped because the C code exited under murky circumstances.
        --------------------------------------------------
        Attempting test csf_004.c
        Compiling csf_004.c to Foster... (71 lines)
        Compiling and running the generated Foster code...
        Diffing... (1 lines)
        Test case csf_004.c passed
        --------------------------------------------------

Many of the programs generated by CSmith will fail at runtime,
usually with a floating point exception.

If we tickle a bug in the Foster compiler, it'll look like this:

        --------------------------------------------------
        Attempting test csf_001.c
        Compiling csf_001.c to Foster... (233 lines)
        Compiling and running the generated Foster code...
        Diffing... (1 lines)
        Test case csf_001.c             FAILED (wrong output)
        --------------------------------------------------

test_c2f.py will re-run tests on saved programs in the  test/c2f/  directory,
        or re-run tests on residual csmith2foster-generated programs
        (depending on how you edit the last few lines of the file).

You can run  c2foster  to translate a C file to Foster (on stdout).
Be aware that you'll need to provide search paths for #include files
explicitly. In particular, for CSmith-generated programs, you'll need
something like

        $ c2foster csf_001.c -I $HOME/sw/local/csmith-2.3.0/include/csmith-2.3.0 > csf_001.foster

c2foster will dump Clang's warning to stderr, but those can be ignored.

Running an individual C file can be done with  runclang  .
Note as before that you'll need to provide search paths explicitly.

Individual Foster programs can be run with  runfoster  .

You can see a (colorized) output of Clang's AST for a given C program
with   clang -fsyntax-only -Xclang -ast-dump  <includes> <program>

For piping to file, add -fno-color-diagnostics.