DistAlgo is a very high-level language for programming distributed algorithms. This project implements a DistAlgo compiler with Python as the target language. In the following text, the name 'DistAlgo' refers to the compiler and not the language.
DistAlgo requires Python version 3.4 or higher, which can be obtained
from http://www.python.org. This document assumes that your installation
uses the default name python for the Python executable.
NOTE: If your system has both Python 2.x and Python 3.x installed, your
Python executable is likely Python 2. In that case, you should replace
python with python3 (or pythonX.Y where 'X.Y' is the exact Python
version you want to use) in all following command line examples. To find
out which version of Python is installed on your system, type:
python --version
DistAlgo has been tested on GNU/Linux and Microsoft Windows. The command line instructions given in this document use GNU Bash syntax. If you are using a different shell (e.g., Windows 'cmd.exe' shell), please adjust the commands accordingly.
In all following commands, please replace <DAROOT> with the path of the
DistAlgo root directory (the directory containing this file).
Installation of DistAlgo is optional for running DistAlgo. You can install
DistAlgo by using the Python distutils module, by adding the DistAlgo
root directory to your PYTHONPATH environment variable, or by using the
Python package manager, pip.
To see full usage description, type:
cd <DAROOT>; python setup.py --help
The following command installs DistAlgo as system-wide package:
cd <DAROOT>; python setup.py install
The following command installs DistAlgo for the current user:
cd <DAROOT>; python setup.py install --user
If you have installed DistAlgo for both system and user, the user installation takes precedence.
Simply add the DistAlgo root directory to your PYTHONPATH environment
variable:
export PYTHONPATH=<DAROOT>:${PYTHONPATH}
This takes precedence over distutils installations.
After installation, the da module will be available for use.
pip is a command line utility for installing Python packages from the
Python Package Index(PyPI). pip is the recommended method of installing
DistAlgo. Using pip, you do not need to manually download the DistAlgo
distribution package or setup environment variables, as pip will manage
all of that for you. The name of the DistAlgo package on PyPI is
'pyDistAlgo'.
To install DistAlgo as a system-wide package:
pip install pyDistAlgo
This command will likely require administrator privileges.
To install DistAlgo for the current user only:
pip install --user pyDistAlgo
Just as with distutils, if you have installed DistAlgo for both system
and the current user, the user installation will take precedence.
To upgrade an existing DistAlgo installation:
pip install --upgrade [--user] pyDistAlgo
Directory <DAROOT>/bin contains two scripts, dac and dar, that run
the DistAlgo compiler and runtime, respectively. Running these scripts
properly sets sys.path so no installation is needed. To avoid typing
<DAROOT>/bin in running the scripts, add it to your PATH environment
variable:
export PATH=<DAROOT>/bin:${PATH}
NOTE: The scripts assume your Python executable is installed to
/usr/bin/python3; if that is not the case, you must modify the first line
in the scripts to point to your Python executable.
For Windows only: The Windows program loader does not recognize the
"Shebang" (#!) sequence, so scripts dac and dar will not work under
the 'cmd.exe' shell. To work around this limitation, the '/bin'
directory also contains dac.bat and dar.bat, which are simple batch
script wrappers for dac and dar. To use these batch scripts under
Windows, substitute dac and dar with dac.bat and dar.bat,
respectively, for all following examples. The batch scripts assume your
Python executable is python, if that is not the case then you must
modify the batch files with the full path to your Python executable.
The DistAlgo system consists of a compiler and a runtime. Under normal circumstances, you do not need to invoke the compiler directly, because the runtime will call the compiler if it detects that the generated Python code for your DistAlgo source file is missing or outdated.
For both the compiler and runtime, use command line argument -h to see a
full description of available options.
If you have installed DistAlgo, run module da.compiler, passing a
DistAlgo source file <SOURCE> as argument:
python -m da.compiler <SOURCE>
Otherwise, call the dac script (adding prefix <DAROOT>/bin/ if you did
not add <DAROOT>/bin to your PATH variable):
dac <SOURCE>
To run DistAlgo programs, run the da module, passing a DistAlgo source
file as argument:
python -m da <SOURCE>
or call the dar script:
dar <SOURCE>
The --version(or -v) command line option will print out the version
number of the installed DistAlgo system. The --help(or -h) command
line option will print out a list of all command line options along with
a brief description of each option.
Instead of passing a path to a DistAlgo source file on the command line, you can use the '-m' option to run a DistAlgo module as though it were a script:
python -m da -m <MODULE>
The DistAlgo command line option '-m' mimics Python's own '-m' option.
<MODULE> must be a DistAlgo module in dotted form and without the '.da'
suffix. The source file for the module is located for using the same
rules that govern Python's own module loading process.
Command line arguments before the <SOURCE> argument are passed to the
DistAlgo runtime; arguments after the <SOURCE> argument are passed to
the DistAlgo source program in the global sys.argv list. Alternatively,
if you are using the '-m' option to run a DistAlgo module, command line
arguments before the -m <MODULE> argument are passed to the DistAlgo
runtime; arguments after the -m <MODULE> argument are passed to the
DistAlgo source program in the global sys.argv list.
For example, the following command passes argument -i to the DistAlgo
runtime, and passes arguments a and 1 to source program mutex.da:
dar -i mutex.da a 1
If you wish to quit your program before it terminates, press Ctrl-C.
Depending on the timing of this interrupt, you may see some exceptions
being thrown. This is expected behavior; simply press Ctrl-C again to
fully terminate the program.
For Cygwin with native Python only: If you are running native Windows
Python under a Cygwin terminal, Ctrl-C will not propagate to the child
processes: only the parent process will be killed, and the children will
continue to run in the background. You need to manually terminate the child
processes from the Windows task manager.
This section assumes you have installed DistAlgo; otherwise, please replace
python -m da with dar or with <DAROOT>/bin/dar.
The following command runs the Lamport mutual exclusion example:
python -m da -m da.examples.lamutex.orig
The following command runs the same program, but passes 20 to orig.da,
causing the program to create 20 processes:
python -m da -m da.examples.lamutex.orig 20
The following command runs the same program, but passes -f to the
runtime, causing a log file to be created for this run:
python -m da -f -m da.examples.lamutex.orig
After running the above command, you should find a file orig.da.log under
the current directory.
The following command runs the same program, but passes -L debug to the
runtime, causing debugging output to be printed to the console:
python -m da -L debug -m da.examples.lamutex.orig
When you start a DistAlgo program, a special DistAlgo process known as the
"node process" is created. The node process is responsible for running the
main method of the program.
By default, the node process is unnamed, and as such will not be able to
talk to other node processes, and any DistAlgo process running on an
unnamed node will not be able to communicate with DistAlgo processes
running on other nodes. In order to have multiple node processes that
communicate with each other, you must give each one a unique name. A node
name can be any string that does not include the characters '@', '#', and
':'. A node process can be named by using the -n <NAME> command line
argument. For example, the command
python -m da -n Node1 -m da.examples.lamutex.orig
will start a node named 'Node1', which then runs the mutual exclusion example. Notice that the formatted process ids in the output of this command now include a "#Node1" suffix, to indicate that the processes are running on the 'Node1' node.
Node names can be used for the at argument when calling the new
function, which instructs the system to create the new processes on the
named node(s) instead of locally. The following program, 'pingpong.da',
creates a Pong process on the node named 'PongNode', then creates a
Ping process on the local node, and finally starts both:
class Ping(process):
def setup(pong): pass
def run():
send(('Ping',), to=pong)
await(received(('Pong',)))
output("Ponged.")
class Pong(process):
def setup(): pass
def run():
await(some(received(('Ping',), from_=ping)))
output("Pinged.")
send(('Pong',), to=ping)
def main():
pong = new(Pong, args=(), at='PongNode')
ping = new(Ping, args=(pong,))
start(pong)
start(ping)
To run this example program, first start a node named anything other than 'PongNode', for example 'PingNode', and tell it to run 'pingpong.da':
python -m da -n PingNode pingpong.da
'PingNode' will run the main method and attempt to create a Pong
process on a node named 'PongNode'. But since it does not yet know which
node is 'PongNode', it will block at the first new statement waiting to
resolve the name 'PongNode'.
In order for the program execution to continue, you must start another node and name it 'PongNode':
python -m da -n PongNode -D pingpong.da
The same 'pingpong.da' file has to be specified on the command line,
because the 'PongNode' needs to have access to the Pong class in order
to create Pong processes. The command line parameter '-D'(or
equivalently, '--idle') tells the system to create an "idle" node. Idle
nodes do not execute their main method, and their only responsibility is
to create DistAlgo processes on behalf of other nodes. If you omit '-D',
then 'PongNode' will also run the main method, creating an additional
Ping and Pong process each, which may or may not be your desired
outcome.
At this point, 'PingNode' will be able to resolve the name 'PongNode', and
the execution of main can continue as usual. After 'PingNode'
terminates, you should be able to observe the line
pingpong.Ping<Ping:eb002#PingNode>:OUTPUT: Ponged.
on the terminal running the 'PingNode', and the line
pingpong.Pong<Pong:54802#PongNode>:OUTPUT: Pinged.
on the terminal running the 'PongNode' (the 5 hex-digit process id values may differ).
By default, DistAlgo nodes and processes are only able to communicate with nodes and processes running on the same host. This is because they only listen for incoming messages on the interface named 'localhost'. The hostname 'localhost' usually resolves to the loopback interface '127.0.0.1', which can only receive messages that were sent from the local host.
In order to run a distributed DistAlgo system with nodes on multiple hosts, you need to perform two additional operations:
-
instruct the DistAlgo processes to listen on an interface that is able to receive network messages from remote hosts, by using the '--hostname'(or equivalently, '-H') command line parameter;
-
instruct DistAlgo nodes to connect to an existing node running on a remote host, by using the '--peer'(or equivalently, '-R') command line parameter.
You can specify either a hostname that resolves to a local interace, or the
IP address of a local interface as argument to '-H'. To obtain a list of
local interfaces and their corresponding IP addresses, use ifconfig on
Linux systems, or ipconfig on Windows. The special IP address '0.0.0.0'
can be used to indicate all available interfaces on the local host.
As an example, consider the scenario of running the 'pingpong' program on two hosts, 'jupiter' and 'io'. First, start the 'PingNode' on host 'jupiter', instructing the system to listen on all interfaces:
jupiter> python -m da -H 0.0.0.0 -n PingNode pingpong.da
You can now start the 'PongNode' on host 'io', telling it to connect to the node on 'jupiter':
io> python -m da -H 0.0.0.0 -R jupiter -n PongNode -D pingpong.da
The DistAlgo nodes network are conceptually fully connected, so for a new node to join a network of existing nodes, it is sufficient to specify the address of any node in the existing network. Continuing the previous example, suppose you want to start another node, 'PongNode2', on the host 'thebe' and have it join the network of 'PingNode' and 'PongNode' already running on 'jupiter' and 'io', then the commands
thebe> python -m da -H 0.0.0.0 -R jupiter -n PongNode2 -D pingpong.da
and
thebe> python -m da -H 0.0.0.0 -R io -n PongNode2 -D pingpong.da
will have the same effect.
In a DistAlgo system involving multiple nodes, a pre-shared secret key, known as a "cookie", can be used to authenticate processes and prevent unauthorized processes from sending messages to DistAlgo processes. Cookies can be set when starting a node process, and any DistAlgo processes started on that node will automatically inherit its cookie value. You can set the cookie for a node by using the '--cookie' command line option:
python -m da --cookie SECRET -n PongNode pingpong.da
In this case, any process that does not have a matching cookie will not be able to send messages to 'PongNode' or any DistAlgo process running on 'PongNode'.
Alternatively, you can store the cookie value in a file named '.da.cookie' under you home directory:
echo -n "SECRET" > ${HOME}/.da.cookie
chmod 600 ${HOME}/.da.cookie
This way, all DistAlgo nodes will automatically use the contents of '${HOME}/.da.cookie' as their cookie, unless you explicitly specify one on the command line using '--cookie'.
As a special case, when an unnamed node is started, it sets its cookie to a random value, thus preventing this node and any DistAlgo processes created by this node from accidentally communicating with other nodes and processes.
SECURITY WARNING: Any remote or local process that knows your cookie and can send UDP packets to the UDP port or make TCP connections to the TCP port used by any DistAlgo process, will be able to trick the DistAlgo system into executing arbitrary code on your system. Never share your cookie with untrusted parties!
For a full description of the DistAlgo language, see
<DAROOT>/doc/language.pdf. For a quick reference of all DistAlgo
built-in functions, run the following command:
python -m da -B