The Steward Alerts for Science System

Welcome to the Steward Alerts for Science Sytem (SASSy).

This work is heavily based upon MARS, the Las Cumbras Observatory alert broker, and extended to ingest both ZTF and TNS public alerts. We also support LIGO data scraped from TNS and both the GLADE and GWGC catalogs.

Code modified in this system originated from the following sources:

• http://github.com/lcogt/ztf-alert-server
• https://github.com/ZwickyTransientFacility/ztf-avro-alert
• https://wis-tns.weizmann.ac.il/sites/default/files/api/api_bulk_report.zip
• https://github.com/ericmandel/js9

Pre-Requisites

• Linux (we use Ubuntu 18.04.2 LTS)
• Python 3.6.6 (it will not work with Python < 3.6.5)
• PostGreSQL 12.x (but will probably work with earlier versions)
• JS9
• Bootstrap 4.5.x
• jQuery 3.5.1
• An account at TNS (optional)
• A Memorandum of Understanding (MoU) with ZTF to ingest alerts in real-time.
  • If you don't have a MoU, daily alerts are available at the University of Washington.

Get The Software

• Obtain a copy of the software from GitHub.

• Install dependencies:

  % pip3 install --upgrade pip
  % pip3 install -r requirements.txt
  

Create User

As root, create a sassy user (change the password to suit your site):

 % adduser sassy --gecos "sassy, unknown, (xxx) xxx-xxxx, unknown" --disabled-password
 % echo "sassy:secretsanta" | chpasswd
 % usermod -aG sudo sassy
 % usermod -aG users sassy

Create and Populate The Database and Tables

NB: All utility bash-shell scripts in ${SASSY_BIN} support the --help argument for further information and the --dry-run option to show executable commands without invoking them. You would be well-advised to use them!

• Create the sassy database

  To create a database called sassy with username sassy and password my_secret (using the PostGreSQL server defaults of localhost:5432), execute:

  % bash ${SASSY_BIN}/sassy.database.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.database.sh
  % bash ${SASSY_BIN}/sassy.database.sh --database=sassy --password=my_secret --username=sassy

• Create the tables (using the same credentials as defined above):

  % bash ${SASSY_BIN}/sassy.glade.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.glade.sh
  % bash ${SASSY_BIN}/sassy.glade.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.glade_q3c.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.glade_q3c.sh
  % bash ${SASSY_BIN}/sassy.glade_q3c.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.gwgc.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.gwgc.sh
  % bash ${SASSY_BIN}/sassy.gwgc.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.gwgc_q3c.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.gwgc_q3c.sh
  % bash ${SASSY_BIN}/sassy.gwgc_q3c.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.ligo.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.ligo.sh
  % bash ${SASSY_BIN}/sassy.ligo.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.ligo_q3c.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.ligo_q3c.sh
  % bash ${SASSY_BIN}/sassy.ligo_q3c.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.tns.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.tns.sh
  % bash ${SASSY_BIN}/sassy.tns.sh --database=sassy --password=my_secret --username=sassy

  % bash ${SASSY_BIN}/sassy.tns_q3c.sh --database=sassy --password=my_secret --username=sassy --dry-run
  % cat /tmp/sassy.tns_q3c.sh
  % bash ${SASSY_BIN}/sassy.tns_q3c.sh --database=sassy --password=my_secret --username=sassy

• Populating the database tables

  To populate the GLADE tables, execute:

  % cd ${SASSY_UTILS}
  % python3 ${SASSY_SRC}/models/glade.py --catalog --output=glade.local.csv
  % python3 ${SASSY_UTILS}/glade_read.py --file=glade.local.csv
  % python3 ${SASSY_UTILS}/glade_q3c_read.py --file=glade.local.csv

  To populate the GWGC tables, execute:

  % cd ${SASSY_UTILS}
  % python3 ${SASSY_SRC}/models/gwgc.py --catalog >> gwgc.dat
  % python3 ${SASSY_UTILS}/gwgc_read.py --file=gwgc.dat
  % python3 ${SASSY_UTILS}/gwgc_q3c_read.py --file=gwgc.dat

  The ligo, ligo_q3c, tns and tns_q3c tables can be populated using the web-scraping code via the crontab entries.

  The ZTF alerts table is populated by the nightly crontab ingestion or manually via (Eg to retrieve the data for the first night and assuming you have modified the scripts to suit your local installation):

  % bash ${SASSY_CRON}/ztfgz.pull.sh --date=20180601 --dry-run
  % bash ${SASSY_CRON}/ztfgz.pull.sh --date=20180601
  % bash ${SASSY_CRON}/ztfgz.unpack.sh --date=20180601 --dry-run
  % bash ${SASSY_CRON}/ztfgz.unpack.sh --date=20180601
  % bash ${SASSY_CRON}/ztfgz.updatedb.sh --date=20180601 --dry-run
  % bash ${SASSY_CRON}/ztfgz.updatedb.sh --date=20180601

• Database entity-relationship diagram

  If you have eralchemy installed, an entity-relationship diagram can be generated:

  % eralchemy -i "postgresql+psycopg2://${SASSY_DB_USER}:${SASSY_DB_PASS}@${SASSY_DB_HOST}:${SASSY_DB_PORT}/${SASSY_DB_NAME}" -o ${SASSY_DB_NAME}.pdf

• Database utilities

  We provide the following, generic, utilities for database manipulation:

  % bash ${SASSY_BIN}/psql.size.sh --help
  % bash ${SASSY_BIN}/psql.backup.sh --help
  % bash ${SASSY_BIN}/psql.restore.sh --help

Create CRON Jobs

• See ${SASSY_CRON}/README.md for further information

Configure For Local Site

You should now copy ${SASSY_BIN}/Sassy.template.sh, ${SASSY_ETC}/Sassy.template.sh and ${SASSY_SRC}/common.template.py and edit the copies to suit your site and change:

• local installation directory
• database server and credentials
• mail server and credentials

% cp ${SASSY_BIN}/Sassy.template.sh ${SASSY_BIN}/Sassy.sh
% vi ${SASSY_BIN}/Sassy.sh
% cp ${SASSY_ETC}/Sassy.template.sh ${SASSY_ETC}/Sassy.sh
% vi ${SASSY_ETC}/Sassy.sh
% cp ${SASSY_SRC}/common.template.py ${SASSY_SRC}/common.py
% vi ${SASSY_SRC}/common.py

JS9, JavaScript and Bootstrap

We do not ship these items. Please refer to the README.md files in the static sub-directories for getting and installing these products. The web-site will not work without them!

Quick Start Guide

If you carried out the above, and assuming the codebase is in /home/sassy/SASSy, you can start the application:

• Local development server

  % cd /home/sassy/SASSy
  % source etc/Sassy.sh `pwd` dev
  % bash ${SASSY_BIN}/Sassy.sh --type=dev --source=/home/sassy/SASSy --command=start --dry-run
  % bash ${SASSY_BIN}/Sassy.sh --type=dev --source=/home/sassy/SASSy --command=start

  then point a browser to the local ORP development server.

• Local production server

  % cd /home/sassy/SASSy
  % source etc/Sassy.sh `pwd` prod
  % bash ${SASSY_BIN}/Sassy.sh --type=prod --source=/home/sassy/SASSy --command=start --dry-run
  % bash ${SASSY_BIN}/Sassy.sh --type=prod --source=/home/sassy/SASSy --command=start

  then point a browser to the local ORP production server.

• WSGI

  If you wish to run a WSGI-based server, we provide:

  • ${SASSY_HOME}/sassy.wsgi file which should work as-is
  • ${SASSY_HOME}/sassy.conf which should be edited: % sed 's/__myhost__/127\.0\.0\.1/g' >> /etc/apache2/sites-available/sassy.conf (or whatever IP address you have)
  • Enable the site via Apache2 in the usual way

---

Last Updated: 20200702