Winddb

Introduction
--------------------------------------------------------------------------------
winddb is a system to merge datasamples from several types of weather station
databases and export it to the web in a common JSON-format. winddb does not
store any weather data on its own.

The winddb output can be distributed to provide an API for apps to access data
from several types of weather stations in a uniform way -- regardless of brand,
model or connectivity. Examples of consumers are web sites and mobile apps.

winddb maintains a database of weather stations, geographic location, name and
other sorts of metadata. Each station is tied to a data source. Data sources
are implemented in a plugin-like fashion in winddb, one for each type of station.
Data sources allow winddb to fetch data samples without knowing details about
how they are stored or gathered.

Multiple stations may use the same data-source, since datasamples are grouped by
station name.

Dependencies:
--------------------------------------------------------------------------------
- Python 2.x (tested with v2.7.3)
- Python JSON ("apt-get install python-simplejson" on debian)
- The AWSXD source also requires MySQL interface support.
  ("apt-get install python-mysqldb" on debian).

Configuration:
--------------------------------------------------------------------------------
winddb itself relies mostly on command-line arguments for configuration.
Therefore, it is advised to review "winddb.py --help" carefully.

In addition, each source may require individual configuration. By default winddb
reads winddb.conf, where each source has its own block with configuration keys.
Review the winddb.conf.sample for details.
As an example, the awsxd data-source needs to be configured with MySQL details,
and the osod data-source needs to know the path of the osod sqlite database.

To actually output anything, the winddb sqlite-database (default wind.db) must
be populated. Each row in the "stations" table maps to a weather station. The
handler column maps the weather station to a specific data source. For example
to add a DPS AWS-X weather station, the handler should be set to "awsxd".

Supported handlers:
awsxd
vivad
osod

Output reference
--------------------------------------------------------------------------------
This section describes the contents of the files output by winddb.

$ROOT                                   Root directory
$ROOT/index.json                        Station index
$ROOT/$ID/$TYPE/$FUNC/latest.json       Last n samples of type $TYPE.
$ROOT/$ID/$TYPE/$FUNC/samples.json      Last sample of type $TYPE.

The $ROOT directory is specified using command-line argument -o to winddb.

index.json contains a list of all stations monitored by winddb. Each output
station is an object with the following keys:

Member        Type    Description
------        ------  -----------
id            string  Unique station identifier.
description   string  Station descrition. May be null.
pos_lon       float   Geographic location, longitude component. May be null.
pos_lat       float   Geographic location, latitude component. May be null.
pollrate      int     Expected update rate, in seconds.
friendlyname  string  Complete station name. A more human-readable version of
                      the id. Typically used for user presentation.
caps          list    Station capabilities. This list describes what type of data
                      samples the station offers (e.g wind speed, temperature).
                      Each capability is a tuple with the type of data and the
                      group function associated with the data. For example
                      ('wspeed', 'avg') indicates the station is capable of
                      delivering average windspeed samples. ('wspeed', 'max')
                      indicates it is capable of delivering the maximum
                      windspeed samples. See source.py for possible types and
                      group functions.

For each listed station, there is a subdirectory named according to the station
ID (see $ID in the directory scheme above). The subtree holds the actual sample
data for the given station. $TYPE is the type of sample, $FUNC is the goup
function. For example:

    $ROOT/onsala/wspeed/avg

would hold the average windspeed samples for the station named 'onsala'. Each
of these dirs hold two files, latest.json and samples.json. Latest.json contains
the latest sample of the given type ($TYPE, $FUNC) while samples.json holds a
list of the n most recent samples of the given type ($TYPE, $FUNC).

The list of samples in samples.json is always ordered by the timestamp in
descending order.

In addition to the .json files (index, latest, history), winddb by default also
outputs the corresponding JSONP files. They contain the same JSON structure, but
wrapped in a javascript function call. This is primarily used to bypass same
origin policy that most browsers implement. The function name is set using
command-line argument -c.

API guidelines
--------------------------------------------------------------------------------
It is recommended to output the generated files to a subdir of your webroot.
This allows for thirdparty applications (such as mobile apps) to download them
using a standard HTTP request.

For each station, winddb provides the most recent sample aswell as a log of the
n most recent samples. Typically, applications want to download the history file
once during initialization (to populate graphs) and then append with new data
as samples arrive.

Since each station may use different samplerate, it is not recommended to use a
fixed rate for fetching the latest sample. To prevent unnecessary hammering,
the recommended pollrate for each station is defined by the station index file.

In addition to the pollrate, the index file also contains GPS coordinates and
other miscellaneous station infos. The application should only have to download
this file once every session.

* Do not hammer, use the pollrate for the given station.

* Warn user about old samples. Do not assume lastest sample is NEW, check the
dates!

* Do not cache station info in between session. Stations may become inactive at
any time.