Titus pfarandom

Titus's pfarandom script generates a placeholder PFA document, given an input Avro schema and an output Avro schema. The action generated by this script ignores the input and produces random output. That is, the generated PFA file is determined, but the output generated when the scoring engine runs is random.

This script is typically only used for testing a scoring engine consumer. A particular application may need data in a specified form (specified by Avro schemas), but it's not important what that scoring engine does. The pfarandom script quickly makes such a test function.

Before you begin...

Download and install Titus. This article was tested with Titus 0.8.3; newer versions should work with no modification. Python >= 2.6 and < 3.0 is required.

Help document

A good place to start is pfarandom's own help text, which will always show the latest options.

usage: pfarandom [-h] [--check] [--exception EXCEPTION] [--name NAME]
                 [--randseed RANDSEED] [--doc DOC] [--version VERSION]
                 [--metadata METADATA] [--options OPTIONS]
                 input output [fileName]

Given an input and an output schema, generate a PFA file that only computes
random outputs (for tests).

positional arguments:
  input                 input schema file (Avro schema in JSON format)
  output                output schema file (Avro schema in JSON format)
  fileName              output PFA file name or "-" for standard out

optional arguments:
  -h, --help            show this help message and exit
  --check               if supplied, the PFA will be tested with 100 samples,
                        written to standard error
  --exception EXCEPTION
                        probability of runtime exception (must be between 0
                        and 1, inclusive)
  --name NAME           name field for output PFA file
  --randseed RANDSEED   randseed field for output PFA file (must be an
                        integer)
  --doc DOC             doc field for output PFA file
  --version VERSION     version field for output PFA file (must be an integer)
  --metadata METADATA   metadata field for output PFA file (must be a JSON map
                        of strings)
  --options OPTIONS     options field for output PFA file (must be a JSON map)

Examples

Create a simple input.avsc (integers) and a complex output.avsc (records), then test-run it in Hadrian-Standalone.

% cat > input.avsc
"int"
% cat > output.avsc
{"type": "record",
 "name": "Test",
 "fields": [{"name": "one", "type": "int"},
            {"name": "two", "type": "double"},
            {"name": "three", "type": "string"},
            {"name": "four", "type": {"type": "array", "items": "boolean"}}]}
% pfarandom input.avsc output.avsc test.pfa
% java -jar hadrian-standalone-0.8.3-jar-with-dependencies.jar -i json -o json test.pfa
1
2
3
4
5
6
7
8
9
10
{"one":1454025743,"two":0.12220112315358278,"three":"綱᪌㆓䫻❻າࣖ䜑綵㊑","four":[true,true,true,true,true,false,true]}
{"one":780849570,"two":0.6509713360242374,"three":"鞓烊啉꩓讴猴붏봯竻뿋","four":[false,true,true,true,true]}
{"one":1896044974,"two":0.11150257478213721,"three":"ꩬ䛍ꚏ᷂栵숌믄ᤱ獰캾","four":[]}
{"one":-314833561,"two":0.29795393310.8.34,"three":"痂쳊臎✫䟜㒌辻倭쓐粼","four":[true,true,false,false,false,true,true]}
{"one":905313595,"two":0.9472541162752725,"three":"䤬뾊뉇ӡ㜵盿掜푇菒铜","four":[true,true,false]}
{"one":684710844,"two":0.13660530607163113,"three":"顁덭策볃ᢽ醺埽뽻⵵ⴈ","four":[false,true,false,false,true,false,false]}
{"one":-2079849390,"two":0.2848117237909671,"three":"⊄凥ꀙ㳗ꑅ뗪븠檝勡浅","four":[false,true,false]}
{"one":500742813,"two":0.2914132039994096,"three":"띱ꭖ큃囏仉处梍읜欑륣","four":[false]}
{"one":1054542927,"two":0.6602397653258356,"three":"⪼駎í癆緿䋮啠碘沓迋","four":[false,true,true,true,false,false,true,true]}
{"one":-1066964836,"two":0.5438591725674744,"three":"餲∱淌罕擅䚢템㞏⢌瘲","four":[true,false,false,true,true]}

Give it a 30% chance of throwing an exception, rather than returning a result.

% pfarandom input.avsc output.avsc test.pfa --exception 0.3                                                            
% java -jar hadrian-standalone-TRUNK-jar-with-dependencies.jar -i json -o json test.pfa
1
2
3
Exception in thread "Hadrian-engine-0" com.opendatagroup.hadrian.errors.PFAUserException: thrown with probability 0.3 at JSON line:col 1:23 (PFA field "action -> 0 -> then")
        at PFA_Engine_1$2$1$1.apply(Unknown Source)
        at PFA_Engine_1$2$1.apply(Unknown Source)
        at PFA_Engine_1$2.apply(Unknown Source)
        at PFA_Engine_1.action(Unknown Source)
        at PFA_Engine_1.action(Unknown Source)
        at com.opendatagroup.hadrian.standalone.Main$$anonfun$main$1$EngineRunnable$1.action(standalone.scala:196)
        at com.opendatagroup.hadrian.standalone.Main$$anonfun$main$1$EngineRunnable$1$$anonfun$19.apply(standalone.scala:221)
        at com.opendatagroup.hadrian.standalone.Main$$anonfun$main$1$EngineRunnable$1$$anonfun$19.apply(standalone.scala:221)
        at com.opendatagroup.hadrian.standalone.Main$$anonfun$main$1$EngineRunnable$1.run(standalone.scala:232)
        at java.lang.Thread.run(Thread.java:745)