-
Notifications
You must be signed in to change notification settings - Fork 1
Tutorial 3. Extended Example
In this tutorial we will begin to see some of the advantages of programming with data flows. We will do this by incrementally extending in various ways the Hello World example from Tutorial 2. Hello World.
You have probably already tried modifying the hello1.yaml workflow file to output a different greeting. Indeed, this can be done easily by modifying the greeting given to the value constant of the CreateGreeting node. For example, to introduce a Shakespearean theme modify the sixth line of the CreateGreeting node as shown and save the file as hello2.yaml:
imports:
- classpath:/common/groovy/actors.yaml
- classpath:/common/directors.yaml
components:
- id: HelloWorld
type: Workflow
properties:
director: !ref MTDataDrivenDirector
nodes:
- !ref CreateGreeting
- !ref RenderGreeting
- id: CreateGreeting
type: Node
properties:
actor: !ref ConstantSource
sequences:
value:
- Hello World
- Good Afternoon, Cosmos
- Good night, and good luck
outflows:
value: /messages/greeting/
- id: RenderGreeting
type: Node
properties:
actor: !ref PrintStreamWriter
inflows:
message: /messages/greeting/
Now run the workflow as before, and you should see something like the following:
$ restflow -f hello2.yaml
Good night, sweet prince.
$
Perhaps more interesting, you can provide CreateGreeting with a list of greetings to send. Replace the constants property of the CreateGreeting node with a sequences property, and assign it a list of greetings as shown below. Save the new workflow as hello3.yaml. (Only the CreateGreeting component is shown here. The rest of the workflow definition is still required).
- id: CreateGreeting
type: Node
properties:
actor: !ref ConstantSource
sequences:
value:
- Hello World
- Good Afternoon, Cosmos
- Good night, and good luck
outflows:
value: /messages/greeting/
Running this workflow will then yield the following:
$ restflow -f hello3.yaml
Hello World
Good Afternoon, Cosmos
Good night, and good luck
$
What happened? The CreateGreeting component stepped three times, once for each item in the list of greetings provided to it via the value sequence. The RenderMessage node received each greeting from CreateGreeting and printed the messages to the terminal. Consequently, RenderMessage, stepped three times as well.
This illustrates a key point. RestFlow workflows and components generally are data driven. They operate on all data provided to them--through constants, sequences, or inflows--and keep working until they have no more data to process.
So far we have only printed some constant strings to the terminal. Now we will add some basic computation. Add the following new component somewhere within the components section of the workflow specification file. The order of the components in the file does not matter.
- id: EmphasizeGreeting
type: Node
properties:
actor: !ref StringConcatenator
inflows:
stringOne: /messages/greeting/
constants:
stringTwo: '!!'
outflows:
concatenatedString: /messages/emphasizedGreeting/
The function of this new node generally is to take two strings, concatenate them, and output the result. In this case the inflows and constants properties configure it to take the greetings provided by CreateGreeting and append two exclamation points to each.
Now change the flow expression for the message inflow on RenderGreeting to /messages/emphasizedGreeting/. This configures RenderGreeting to receive the outputs of the EmphasizeGreeting node, rather than those of CreateGreeting:
- id: RenderGreeting
type: Node
properties:
actor: !ref PrintStreamWriter
inflows:
message: /messages/emphasizedGreeting/
Finally, add a reference to the new EmphasizeGreeting component to the list of nodes given to the HelloWorld component. This tells RestFlow that EmphasizeGreeting is one of the nodes that must be triggered during a run of the workflow. Note that here, too, the order of components in the list does not matter:
- id: HelloWorld
type: Workflow
properties:
director: !ref MTDataDrivenDirector
nodes:
- !ref CreateGreeting
- !ref RenderGreeting
- !ref EmphasizeGreeting
Save the modified workflow as hello4.yaml and then run it. You should see the following output:
$ restflow -f hello4.yaml
Hello World!!
Good Afternoon, Cosmos!!
Good night, and good luck!!
$
The Hello World workflow is now a functioning three-stage software pipeline.
You may have noticed that the names of the inflow and constant for EmphasizeGreeting are stringOne and stringTwo, respectively. The function of EmphasizeGreeting is to concatenate stringOne and StringTwo to produce concatentatedString (the name of the outflow of EmphasizeGreeting). Must stringOne be provided a dynamic value, sent by an upstream workflow node as it is here? And does stringTwo need to be provided a static value via a parameter? The answer to both questions is no. Any information used by a workflow node can in fact come from one of three different places depending on how it is configured.
First, it can be provided via a static value, as stringTwo is here. By static, we mean that the value(s) to be provided are fixed before the workflow is run. To provide a static value for stringOne, you only need to delete the stringOne inflow and create a stringOne constant analogous to the stringTwo constant. (But then, of course, the node would not process any information from CreateGreeting!) Second, data can be provided to a node via an inflow. Finally, the node may use a default value assigned in the definition of the actor underlying the node. We will look at actors and how they are defined in Chapter 5.
When you declare more than one inflow for a node, the data flowing to the two inflows may come from the same or different upstream workflow nodes. To see this in practice, consider this updated version of Hello World, hello5.yaml:
imports:
- actors:actors.yaml
- classpath:/common/directors.yaml
components:
- id: HelloWorld
type: Workflow
properties:
director: !ref MTDataDrivenDirector
nodes:
- !ref CreateGreeting
- !ref EmphasizeGreeting
- !ref RenderGreeting
- !ref CreateEmphasis
- id: CreateGreeting
type: Node
properties:
actor: !ref ConstantSource
sequences:
value:
- Hello World
- Good Afternoon, Cosmos
- Good night, and good luck
outflows:
value: /messages/greeting/
- id: CreateEmphasis
type: Node
properties:
actor: !ref ConstantSource
sequences:
value:
- "!"
- "!!"
- "!!!"
outflows:
value: /messages/emphasis/
- id: EmphasizeGreeting
type: Node
properties:
actor: !ref StringConcatenator
inflows:
stringOne: /messages/greeting/
stringTwo: /messages/emphasis/
outflows:
concatenatedString: /messages/emphasizedGreeting/
- id: RenderGreeting
type: Node
properties:
actor: !ref PrintStreamWriter
inflows:
message: /messages/emphasizedGreeting/
We have added a new component, CreateEmphasis, that produces three strings, "!", "!!", and "!!!", and sends them to the new stringTwo inflow of the EmphasizeGreeting node. Running the updated workflow gives these results:
$ restflow -f hello5.yaml
Hello World!
Good Afternoon, Cosmos!!
Good night, and good luck!!!
$
When workflows become more complex than this it is often useful to see what data each node produces during a run and in what order. Tutorial 4. Workflow Traces will show you how to do this.