Using the Logical Graph Editor¶
These are some guidelines on how to use the Logical Graph Editor included in DALiuGE.
General¶
On the left-hand side is the palette where different component types are shown. Users can drag these components and drop them in the central area where a Logical Graph will be build up. When hovering the mouse over a component on the Logical Graph different connection points appear on the borders of the component. By clicking on these and dragging the mouse over to a different component users can draw an arrow between two components signaling a relationship between the two.
By clicking on a component’s text users can also change the label shown by that component on the Logical Graph. This is useful for readability and has no impact on the final output of the Logical Graph.
Also when clicking on a component the editor will show the component’s properties on the bottom-right corner. Users can change here some values associated to the components. Different components support different properties.
Components¶
ShellApp¶
The ShellApp
component represents a bash
shell command.
The command to be run is written using
its different Arg
properties.
For example, to run echo 123
users would have to write echo
in Arg01
and 123
in Arg02
.
When referring to the application’s inputs and outputs
in the command line
%i[X]
and %o[X]
can be used, respectively,
where X
refers to the key
property
of the referenced input/output.
These placeholders will eventually be replaced
with the file path of the corresponding inputs or outputs
when executing the command.
For inputs and outputs that are not filesystem-based
the %iDataURL[X]
and %oDataURL[X]
placeholders
can be used instead.
Note that sometimes an application is connected
to an input or output component
representing more than one physical input or output
(e.g., an application outside a Scatter
component
connected to a Data
component inside the Scatter
).
In these cases the corresponding placeholder will expand
to a semicolon-delimited list of paths or URLs.
It will be the responsibility of the application
to deal with these cases.
Data¶
The Data
component represents a payload.
When transitioning from the Logical Graph to the Physical Graph
these components generate InMemory
drops currently,
but support will be added in the future
to generate drops with different storage mechanisms.
Scatter¶
The Scatter
component represents parallel branches of execution.
Is is represented by a yellow-bordered box
that allows other components to be placed within.
All the contents of the body of a Scatter
will be replicated as parallel branches of execution
when performing the Logical Graph to Physical Graph transition
depending on the value of its num_of_copies
property.
Components outside a Scatter
can be connected to components inside the Scatter
.
In such cases, when he Physical Graph is generated,
the outside component will appear as connected
to the many copies generated from inside the Scatter
.
Examples¶
Simple scatter¶
In this example we will have an application producing many outputs, identical in nature, which are then processed in parallel by a second application. The initial application also outputs a different, single file.
To do this we drop a ShellApp
into the Logical Graph.
Next to it we drop a Scatter
component.
Inside the Scatter
component we drop a Data
component,
and next to the Data
component we finally drop
a second ShellApp
component.
One can then draw an arrow
from the first ShellApp
to the Data
component,
and a second one
from the Data
component to the second ShellApp
.
Finally, drop a Data
component outside the Scatter
and draw an arrow from the first ShellApp
component into it.
Optionally change the names of the components
for readability.
The final result should look like this:
If you save the Logical Graph form above, and then generate the Physical Graph it will look like this:
Here it can be clearly seen
how the Scatter
component’s body
has been replicated according to its configuration.
It also shows how the first application
now produces multiple outputs.