Making Flux workflow’s reusable

One of the most popular features of Flux is the ability to make workflows reusable by using namespaces and runtime variables.

The Flux engine, at the time a workflow is submitted to an engine namespace, assigns values to workflows designed with runtime variables. Workflow runtime variables receive the values of namespace variables, making the workflows ‘dynamic’ and configurable at runtime.

Runtime variables allow your workflow designers to separate the specific values a workflow requires from the workflow itself. In this manner you can separate variables such as server names or file locations from the workflow itself. It is common for Flux users to have different runtime configuration files for different environments, and place these under version control. For instance, production, QA, and developer instances of Flux may all utilize the same workflows, but with different runtime configuration files.

Flux namespaces, as most namespaces, are structured in a hierarchy to allow reuse of workflows and variables in different contexts. Runtime variable substitution allows one to assign values to workflows based on the workflow’s namespace. Values assigned to namespaces are inherited by namespaces lower in the hierarchy unless they are overridden at a lower level.

One application of this capability is in the performance of periodic database maintenance. Database maintenance scripts are often run during the maintenance window of the application. Say for instance on Sunday at 2 am. Namespace values would include the priority of jobs, specific SQL scripts to run, notifications to send out on completion and error situations, and error handling flows. A Flux database maintenance script for a MySQL database would look as follows:

  • /prod/sql/concurrency_throttle=1

    /prod/sql/flux/reindex-script=mysqlcheck -u root -p password –check –databases flux





This script could then be triggered within a Flux workflow, such as this generic SQL execution workflow:

Flux generic SQL script

Often workflows have dependencies with other workflows completing successfully. Declaring these dependencies explicitly as namespace values simplifies maintenance and improves transparency.

Workflow processes that are homogeneous in nature and only differ by time run or a set of parameters are best managed via namespaces.

Finally, namespace value assignment itself can be managed dynamically. Placing runtime values in a database allows one to collect updates to the runtime configuration. Defining a workflow that periodically applies these namespace changes to the runtime configuration file provides flexibility and increased control over the workflow environment. 

You can read about workflows, namespaces, runtime variable substitution and lots more in the Flux documentation.