Flux – File Orchestration Platform

Managing Flux Workflows Using Scripts

Oftentimes, users interact with and control Flux using its web-based Operations Console. However, not everyone knows that you can also control Flux using command line scripts.

These scripts allow your Flux operations to become automated, providing you with greater control and efficiency within your operations teams. They also allow your operations teams to interact with Flux in ways that aren't possible or practical when using the Operations Console.

For example, these Flux scripts can be used to:

  1. Purge or restart a large number of failed workflows from the command line, something that could be cumbersome when performed through Flux's user interface.
  2. Upload and download workflows to the Flux repository, a scripting task that can be incredibly useful when automating processes.
  3. Bootstrap workflows directly to your Flux cluster, making your implementation and deployment practices simpler, faster, less error prone, and more transparent to your IT team.

Flux provides some scripts in a publicly available Git repository:

https://github.com/aruld/flux-scripts

Upload Workflow Files to Flux Repository

The Upload script (all scripts mentioned here are available from the Git repository listed above) uploads a workflow template to your Flux repository. The workflow can be uploaded to a custom namespace (a hierarchical organization of workflows) by passing in a second parameter. By default, the workflow uploads to the namespace defined in the workflow template file.

Here is the command line:

upload-workflow.sh "payment workflow template.ffc" "/payment workflow v1"

Here is sample output when uploading a workflow template to the “/payment template v1” namespace in the Flux repository.

$ upload-workflow.sh "payment workflow template.ffc" "/payment workflow v1"
Uploading workflow payment template.ffc to namespace /payment workflow v1...
Success! Done uploading payment workflow template.ffc!

Download Workflow Files from Flux Repository

The Download script downloads a workflow template from your Flux repository and saves it to your computer.

Here is the command line:

download-workflow.sh "/payment workflow template"

Here is sample output when downloading a workflow template.

$ download-workflow.sh "/payment workflow template"
Downloading workflow "/payment workflow template" from repository...
Done downloading /payment workflow template!

Deploy Workflow to Flux

The Deploy Workflow script exports a new workflow instance from a file into Flux. The script carefully removes and replaces any existing workflow instance at the same namespace location.

Here is the command line:

deploy-workflow.sh "/ingest/video" ingest-video.ffc

Here is sample output when deploying a workflow template from a file.

$ deploy-workflow.sh "/ingest/video" ingest-video.ffc
Exporting ingest-video.ffc...
One workflow exported. Its name is: /ingest/video

Remove Workflows from Flux

The Kill script carefully removes workflow instances from Flux. During integration testing, users may run into scenarios where they have large number of failed workflows due to external conditions such as network outages or third-party web service unavailability. This script removes multiple workflows from Flux in one step.

kill-workflow.sh /allpayments/

Here is sample output when removing multiple workflows in one step.

$ kill-workflow.sh /allpayments/
Removing /allpayments/...
Removed 500 workflows.

Start Workflow from Repository

The Start Workflow script starts a new workflow instance from the repository. The script carefully removes and replaces any existing workflow instance at the same namespace location.

Here is the command line:

start-workflow.sh /ingest/video

Here is sample output when starting a workflow template from the repository:

$ start-workflow.sh /ingest/video
Starting /ingest/video...
One workflow started. Its name is: /ingest/video
Contact Us!
  • *
  • *
  • *
  • *