Making Reusable File Transfer Workflows

How does Flux make file transfer workflows easier? A Frequent Question An example of a frequent query (edited here for clarity) received by our technical support staff: “… we have a lot of file transfers that all implement the same logic. So we’d like to design this logic once and then just feed the transfer details as parameters for runtime …

Digital Catalog Job Scheduling

The number and kinds of batch scheduling workflows are diverse. One valuable category of batch job scheduling activity is the scheduling and processing of digital media into on-line catalogs. The nature of digital media varies widely, from audio files to digital photos and movies to PDF-formatted contracts and legal documents. Frequently this media is processed and loaded into a digital …

Categorizing Batch Schedules

There exist many kinds of batch schedules. These schedules may vary in complexity but can still share a high degree of commonality. Simply categorizing your batch schedules affords insight into this commonality and offers opportunity to simplify or find other scheduling opportunities. Some batch schedule categories are provided below as candidate guidance to jumpstart organizing your enterprise’s batch schedules. 1. …

Daylight Saving Time and Job Schedulers

Support for Daylight Saving Time (“DST”) within job schedulers varies widely. Different schedulers handle DST differently. An often-asked question of users of schedulers is “what happens during a DST event?” Flux provides two types of scheduling definitions – cron-style and relative time expressions.

Workload Automation Models

Workload automation sounds simple at its surface. A business consumer (e.g., a department head or a senior executive) may state requirements in “if this, then that” form: If a department budget file arrives, validate it and send to the accounting system for processing. At midnight, backup and index the payroll database. On demand, generate an email to a prospect list …

Candy, Vitamin, or Painkiller?

The start of the new year is a time of reflection – reflecting on past new years’ resolutions and now considering new resolutions. Consider your automation initiatives during this reflection. Were your prior initiatives successful, and if so, what truly was the value they created? Were the results satisfying and long term? What new initiatives do you foresee and is …

Workload Automation and File Processing

There exists many forms of workload automation (WLA), ranging from scheduling recurring jobs such as database tasks, DevOps tasks involving software build and deployments, to virtual machine and container provisioning for software rollouts. Another significant workload automation area is that involving file processing. The job scheduling of file-related activities can vary from very simple to highly complex. Sometimes the scheduled …

Job Scheduling File Naming Conventions

Job scheduling and batch processing often involve the processing of files. The approach taken to naming and organizing your jobs’ files can significantly impact your ability to find those files later and to understand what they contain. The Smithsonian lists five precepts for its file naming and organization to make it obvious where to find specific data and what the …

What to Run Next: Workflow Order of Execution within Flux

Flux provides a powerful set of facilities to control the order of execution of workflows on a Flux engine. When multiple workflows are waiting to be executed on a Flux engine, the engine must determine which workflow should be executed next. To make this decision, the engine considers three primary factors: Eligible Execution Time, Priority, and First-In-First-Out Scheduling. Eligible Execution …

Flux Variables and Variable Managers

Flux’s variables and variable managers provide a powerful vehicle to pass data through a workflow to govern the processing and decision-making that takes place. Using variables can reduce the number of workflows that need to be created since an individual workflow can be made to “flex” depending upon the data it creates and encounters. Variables can be created within the …