Daylight Saving Time and Job Schedulers

Support for Daylight Saving Time (“DST”) within job schedulers varies widely. Different schedulers handle DST differently. An often-asked question of users of schedulers is “what happens during a DST event?” Flux provides two types of scheduling definitions – cron-style and relative time expressions.

Workload Automation Models

Workload automation sounds simple at its surface. A business consumer (e.g., a department head or a senior executive) may state requirements in “if this, then that” form: If a department budget file arrives, validate it and send to the accounting system for processing. At midnight, backup and index the payroll database. On demand, generate an email to a prospect list …

Candy, Vitamin, or Painkiller?

The start of the new year is a time of reflection – reflecting on past new years’ resolutions and now considering new resolutions. Consider your automation initiatives during this reflection. Were your prior initiatives successful, and if so, what truly was the value they created? Were the results satisfying and long term? What new initiatives do you foresee and is …

Workload Automation and File Processing

There exists many forms of workload automation (WLA), ranging from scheduling recurring jobs such as database tasks, DevOps tasks involving software build and deployments, to virtual machine and container provisioning for software rollouts. Another significant workload automation area is that involving file processing. The job scheduling of file-related activities can vary from very simple to highly complex. Sometimes the scheduled …

Job Scheduling File Naming Conventions

Job scheduling and batch processing often involve the processing of files. The approach taken to naming and organizing your jobs’ files can significantly impact your ability to find those files later and to understand what they contain. The Smithsonian lists five precepts for its file naming and organization to make it obvious where to find specific data and what the …

What to Run Next: Workflow Order of Execution within Flux

Flux provides a powerful set of facilities to control the order of execution of workflows on a Flux engine. When multiple workflows are waiting to be executed on a Flux engine, the engine must determine which workflow should be executed next. To make this decision, the engine considers three primary factors: Eligible Execution Time, Priority, and First-In-First-Out Scheduling. Eligible Execution …

Flux Variables and Variable Managers

Flux’s variables and variable managers provide a powerful vehicle to pass data through a workflow to govern the processing and decision-making that takes place. Using variables can reduce the number of workflows that need to be created since an individual workflow can be made to “flex” depending upon the data it creates and encounters. Variables can be created within the …

Flux Relative Time Expressions

Describing time – even in a job scheduler – is not a trivial matter. While Flux does not provide a plutonium reactor and a capacitor, Flux does provide a clean and concise way to describe time. Time expressions are simple, compact, textual representations of time- and date-based schedules. Time expressions are used in several places throughout Flux (including engine configuration …

Time Expressions and Business Calendars

Time expressions are simple, compact, textual representations of time- and date-based schedules. Time expressions are used in several places throughout Flux (including engine configuration options as well as trigger and action properties) to specify when and how often particular tasks will execute on the Flux scheduling engine. Time expressions make it easy to instruct workflows and events to run at …

Jobs-as-Code

Jobs-as-Code denotes a capability of maintaining application workflows, workloads, and jobs as software artifacts that can be managed using existing version control and deployment toolchains. Today most workload automation tools maintain their definitions and executions in proprietary tools and formats that are not amenable to standard development and deployment practice. Some examples of common workflows include file transfers and validation, …