I’m Late, I’m Late. Handling the Late Scheduling of Jobs
A scheduled batch job might not run on time (i.e., be “late”) for many reasons - for example:
- the scheduler might be down for an extended period of time, during which one or more jobs would normally have run.
- there is not enough concurrency available to process jobs in a timely manner, so a queued job may not be reached for scheduling until after it’s time to execute has passed.
- A required resource for your job may not be available so your job must wait. This is common when your job performs file transfers from a remote service and the remote service is down.
Such circumstances can often be handled manually by pausing or restarting jobs, but this can quickly become overwhelming, especially when many jobs are involved.
To handle this, jobs need a defined way of telling the scheduler what the absolute latest time the job can run without being skipped. This is referred to as a “late time window.”
For example, a late time window of "+5m" (five minutes) means that the job can fire up to five minutes past its scheduled firing time. A late time window of "+1d" would allow the job to fire up to one day late.
If the job attempts to run but the late time window has elapsed, the scheduler will simply determine the next scheduled date, skip the late job, and continue running as normal (unless makeup firing is enabled – see further down in this post).
|Scheduled Date/Time||Late Time Window||Date on Clock Now||Time Expression for Next Run||Result|
|April 30, 2020 08:00:00||+6H||April 30, 2020 10:30:00||+6H||Actual date is within the late time window, so the job fires. Only one firing was missed so the job continues as normal, scheduling the next firing for 14:00:00.|
|April 30, 2020 08:00:00||+5m||April 30, 2020 08:06:00||+6H||Actual date is not within the late time window, the job does not fire. The scheduler simply schedules the next execution at 14:00:00 as though no firing was missed.|
|April 30, 2020 08:00:00||+30m||April 30, 2020 08:25:00||+10m||Actual date is within the window, so the job fires. Two additional firings were missed (08:10:00 and 08:20:00); the job fires again for each of those before setting the next date (April 30, 2020 08:30:00).|
|April 30, 2020 08:00:00||+1H||April 30, 2020 10:30:00||+1H||Actual date is not within the late time window, the job does not fire (even though one of the missed firings was within the window). The next firing date (April 30, 2020 11:00:00) is scheduled.|
Makeup Firing Enabled
This property is only evaluated if the late time window has elapsed but there are one or more missed firings.
If makeup firing is enabled, at least one firing was missed, but that firing did not fall within the late window, then Flux will fire a single makeup firing before advancing to the next scheduled trigger date.
This is useful in cases where several firings were missed and did not fall within the late time window, but you still want to fire once to compensate for the missed firings.
Examples - Assume the scheduler was unavailable from April 30, 2020 07:59:00 to April 30, 2020 15:00:00
|Scheduled Date/Time||Late Time Window||Date on Clock Now||Time Expression for Next Run||Enable Makeup Firing||Result|
|April 30, 2020 08:00:00||Not specified||April 30, 2020 15:00:00||+6H||Not specified (default is false)||The trigger fires at April 30, 2020 15:00:00. The next date (April 30, 2020 20:00:00) is scheduled.|
|April 30, 2020 08:00:00||+6H||April 30, 2020 15:00:00||+6H||true||Late time window has elapsed but makeup firing is enabled, so the trigger fires at April 30, 2020 15:00:00. The next date (April 30, 2020 20:00:00) is scheduled.|
|April 30, 2020 08:00:00||+6H||April 30, 2020 15:00:00||+6H||false||Late time window elapsed and makeup firing is disabled, so the trigger does not fire. The next date (April 30, 2020 20:00:00) is scheduled.|
|April 30, 2020 08:00:00||+1H||April 30, 2020 10:32:00||+5m||true||Late time window has elapsed but makeup firing is enabled, so the the trigger fires. The next date (April 30, 2020 10:35:00) is scheduled.|