One of the least productive uses of staff time is manually determining when to run particular tasks. Sometimes tasks are simply time based, such as running reports at the end of the month or end of week. But often the requirements are a bit more complex. In such instances the utility of some kind of scheduler becomes obvious.
If your organization already has in place an enterprise scheduler, then you are indeed fortunate. Operations at the data center such as virtual machine deployments and allocations, application rollouts, and data center migrations are all ideally suited to enterprise schedulers.
Yet while enterprise schedulers are very capable they tend to be expensive. Many organizations have difficulty justifying the expense, training, deployment issues, and required infrastructure changes in order to accommodate enterprise schedulers.
But just because your organization may not have an enterprise scheduler does not mean you need to go wanting, particularly if your requirements are less data-center centric and more application or job focused.
Evaluating schedulers for organization need
No single scheduler, enterprise or not, fits all needs. You may need a more targeted job or application scheduler. Such schedulers provide a balance of lightweight memory, CPU, and installation footprint, simple installation, customizability, and a focused feature set to provide control and monitoring of your application’s jobs. Or you may need a scheduler that can be embedded into your application, or one that can be extended by your development staff to meet specific requirements of your application.
Flux is one such scheduler. Flux provides a large percentage of the functionality of enterprise schedulers at a much lower price point. Flux addresses requirements that are less data-center operations centric and more application or job focused. Schedulers such as Flux provide the ability to:
- construct and store schedulers and dependency flows
- schedule, pause, expedite, restart, and cancel jobs
- schedule jobs to run once
- schedule jobs to run a fixed or configurable number of times
- schedule recurring jobs that repeat indefinitely or until some ending date
- schedule jobs that are sensitive to local holidays, such as banking holidays or stock exchange trading calendars
- programmatically restart and recover jobs in error circumstances
There exists wide diversity in the features and capabilities of schedulers. Basic schedulers – such as those provided using Windows Task Scheduler or Unix Cron, only handle time-based triggering of work. Schedulers such as Flux provide support for more complex triggers dependent upon time, database conditions, the presence or absence of files, incoming mail, incoming web requests, the completion of other work, and more. Additional features such as distributed processing, concurrency management, and job pinning also appear.
Schedulers may also be customizable and extensible using scripts and code. Flux, being 100% Java, is fully extensible using Java. Other schedulers have other language support.
Schedulers may come with additional or hidden costs. Many schedulers require an additional investment in a relational database such as Oracle or SQL Server. Schedulers that support web browsers may require some form of application server.
Flux, in its installation package, contains an embedded database and application server, thus minimizing the cost of deployment. Flux even supports in-memory databases for even lighter-weight deployments. Flux agents do not require a database or application server at all. Flux also supports most major relational database vendors and Java application servers if these are already available in the organization.
Scheduler deployment requirements
Different schedulers have different deployment sweet spots. Deployment and application rollout requirements can impact scheduler selection. Sample deployment requirements to consider are:
- Need to deploy a scheduler engine and a small number of schedules on dozens, or thousands, of machines. One top ten US bank for example uses Flux to perform client statement delivery to, and data collection from, thousands of its customers world-wide. Millions of statements per month are processed.
- Need to embed the scheduler into an application that requires extensions and customizations. A number of financial service providers embed Flux in their payment processing applications for commercial sale to the banking industry. These providers embedded their own custom validation, settlement, and reporting business logic within a suite of Flux schedules to create a set of commercial products. Over 500 billion USD dollars a month in payments are processed through such Flux-embedded applications.
- Need to schedule the downloading, collection, validation, and forwarding of files from customers to translation, media transcoding, data analytics, and payment processing back ends. A number of SaaS providers and digital media providers do this today using Flux. Some of the largest media companies in the world use Flux for such applications.
- Need to schedule the processing, consolidation, and distribution of corporate insurance benefit statements from a variety of backend systems to thousands of corporate HR departments.
Requirements such as those illustrated above are ideally suited to the targeted and focused capabilities of an application scheduler like Flux. Many of the above requirements are satisfied by a small number of schedules that are executed sometimes thousands of times a day to support the mission-critical processing requirements of the organization.