The Enterprise Scheduler

The IT Manager’s Quest for Sleep

This is not a story of vagaries and euphemisms. Unfortunately, this story is also often-told. This is a story starting with anguish and suffering. It may end with a happy ending, it may not. But it’s all about you – The IT Manager.

Once Upon a Time

You know the story. For all we know, you may be in the midst of this story. As a development manager or IT manager, you get the late night call. Job “XYZ” is running too long, and impacting multiple jobs. In a bank the dreaded words are “we cannot post.” Other enterprises have different phrases or different events (e.g., “we cannot close the month”) but in each case these words elicit a sense of dread. You’re not going to get any sleep, you’re going to have to rouse a legion of people from a sound sleep, and you’re looking forward to a litany of “root cause” meetings and incident reports in the coming days. What happened? You just potentially experienced:

  • Unexpected downtime, possibly caused by human error (i.e., one of your staff)
  • Your operations staff complaining they have no control over production processing, making recovery difficult
  • You just got a Monday morning “surprise” due to weekend errors
  • No one was notified about delays and errors in a mission-critical job stream

Off with their Head!

And now, for whatever reason, it’s deemed your fault! Even if it’s stated in more euphemistic terms, This is not a story of vagaries and euphemisms. Unfortunately, this story is also often-told. This is a story starting with anguish and suffering. It may end with a happy ending, it may not. But it’s all about you – The IT Manager.

If the event is severe enough (one customer refers to their event as “The Saint Valentine’s Day Massacre”, a task force or “consultant” is summoned in search of a solution. They are tasked with, for lack of other wording, to find you a better way to do your job. They do their research and conclude, from many sources, that job automation (or the keyword du jour e.g., IT process automation, workloadautomation, even self-healing systems) has shown significant benefit in enterprise activities such as:

  • Backup and restoration
  • Disaster recovery/failover
  • Service request/fulfillment
  • Incident management/event response
  • Data movement

In addition, they cite a litany of opportunities for savings. Their research shows opportunities for:

  • Savings due to your potentially using fewer or less costly personnel
  • Savings realized by your reducing or eliminating manual, unnecessary or repeating processes
  • Savings enjoyed by your no longer missing SLAs, and reduced customer dissatisfaction
  • Savings on your meeting regulatory and other compliance issues
  • Savings through your purchasing fewer or smaller quantities of software and licenses
  • Savings on annual software maintenance costs
  • Savings on homegrown software development, testing, and maintenance
  • Savings through your use of fewer hardware, virtual, database or OS platforms

Here Ye, Here Ye

The business requirements get cited. A proclamation is made. You are directed to find a way to:

  • Simplify the way you create and control batch and on-line processes
  • Align views of IT processes with the business
  • Improve your customer response times and IT resource utilization
  • Reduce your resolution times
  • Get more out of your existing assets
  • Schedule jobs to execute in a consistent and repeatable manner
  • Reduce the number of manual intervention and errors
  • Lower and standardize your IT staff maintenance tasks
  • Increase your responsiveness to business demands
  • Better utilize available systems and the network
  • Decrease risk
  • Rapidly update jobs

And so the edict is issued. Senior management demands that you procure a job scheduler. But not a simple job scheduler, an “enterprise” job scheduler! You convene a team to interview internal and external customers of the enterprise on a list of requirements. Requirements like – well like the following:

BENEFIT TO THE ENTERPRISE ACHIEVED HOW?
Simplify the way you create and control batch and on-line processes Through a graphical user interface and easy-to-use scripting
Align views of IT processes with the business By correlating IT job names to business-relevant names and categories
Improve your customer response times and IT resource utilization With real-time, event-driven automation
Reduce your resolution times With proactive intervention and remediation of problems
Get more out of your existing assets Through seamless integration of applications and systems
Schedule jobs in a consistent and repeatable manner Across multiple operating systems, packaged and custom apps
Reduce the number of manual intervention and errors Via automated response to exceptional events
Lower and standardize your IT staff maintenance tasks Allowing your staff to work on other, more important projects
Increase your responsiveness to business demands By synchronizing job executions with external application or system events
Enhance systems and network performance To better meet service level goals
Decrease risk Through improved security and resiliency
Rapidly update jobs By allowing changes to be done quickly

And the Quest Begins

And your quest for a tool or technology begins. Taking the above requirements, you begin to define the attributes of the technology you are looking for. For instance:

BENEFIT TO THE
ENTERPRISE
ACHIEVED HOW? USING
Simplify the way you create and control batch and on-line process Through a job management and monitoring user interface and graphical workflow design / easy-to-use scripting Fat client
Browser Client
Zero-configuration
Graphical
API
REST
Scripting
Workflow definition without writing code
Align views of IT processes with the business By correlating IT job names to business-relevant names and categories Naming convention
Category or Description
Namespaces
Hierarchical structuring and viewing of processes
Improve your customer response times and IT resource utilization With real-time, event driven automation Automation support for conditional branching, dependencies, looping, parallelism, splits, and joins
Event support for cron expressions, database conditions, message queues, web service requests, workflow conditions, and file existence and state
Reduce your resolution times With proactive intervention and remediation of problems Email notification
Automatic web service calls to problem ticket systems
Job event and subscription notifications Audit trail
Log messages
Easily understood and user-definable messages
Automated recovery processes
Get more out of your existing assets Through seamless integration of applications and systems Agentless deployment
Agent deployment
Multi-platform support
Clustering support
Schedule jobs in a consistent and repeatable manner Across multiple operating systems, packaged and custom apps Relative and absolute time-based scheduling
Scheduling precision at least to the minute
Multiple business calendar support
Dependency management between workflows, time, and external events (e.g. incoming web service calls)
Reduce the number of manual interventions and errors Via automated response to exceptional events Automated error handling
Default and workflow specific handling
Proactive notification of SLA pending and exceeded events
Lower and standardize your IT staff maintenance tasks Allowing your staff to work on other, more important projects Standardized workflows that provide for dynamic configuration from files, database tables, or web service calls
Increase your responsiveness to business demands By synchronizing job executions with external application or system events Load balancing
Resource dependency management
Enhance systems and network performance To better meet service level goals Distributed processing across nodes – even within one workflow
Pinning particular jobs to specific nodes when required
Dedicating resources to specific jobs for unique requirements – such as leveraging IT appliances.
Decrease risk Through improved security and resiliency Leveraging distributed resources for increased reliability and scalability
Granular access controls with integrated authentication with Active Directory and LDAP
Automated failover and recovery
Warm and cold recovery modes
Rapidly update jobs By allowing changes to be done quickly Real time schedule changes, support for ad hoc submissions, and dynamic job reconfiguration

Choose Wisely

The good news is there exists a variety of products that provide significant coverage for these requirements. You are not going where no one has gone before. An obvious place to start is Wikipedia’s List of Job Schedulers. But be aware, with great power comes often a great (as in expensive) price. So balance your wish list with your “I must absolutely have in order to survive” list.

Many vendors have free evaluation software, as well as demonstration sites to view online. But during the selection process, be prepared for the following due-diligence process:

  • Prepare an RFP, or at least a clear list of requirements. Steal from above if you like.
  • Determine exactly how you will finally qualify a vendor for inclusion in your short list.
  • Get a demonstration of the system from the vendor.
  • Look at the training outline for vendor’s product.
  • Define and share the Proof of Concept requirements with your vendor.
  • Schedule and execute the POC.
  • Evaluate the results of the POC.
  • Document the lessons learned and best practices.
  • Make your selection.
  • Distill all lessons and practices into a single rollout and deployment process (or process set).
  • Pilot your selection on some high value moderate risk workflows using the defined process.
  • Evaluate the lessons learned and best practices discovered.
  • Refine the rollout and deployment process.
  • Rollout the implementation.
  • Return to a good night’s sleep.

Happily Ever After?

Declare a win and take solace that your efforts will indeed make you show the world you can do your job better. Your return to getting a good night’s sleep may not necessarily be long-lived for (let’s be realistic) you are an IT Manager!

Image

About Flux

The Flux software platform orchestrates file transfers and batch processing workflows for banking and finance. First released in 2000, Flux has grown into a financial platform that the largest US, UK, and Canadian banks and financial services organizations rely on daily for their mission critical financial systems. Flux provides Electronic Bank Account Management (eBAM) solutions for banks. Electronic bank account management replaces slow paper-based processes with electronic efficiencies, reducing human errors and providing greater transparency into bank and corporate operations.

Banks that offer an eBAM solution possess a critical market advantage in their efforts to expand and retain their corporate customer base.

Download Flux