File Orchestration: Building a Business Case
MFT + Workflow = File Orchestration
Navigating the complex web of file transfers in your enterprise is no simple task. Driven by increasingly stringent privacy regulations and the need to quickly transfer and track files globally, your customers, employees, contractors and business partners are generating more and more file transfers, often using a disorganized hodgepodge of techniques. And year over year the volume and size of files is growing.
Building the business case for migrating from legacy file transfer scripts and homegrown applications to a standardized and integrated file transfer platform is not simple. While the value of a standardized approach seems intuitively obvious, quantifying the value in a manner that gets project budget approval requires some research and background.
MFT (“Managed File Transfer”) is a term which emerged in the mid-2000s, as companies began employing new approaches to file transfers which moved beyond FTP (“File Transfer Protocol”) and strictly proprietary protocols.
MFT includes two essential components: management of file transfers, and monitoring of file transfers. Management and monitoring are each performed at a central location, integrated throughout the enterprise. Many organizations today continue to replace their legacy FTP systems with MFT solutions.
MFT itself continues to evolve into what some term as File Orchestration. File Orchestration – refined from definitions used in the service-oriented architecture and business process modeling communities – includes the ability to control operational flows based on business rules; the ability to invoke file transfers and apply automation to file processes, and to chain file calls; to utilize complex workflows, service level agreements, transformations, messaging, alerts, and other attributes of complex enterprise systems; and to assemble file orchestrations themselves from a set of reusable and descriptive patterns describing their behaviors and capabilities.
The business case for managed file transfer and file orchestration derives fundamentally from two components: a) reducing the cost of resolving file transfer and processing issues and b) allowing the number of file transfers and related processes to increase without incurring additional support costs. Other components come into play but these two are generally the largest drivers of the business case.
Numbers are not easy to come by but recent published research by the Aberdeen Group  identified the following:
- the number of end users requiring file transfers increases by up to 9% per year
- 4-5% of file transfers are affected by errors or exceptions
- IT staff spend an average of 4-5 hours per incident correcting those errors
These costs quickly add up. Consider a sample case where 1,000 files are transferred per month. At a cost of ~$50/hr. for IT staff, this amounts to:
|File Transfers||Number of Errors||Hours Resolving Errors||Total Cost|
According to Aberdeen’s research, implementing an MFT solution like that within Flux provides a 26% reduction in overall errors, and allows IT staff to resolve the errors that do occur 4.8 times faster. In other words, with a solution such as Flux in place, the same table would look like:
|FIle Transfers||Number of Errors||Hours Resolving Errors||Total Cost|
For an enterprise running 1,000 file transfers per month, average savings per month as a result of implementing just MFT range from ~$8,446 - ~$10,540.
Boosting the Case
Of course, these figures only account for directly quantifiable costs. Other indirect costs, like savings in workflow creation time, consolidation of disparate systems, improved auditability, and increased management insight into file processes are also beneficial and may result in significant additional savings for your organization based on your individual use case.
Remember also, these calculations were done on a volume of 1,000 file transfers per month – you can easily scale these up to reflect your own environment. It’s pretty easy to see that it doesn’t take very much volume to justify the cost of implementing and supporting a solution like Flux. (In fact you might even save in operational costs, from the benefits of having a more uniform and efficient file orchestration “platform”.)
Moving Beyond MFT to File Orchestration
Reviewing these figures – and adjusting to match your own organization's file transfer requirements, volume, error response thresholds, and SLAs – it's easy to see the gauge the value of an MFT tool within your enterprise. So what makes Flux unique among vendors offering MFT solutions? Why would someone want to go beyond managed file transfer to file orchestration?
The Aberdeen research is limited to the cost/benefit associated with MFT. File orchestration platforms provide additional benefits in that they encapsulate MFT into a robust workflow and scheduling integration platform that integrates business logic, diverse endpoints such as database updates and inserts, email interaction, and web service invocation. These facilities allow file orchestration platforms to serve as an integration hub, coordinating interactions between your business partners and internal departments into with your enterprise systems.
Matching the Solution to the IT Need
Aberdeen identifies 8 fundamental drivers for MFT and workflow adoption. MFT + workflow= file orchestration. These drivers, ranked by percentage of total respondents reporting the associated factor, are:
- 1. Improve Productivity (65%) 2. Security-Related Incidents (e.g., Data Loss) (39%) 3. Collaborate With Trusted Third Parties (37%) 4. Improve Reliability (33%) 5. Industry Standards and Best Practices (32%) 6. Address Audit Deficiencies (25%) 7. Replace Other Delivery Mechanisms (25%) 8. Reduce Total Cost (22%) File orchestration platforms such as Flux make them ideally suited to address each of these factors. (1) Productivity – identified as the most important individual factor in adopting an MFT platform – is the driving force behind file orchestration platforms such as Flux. Most simply, file orchestration improves productivity by reducing (or entirely eliminating) many of the manual interventions your operators must perform. Through the processing of workflows, tasks are launched automatically, on schedule or based on events; operations are grouped logically and efficiently, and distributed where resources are available.
- (1,2,3,4) Centralization in a file orchestration platform also means that your team becomes more productive as tasks that were once spread across multiple systems are managed centrally, organized logically in workflows, and monitored in just one location. As a natural consequence of this process, redundancies become obvious and can be removed; tasks can be aligned to execute efficiently and take advantage of concurrency and parallelism; and once loosely-defined relationships between files, processing steps, and other tasks in the system become evident and highly manageable.
- (5,6) The workflow model also allows the hierarchies of responsibility – for failures in auditing, errors in file processing and workflows, policy violations, and so on – to become naturally apparent. File orchestration’s merging of MFT and workflow defines and organizes the relationships between users, administrators, and operators. When an error occurs, the platform provides clear “ownership” of the error: the file orchestration software itself may attempt to automatically remediate from the error, notify the end user, alert an administrator, or raise a notice to an operator. In many cases, lag in recovery time arises from simple confusion about the responsibility for an error; with file orchestration, this confusion is eliminated, response times are increased dramatically, and key players are always aware of the information they need, when they need it.
- (4) Automatic error remediation is a key component in the workflow model. File orchestration platforms determine where and why an error occurred, and (based on your specifications) allocates resources, performs tasks and updates, and generally attempts to recover from errors immediately when they occur, saving valuable time for your IT staff. At any point, an operator can be alerted to make a manual intervention, or simply keep an eye on the automated remediation process.
- (4,5) Monitoring of workflows and file transfers is available in real-time from a centralized location, with system-wide visibility into every task executing within the platform at any time, and tasks scheduled for future information.
- (5,6) Historic data is audited and available to operators through a web interface, and stored to an enterprise database – a vital component of any MFT or workflow solution, given the degree to which compliance errors drive adoption of MFT platforms. Accessibility and security policies are available through a centralized management interface, protecting your data according to your business rules.
- (7) File orchestration platforms generally provide support for multiple delivery protocols, including FTP, SFTP, FTPS, Unix SCP, and local file transfers. Some file orchestration platforms can be extended to support other protocols such as IBM’s Connect:Direct, Aspera, and even email attachments. This allows such platforms to be used as a replacement for existing delivery systems.
- (8) By reducing the occurrence of file processing related issues, reducing the resolution time of such issues when they occur, facilitating the onboarding of new file transfers and associated workflows, and replacing existing delivery systems into a cohesive platform, file orchestration platforms address the need to reduce overall costs.
Benefits Compounded Over Time
Given the numerous benefits of MFT and workflow outlined here, it's easy to see why it is so quickly becoming the industry standard. And as Aberdeen indicates, statistics back up what these benefits argue: companies who adopt MFT and workflow have fewer errors, fewer security incidents, and fewer calls. For companies who do not adopt MFT and workflows, these become compounding problems (and for those who do – compounding benefits):
|Year-Over-Year Changes In||MFT Users||MFT Non-Users|
|Errors, exceptions, and problems||-3.7%||+1.4%|
|Calls and complaints||-0.4%||+0.2%|
File orchestration platforms such as Flux provide what you need to stop errors and incidents from occurring. If they do, you get the information you need to address them quickly – and make sure they don't happen again. Such platforms not only free up the valuable time your IT staff could spend fixing errors: it allows you to re-invest that time, insuring against future problems, generating more and more stability as time goes on. As many IT managers know, projects tend to fall into three categories: projects that help run the business; projects that help grow the business; and projects that have the potential to transform the business. File orchestration platforms like Flux provide capabilities that allow them to address any of these categories – whether you need Flux as the transformational driver of your technological platform, or as a rock-solid backbone for file transfers across your business, file orchestration platforms such as Flux can fit into your business and budgetary requirements’ and begin saving you money today.
Summary and Key Takeaways
- Volume of file transfers, size of files, and number of end users increase every year – even as privacy regulations become increasing stringent
- Compliance incidents increase by 4% annually, errors by 4-5%, and IT staff spend 4-5 hours per error remediating
- At ~$50/hr and 1,000 file transfers per month, errors in file transfers can cost an organization up to $12,500 per month
- At just 1,000 file transfers per month, an MFT solution, like that embedded within Flux, can save over $8,000 per month against this cost
- Other tangential costs, more difficult to quantify, can magnify these savings
- Savings increase with volume – at 10,000 or 100,000 file transfers a month, savings increase dramatically
- Extending MFT, Flux provides a complete File Orchestration suite
- File orchestration extends MFT with workflow, job scheduling, and automated error handling
- Flux not only coordinates file transfers, but seamlessly integrates into all or nearly all business components
- Aberdeen identifies 8 key drivers for adopting MFT + workflow – Flux succeeds in each
- Productivity improves across-the-board on implementation of file orchestration
- Workflows and tasks naturally improve during the workflow development cycle
- Ownership of errors is clearly defined when adopting file orchestration
- Extensive, central monitoring available through web interface
- Security, high auditability – including historic data stored in enterprise database
- File orchestration benefits are long term – after implementing errors reduce year-over-year; without such a solution errors increase year-over-year
The business case for MFT in today’s interconnected business economy is substantial. File orchestration extends MFT to further extend the value and reduce risk. Maintaining a status quo of legacy file transfer scripts and homegrown applications is no longer necessary as there exist readily available and commercially supported alternatives. It’s simply a matter of building the business case.
Flux’s workflow and scheduling features extend managed file transfer with automated scheduling, workflow dependencies, and automated error recovery and exception handling. As such, Flux offers a platform-agnostic means to integrate file processing and management into an enterprise’s workflow.
Flux assists enterprises in provisioning, onboarding, scheduling, tracking, and reporting an enterprise’s file orchestration processes. These orchestrations vary from simple file transfers to highly complex workflows involving extensive processing, many routes, varied alerts, and complex decisioning. First released in 2000, Flux has grown into a file orchestration and workflow scheduling platform that enterprises rely on daily for their mission critical systems.: http://resources.ipswitchft.com/Aberdeen-Creating-a-Mature-File-Transfer-Process.html
The Flux software platform orchestrates file transfers and batch processing workflows for banking and finance. First released in 2000, Flux has grown into a financial platform that the largest US, UK, and Canadian banks and financial services organizations rely on daily for their mission critical financial systems. Flux provides Electronic Bank Account Management (eBAM) solutions for banks. Electronic bank account management replaces slow paper-based processes with electronic efficiencies, reducing human errors and providing greater transparency into bank and corporate operations.
Banks that offer an eBAM solution possess a critical market advantage in their efforts to expand and retain their corporate customer base.