Call Sales +1 (720) 930-4239 | Email Sales sales@flux.ly — Call Support +1 (720) 441-1844 | Email Support support@flux.ly

Job Scheduling and File Transfer

Batch Job Scheduling and Managed File Transfers

batch job scheduling and managed file transfers

Transferring files using a batch job scheduler with fine-grained control and configuration at runtime is a key and often used Flux feature. The following describes one such Flux workflow developed for a large mortgage broker in their transfer and tracking of mortgage files. The same model is in use in insurance, invoicing, logistics, and many other industries needing flexible and configurable file transfers.

This workflow is reusable for multiple customers and configurable at runtime. Submitting the workflow to a Flux engine as the name “/Acme/weekly” will force the engine to configure the workflow based on a runtime configuration specific to weekly runs defined for the Acme customer. The same workflow template can be submitted repeatedly with different names to the Flux engine for various customers and time periods (e.g., customers Acme, Baker, and weekly, daily, and monthly).

The workflow template diagram is depicted below. It can be reused without modification (relying on runtime configuration and not workflow modification) for many SFTP file exchange partners.

example flux managed file transfer workflow
Example Managed File Transfer Workflow

Each customer (or file exchange partner) is configured with a set of runtime parameters – loaded at the time the workflow starts for that customer. The runtime configuration is refreshed automatically when the configuration file is changed (note that the frequency of this refresh is configurable). These parameters include when to trigger the workflow, where to place downloaded files, and what files to download. Adding a new customer or exchange partner involves adding the required entries to this file through the Flux user interface, and then submitting the workflow template to the Flux engine for that customer.

/Acme/daily/StartRunTime=+1d
/Acme/weekly/StartRunTime=+1w
/Acme/monthly/StartRunTime=+1m
/Baker/daily/StartRunTime=+1d
/Baker/weekly/StartRunTime=+1w
/Baker/monthly/StartRunTime=+1m

# Locations for specific downloads
/Acme/output=c:/MFT/Acme
/Acme/daily/output=c:/MFT/Acme/Daily
/Acme/weekly/output=c:/MFT/Acme/Weekly
/Acme/monthly/output=c:/MFT/Acme/monthly
/Baker/output=c:/MFT/Baker
/Baker/daily/output=c:/MFT/Baker/Daily
/Baker/weekly/output=c:/MFT/Baker/Weekly
/Baker/monthly/output=c:/MFT/Baker/monthly

# Server and Password user names
/SFTPServer=test.flux.ly
/SFTPUser=testfluxly
/SFTPPassword=****
/Acme/SFTPServer=acme.com
/Acme/SFTPUser=acmeUser
/Baker/SFTPServer=baker.com
/Baker/SFTPUser=bakerUser

# File include patterns
/includepattern=./public/*.*
/Acme/includepattern=./public/*.*
/Baker/includepattern=./public/icons/*.*

A workflow template for generating reports is also provided, as depicted below:

flux mft generate report example

This workflow template also has a set of runtime configuration parameters that allow the template to be dynamically configured at runtime to execute selected reports over particular time ranges. These reports were built using Jasper Report Community Edition. This report generator workflow can be tailored to call any report writer to create reports specific the the customer’s needs. No report server installation is required.

# — Reporting Component Parameters
/RptTrigger=+60m
/RptDbType=generic
/RptServerName=10.0.1.3
/RptDbName=flux80
/RptDbURL=jdbc:sqlserver://10.0.1.3:1433;databasename=Flux80
/RptUser=Flux80
/RptPassword=flux
/RptDriver=com.microsoft.sqlserver.jdbc.SQLServerDriver
/RptOutputDir=c:/flux80maint/opsconsole/src/main/webapp/reports
/Reports/DAYS=1
/Reports/Daily/DAYS=1
/Reports/Weekly/DAYS=7
/Reports/Monthly/Days=30/Reports/Audit/RptPattern=jasperstarter/examples/*Audit*.jasper
/Reports/Daily/Audit/RptPattern=jasperstarter/examples/*Audit*.jasper
/Reports/Weekly/Audit/RptPattern=jasperstarter/examples/*Audit*.jasper
/Reports/Monthly/Audit/RptPattern=jasperstarter/examples/*Audit*.jasper
/Reports/XFER/RptPattern=jasperstarter/examples/XFER*.jasper
/Reports/Daily/XFER/RptPattern
=jasperstarter/examples/XFER*.jasper
/Reports/Weekly/XFER/RptPattern=jasperstarter/examples/XFER*.jasper
/Reports/Monthly/XFER/RptPattern=jasperstarter/examples/XFER*.jasper

The following graphic illustrates one of the generated reports for this MFT example.

Since reports are generated themselves via a Flux workflow, these reports can be emailed, file transferred, and made visible within a web browser via hyperlinks from the Flux Cockpit user interface.

flux jasper report summary

Finally, a common requirement is allowing external customers to view the status of their exchanges and workflows. Utilizing Flux’s security model, customers can be configured to only view selected workflows and their execution based on their names. So any number of users can be configured to see only those items they are permitted to view. For example, here is the Flux monitor configured for an admin role presented above a user’s view with restricted access to “Baker” workflows.

Depicted below is the Flux Operations Console for a user configured with admin role with unrestricted access.

flux operations console admin user

And as a user with restricted access to only Baker workflows.

flux operations console restricted user

Flexibility and reusability are core to Flux workflows. The templates and configuration file and report definition discussed here are available as an available example in the Flux documentation website for download and use by Flux customers and evaluators.

Related posts

Getting to Done Quickly

Data collection made easy for global healthcare provider using Flux With Growth Comes Added...

Can we help? Tell us.

Share with us the contact information for a friend or fellow developer (or even...

Flux or Quartz

Looking for a Java job scheduler? After some searching and introspection, perhaps you arrive...

The Origins of Flux

Sims Computing created Flux, an enterprise component, using Java Technology Sims Computing was founded...
Top