How to maintain and care for your Flux environment
Many organizations run Flux environments in mission-critical systems. The following outlines some basic documents and tasks that help support the overall care and maintenance of a Flux environment.
Documenting a Flux Environment
The following information should be kept in an editable document, reviewed quarterly, and made available to Flux Support Personnel when requested for assisting in Flux-related issues. While this list is not exhaustive, it should serve as a starting point for keeping your Flux environment managed.
- Flux version in use – from the output of running java -jar flux.jar.
- A listing of the Flux key in use (from fluxkey-8.0.txt for example).
- A diagram of the configuration showing the interaction of the Flux engine with other engines, agents, firewalls, proxies, and remote processing nodes.
- Runtime Configuration Properties – which may be in a file or set via code.
- Engine Configuration Properties – which may be in a file or set via code.
- Agent Configuration Properties – which may be in a file or set via code.
- The classpath used to start the Flux engine.
- The JVM Vendor, release, and version of Java. Also whether the Java installed is a JRE or JDK and 32 bit or 64 bit.
- The parameters used to start Flux – either batch files or service property settings. In particular, min and max heap size and the location of Java HOME are required.
- The service account names and privileges allocated to the Flux processes.
- A list and description of Custom Actions and Flux Java Listeners.
- A list and description of 3rd party jar files used in your application and in the classpath to Flux.
- A database export of the Flux_Ready Table and Flux Cluster table when Flux is running on a peak processing day. Note that doing a CSV export often does not properly render large numeric values (such as primary keys) so a database export gives a more complete rendition.
- Machine type Flux is running on:
- Virtual or Physical
- If Virtual – name and version of the virtualization software
- If Physical – name and model of the physical machine
- Memory of the virtual or physical machine.
- Number of processors, and number of cores per processor.
- Virtual or Physical
- Operating System installed and 32 bit or 64 bit.
- Database Vendor, Version, and JDBC Driver in use. A copy of the URL Connection String (excluding any sensitive information).
- Application server and version in use (Weblogic, Websphere, Tomcat, JBoss, none)?
- Change log of specific changes, and the purpose of these changes, made to the Flux environment.
Version Control of Workflows
Flux does not provide a built-in VCS mechanism. Workflows can be downloaded to XML files, from the Repository and the Designer, so if version control is required, Flux recommends the following:
- During workflow design, save workflows to the Repository as normal.
- Once the workflow is in a state where it should be committed to the VCS, use the “Download” option from the Repository to download a local copy of the file.
- If you need to test two different versions of the same workflow, or would like to have a way to tell the different stages of a workflow apart, append a suffix of your choice to the workflow name (e.g., -A, -1, etc.).
- Place the file into the VCS and commit / update as normal through the VCS tool’s interface.
- Once you have your final workflow design, feel free to clean up old versions of the workflow from the workflow repository. This can easily be done from the Flux Monitor, which allows for bulk operations: just select all the workflows you’d like to remove, and click on the ‘Remove’ button found at the top.
Many VCS applications will offer XML comparison / change tracking tools out of the box, while others, like Git, can be extended though custom merge drivers, allowing you to track specific changes within a workflow using standard VCS tools.
Version Control of Flux Environment
In addition to Flux workflows, ensure the following files are under version control so that prior releases can be restored, and differences can be detected and reviewed.
- runtime config
- engine config
- agent config
- batch files using scripts shipped with Flux for the baseline
- custom actions and Java classes
- 3rd party jar files used in your application
Keep your Flux environment clean to make problem determination simpler and more straightforward.
- Keep only one version of the flux.jar and fluxkey-<version>.txt files in the classpath
- Keep the exact same version of Java across environments
- Keep the exact same version of the flux.jar and flux.war files across environments
Coordinating activities on a Flux deployment is essential to provide for effective rollout through different environments such as Dev to QA or QA to production.
- Make SURE the license key is zipped before transferring to the destination computer
- Have a detailed document of the custom environment (if not deployed in the Flux-version folder) including not only info about customized scripts but also noteworthy configs (network file system, etc)
- Be able to update files (flux.jar, etc.) and/or license keys in that environment following the document
- All Flux jars in a production rollout sequence should be the same. Can have different jars when testing new releases but production should never have a newer flux jar than dev or qa
- Make sure that the user starting Flux/Flux services has all the appropriate permissions over the DB, the file system (writing logs, executing scripts, etc), the remote servers it needs to access or execute processes on agents, private keys, etc.
- Adhere to your namespace design guidelines – as mentioned previously in this document.
- Run a weekly job on Flux database tables getting row counts. Review row count growth and shrinkage and ensure changes are reasonable and understood.
- Ensure all servers have the same time across the network (the network time server).
- Construct and maintain documentation on Java actions and the incorporation of third party jars. In particular Jersey and Free marker.
- Keep current on Java Virtual Machine updates.
- Keep current on the JDBC driver jar for your database version.
- Use log4j to allow changing to debug without server restart.
- Ensure that all runtime variables are still in use, and that all passwords are encrypted.
- Ensure runtime variables are separated from runtime configuration properties (in the same file, but using a comment line to keep ’em separated. This makes it a whole lot easier when maintaining runtime config properties).