BMC Software: Why developers are missing a trick by ignoring workflow automation

By Joe Goldberg
In DevOps workflows, automation saves time, conserves resources, reduces errors and ensures consistency. Despite this, developers continue to ignore the...

In DevOps workflows, automation saves time, conserves resources, reduces errors and ensures consistency. Despite this, developers continue to ignore the existence of world-class workflow automation and job scheduling solutions and instead rely on a variety of simple tools to code jobs as they build apps. Since neither these tools nor the developers adhere to the same standards, the applications passed to Operations often fail to meet production standards. And when something breaks, the problem is often hard to find and fix.

Since such large portions of business application processing is performed by job scheduling functions, automating their creation will not only save time during development, testing, and deployment, but also enable developers to focus more time on building the best business logic.

Consider the makeup of an application. Code, such as Java and Python implements the business logic. The application runs on infrastructure, such as a Linux server, web application server, or network. Databases and SQL statements create tables and rebuild indexes. Finally, many dependent jobs and tasks have to execute, often in a particular order. This job scheduling simply determines when jobs run and what to do if a job fails or can’t start when scheduled. When developers spend such a significant amount of their development time defining this essentially administrative functionality, they are, in a very real sense, wasting their time only to create future headaches for Operations that waste even more time.

“Jobs-as-Code” is a way of standardizing and automating job scheduling by embedding code using a simple notation that makes API calls to a scheduling engine, then managing these calls throughout the continuous delivery pipeline, the same way Java or Python code is managed. As such, Jobs-as-Code is fundamentally no different than other code management practices.

Here is how Jobs-as-Code can fit into a streamlined development process:

Don’t think of jobs as business logic

Job scheduling is mere instrumentation. Scripting it requires a significant effort for the build, test and support processes – and errors and inconsistency can carry over into Operations. Instead, a job scheduler that can manage flow relationships, success/failure analysis, output capture, and other functions will ensure consistency across an application and from application to application, saving time and eliminating challenges for both Development and Operations.

Test early and often

Today, the syntax of source code is checked as it’s created. Apply a similar approach to job definitions. In traditional development environments, new, scripted, and therefore inconsistent scheduling may work fine during testing only to fail in production. By automating scheduling, similar notation and interfaces for all jobs can be used at the earliest stages, which in turn allows for early and accurate testing. This is even more effective when “infrastructure as code” is used to provision a test environment that is as close to the production environment as possible, making it possible to anticipate and eliminate resource contention and inconsistencies with other workloads.

Use a source code management system

A source code management (SCM) system (such as GIT, Subversion, CVS, TFS, etc.) creates a central repository for storing application components and managing versions. It enables fallback to previous versions, “diff-ing” versions to identify changes, and managing potential drift in deployed systems. It also provides a quick and reliable method for building or rebuilding the application in new environments.

When any component in the SCM is revised, the build tool (such as Jenkins®) rebuilds and tests the application for errors or functionality regression. Typically, when a test fails, the entire development team must work to fix the problem, causing delays. The use of Jobs-as-Code will naturally reduce the number of errors.

Further, more complex applications require more complex development lifecycle, often involving developing, testing, integrating, staging, pre-production testing, user acceptance testing (UAT), and more. By combining Jobs-as-Code with an SMC and early testing, increased complexity does not have to lead to more downstream challenges. Instead, applications arrive in production as consistent and thoroughly tested as possible.

Consider the value equation

For an application to deliver the greatest return on investment, the development process must be as streamlined and efficient as possible, while the application should run in production with the fewest possible issues. Jobs-as-Code should now be seen as a key element in achieving both these goals.

Finally, keep in mind that even the most rigorous development practices will occasionally fail. Jobs-as-Code adds visibility to applications so that when necessary, the operations and support teams can more quickly identify, analyze, and resolve problems to make the application available again.

By Joe Goldberg, Innovation Evangelist at BMC Software

Share

Featured Articles

5 minutes with: Dr. Juan Bernabe Moreno, IBM

We spoke with Dr. Juan Bernabe Moreno, Director IBM Research Europe, UK & Ireland, about how AI and quantum computing can work to enhance sustainability

Infosys: European firms struggle to generate gen AI value

Research from Infosys forecasts that European companies will increase their generative AI investments by 115% in the next year, up to US$2.8bn

KPMG appoints Global Head of AI to drive AI strategy

KPMG marks next phase in its AI strategy with appointment of Global Head of AI and launch of global framework for design, build and of use of AI solutions

Google unveils Gemini, its largest and most capable AI model

AI & Machine Learning

Technology key to integrating sustainability into strategies

Digital Transformation

Hitachi Vantara addresses cloud demand with Google Cloud

Cloud Computing