BMC Software: Why developers are missing a trick by ignoring workflow automation
In DevOps workflows, automation saves time, conserves resources, reduces errors and ensures consistency. Despite this, developers continue to ignore the existence of world-class workflow automation and job scheduling solutions and instead rely on a variety of simple tools to code jobs as they build apps. Since neither these tools nor the developers adhere to the same standards, the applications passed to Operations often fail to meet production standards. And when something breaks, the problem is often hard to find and fix.
Since such large portions of business application processing is performed by job scheduling functions, automating their creation will not only save time during development, testing, and deployment, but also enable developers to focus more time on building the best business logic.
Consider the makeup of an application. Code, such as Java and Python implements the business logic. The application runs on infrastructure, such as a Linux server, web application server, or network. Databases and SQL statements create tables and rebuild indexes. Finally, many dependent jobs and tasks have to execute, often in a particular order. This job scheduling simply determines when jobs run and what to do if a job fails or can’t start when scheduled. When developers spend such a significant amount of their development time defining this essentially administrative functionality, they are, in a very real sense, wasting their time only to create future headaches for Operations that waste even more time.
“Jobs-as-Code” is a way of standardizing and automating job scheduling by embedding code using a simple notation that makes API calls to a scheduling engine, then managing these calls throughout the continuous delivery pipeline, the same way Java or Python code is managed. As such, Jobs-as-Code is fundamentally no different than other code management practices.
Here is how Jobs-as-Code can fit into a streamlined development process:
Don’t think of jobs as business logic
Job scheduling is mere instrumentation. Scripting it requires a significant effort for the build, test and support processes – and errors and inconsistency can carry over into Operations. Instead, a job scheduler that can manage flow relationships, success/failure analysis, output capture, and other functions will ensure consistency across an application and from application to application, saving time and eliminating challenges for both Development and Operations.
Test early and often
Today, the syntax of source code is checked as it’s created. Apply a similar approach to job definitions. In traditional development environments, new, scripted, and therefore inconsistent scheduling may work fine during testing only to fail in production. By automating scheduling, similar notation and interfaces for all jobs can be used at the earliest stages, which in turn allows for early and accurate testing. This is even more effective when “infrastructure as code” is used to provision a test environment that is as close to the production environment as possible, making it possible to anticipate and eliminate resource contention and inconsistencies with other workloads.
Use a source code management system
A source code management (SCM) system (such as GIT, Subversion, CVS, TFS, etc.) creates a central repository for storing application components and managing versions. It enables fallback to previous versions, “diff-ing” versions to identify changes, and managing potential drift in deployed systems. It also provides a quick and reliable method for building or rebuilding the application in new environments.
When any component in the SCM is revised, the build tool (such as Jenkins®) rebuilds and tests the application for errors or functionality regression. Typically, when a test fails, the entire development team must work to fix the problem, causing delays. The use of Jobs-as-Code will naturally reduce the number of errors.
Further, more complex applications require more complex development lifecycle, often involving developing, testing, integrating, staging, pre-production testing, user acceptance testing (UAT), and more. By combining Jobs-as-Code with an SMC and early testing, increased complexity does not have to lead to more downstream challenges. Instead, applications arrive in production as consistent and thoroughly tested as possible.
Consider the value equation
For an application to deliver the greatest return on investment, the development process must be as streamlined and efficient as possible, while the application should run in production with the fewest possible issues. Jobs-as-Code should now be seen as a key element in achieving both these goals.
Finally, keep in mind that even the most rigorous development practices will occasionally fail. Jobs-as-Code adds visibility to applications so that when necessary, the operations and support teams can more quickly identify, analyze, and resolve problems to make the application available again.
By Joe Goldberg, Innovation Evangelist at BMC Software
Discord buys Sentropy to fight against hate and abuse online
Discord, a popular chat app, has acquired the software company Sentropy to bolster its efforts to combat online abuse and harassment. Sentropy, monitors online networks for abuse and harassment, then offers users a way to block problematic people and filter out messages they don’t want to see.
First launched in 2015 and currently boasting 150 million monthly active users, Discord plans to integrate Sentropy’s own products into its existing toolkit and the company will also bring the smaller company’s leadership group aboard. Discord currently uses a “multilevel” approach to moderation, and a Trust and Safety (T&S) team dedicated to protecting users and shaping content moderation policies comprised 15% of Discord’s workforce as of May 2020.
“T&S tech and processes should not be used as a competitive advantage,” Sentropy CEO John Redgrave said in a blog post on the announcement. “We all deserve digital and physical safety, and moderators deserve better tooling to help them do one of the hardest jobs online more effectively and with fewer harmful impacts.”
Cleanse platforms of online harassment and abuse
Redgrave elaborated on the company’s natural connection with Discord: “Discord represents the next generation of social companies — a generation where users are not the product to be sold, but the engine of connectivity, creativity, and growth. In this model, user privacy and user safety are essential product features, not an afterthought. The success of this model depends upon building next-generation Trust and Safety into every product. We don’t take this responsibility lightly and are humbled to work at the scale of Discord and with Discord’s resources to increase the depth of our impact.”
Sentropy launched out of stealth last summer with an AI system designed to detect, track and cleanse platforms of online harassment and abuse. The company emerged then with $13 million in funding from notable backers including Reddit co-founder Alexis Ohanian and his VC firm Initialized Capital, King River Capital, Horizons Ventures and Playground Global.
“We are excited to help Discord decide how we can most effectively share with the rest of the Internet the best practices, technology, and tools that we’ve developed to protect our own communities,” Redgrave said.