IBM Release CodeFlare, a New Hybrid Cloud AI/ML Framework
IBM Research has unveiled CodeFlare, a new framework for integrating and scaling big data and AI workflows in a hybrid cloud environment. The open-source framework aims to help developers cut back the time they spend creating pipelines to train and optimise machine learning models.
CodeFlare is built on top of Ray, an emerging open-source distributed computing framework for machine learning applications. CodeFlare extends the capabilities of Ray by adding specific elements to make scaling workflows easier, according to IBM.
Researchers and developers have to train and optimise the model first to create a machine learning model today. CodeFlare simplifies this process using a Python-based interface for what’s called a pipeline—by making it simpler to integrate, parallelise and share data. The new framework aims to unify pipeline workflows across multiple platforms without requiring data scientists to learn a new workflow language.
A simpler way to integrate and scale full pipelines
CodeFlare pipelines run with ease on IBM’s new serverless platform IBM Cloud Code Engine, and Red Hat OpenShift, the company explained. It allows users to deploy it just about anywhere, extending the benefits of serverless to data scientists and AI researchers. It also makes it easier to integrate and bridge with other cloud-native ecosystems by providing adapters to event-triggers (such as the arrival of a new file), and load and partition data from a wide range of sources, such as cloud object storages, data lakes, and distributed filesystems.
CodeFlare "goes beyond isolated tasks to seamlessly integrate and scale end-to-end pipelines with a data-scientist-friendly interface--like Python--instead of using containers,'' said Priya Nagpurkar, director, hybrid cloud platform at IBM Research. "CodeFlare can provide a simpler way to integrate and scale full pipelines, while offering a unified runtime and programming interface."
The company has already seen CodeFlare in action and cutting time. For example, one user applied the framework to analyse and optimise approximately 100,000 pipelines for training machine learning models, CodeFlare cut the time it took to execute each pipeline from 4 hours to 15 minutes
GE Autonomous Robot ‘ATVer’ Crosses Terrain in US Army Demo
General Electric’s (GE) Research Lab team, led by Senior Robotics Scientist, Shiraj Sen, have successfully completed Year 1 of a project with the US Army through its Scalable Adaptive Resilient Autonomy Programme (SARA) to develop and demonstrate a risk-aware autonomous ground vehicle that was capable of navigating safely in complex off-road test conditions.
The autonomous robot “ATVer” successfully navigates on its own through unstructured environments, including forests and heavily wooded areas. Sen explained how one of the biggest challenges with autonomous systems is overcoming risk factors, especially when it involved equipment for military operations.
“With the successful demonstration of our ‘risk-aware’ autonomous ground vehicle in our project with the Army, we’ve made progress in removing some of those risks and hopefully, provided a clearer path to more autonomous systems applications further down the road … or off-road,” he added.
“In future Army scenarios, autonomous systems will have to reliably plan in the presence of challenging features they encounter while maneuvering in complex terrain,” Eric Spero, SARA Programme Manager said. “Incorporating risk and uncertainty into the autonomy decision-making process enables our testbed platforms to show us what it looks like to plan a direct path instead of taking the long way around.”
Using AI to take an algorithmic approach
A key factor in enabling the breakthrough in addressing risk was the integration of GE’s Humble AI technology, according to Sen. Humble AI is an algorithmic approach developed by GE artificial intelligence (AI) scientists that is capable of taking a step back and assessing much like a human might do when it encounters an uncertain situation.
GE scientists have already field-tested its Humble AI algorithms to safely optimise the control of wind turbines to maximise energy input. In the case of wind turbines, the Humble AI operates within a zone of competency where it bases its decisions on known operating scenarios with which it is familiar. When it encounters a scenario it has never seen, it is designed to take a step back and relinquish control of the turbine into a default safe mode.
“Our project and partnership with the US Army has really enabled us to make some important advances in autonomous systems,” Sen said. “We believe the advances made on this project will not only help accelerate the deployment of future driver-less vehicle technologies; they will help encourage more autonomous solutions in other industry sectors like energy, aviation and healthcare that people depend on every day.”
GE’s project was one of eight funded by the U.S. Army’s Combat Capabilities Development Command Army Research Laboratory to advance autonomous, off-road navigation capabilities for military ground vehicles.
(Image: GE Research)