Databricks & AWS Join in Push to Enhance Gen AI Capabilities

It is well known that AI and cloud computing have become integral to modern business operations, driving innovation across industries.

This means that, as companies seek to leverage AI capabilities, partnerships between technology providers are emerging to enhance and streamline AI development and deployment.

These collaborations aim to offer improved solutions for businesses looking to harness the power of AI while maintaining control over their data and intellectual property.

In this context, Databricks, a company specialising in data and AI solutions, has announced a strategic collaboration agreement with Amazon Web Services (AWS) to accelerate the development of custom AI models.

According to Databricks, the partnership will focus on utilising AWS Trainium chips to power Databricks' Mosaic AI model that helps build and manage custom AI applications efficiently, training and serving capabilities on AWS.

This collaboration also aims to provide joint customers with the ability to build, deploy and monitor high-quality, customised AI applications while maintaining control over their data.

Aims of the Databricks and AWS collaboration: explained

The agreement builds on the existing relationship between Databricks and AWS, with Databricks' AWS business reportedly surpassing a US$1bn run rate.

This growth has been fueled in part by AWS Marketplace, where Databricks says total contract value has doubled on an annualised basis over the past two years.

As part of the expanded partnership, Databricks and AWS plan to invest in various areas, including:

• Custom model optimisation and enhanced security
• Migration and modernisation for generative AI (Gen AI)
• Tailored industry solutions and accelerators

• Enhanced ease of use through new integrations on AWS Marketplace
• Joint Gen AI go-to-market programmes
• Expanded co-marketing initiatives

The partnership between Databricks-AWS also aims to accelerate the development of custom models built with Databricks Mosaic AI on AWS.

Under this agreement, Databricks will use AWS Trainium chips as the preferred AI chip to power Mosaic AI model training and serving capabilities on AWS.

AWS Trainium is a custom-designed chip for machine learning training workloads, offering high performance and cost-effectiveness.

Ali Ghodsi, Co-founder and CEO at Databricks, says:

Databricks says that joint customers can leverage Mosaic AI to pretrain, fine-tune, augment and serve large language models (LLMs) on their private data, backed by the scale, performance and security of AWS.

LLMs are advanced AI models capable of understanding and generating human-like text.

The collaboration builds on Databricks' recently expanded Mosaic AI capabilities, which include Mosaic AI Model Serving.

This feature is designed to support various model providers, including the selection of models available via Amazon Bedrock, a fully managed service that offers foundation models from leading AI companies.

Impact on AI development, deployment and customers

The partnership is also expected to enable thousands of joint customers to build, deploy and monitor high-quality, customised AI applications without sacrificing control over their data or intellectual property, Databricks reports.

Highlighting the potential for customers to derive real value from their data using GenAI, Matt Garman, CEO of AWS says:

More broadly, the collaboration is expected to benefit customers across various industries and provide value to diverse joint customers.

These include Rivian, an electric vehicle manufacturer; Block, a financial technology company; and SEGA, a prominent video game developer and publisher.

This means that this partnership aims to enhance data-driven innovation across automotive, fintech and gaming industries.

Matt emphasises the potential of the collaboration, saying: “Our collaboration with Databricks unlocks the potential for customers to drive real value from their data using GenAI.

******

Make sure you check out the latest edition of Technology Magazine and also sign up to our global conference series - Tech & AI LIVE 2024

******

Technology Magazine is a BizClik brand