Why most enterprise AI agents never reach production and how Databricks plans to fix it

Many efforts to develop the company’s AI agents never reach production and it is not because technology is not ready. The problem, according to Databricksis that companies are still counting on manual assessments with a slow, incoherent and difficult to evolve process.

Today, at the Data + AI summit, Databricks launched Mosaic Agent Bricks as a solution to this challenge. Technology relies and extends Mosaic AI agent executive The company announced in 2024. In simple terms, it is no longer good enough to be able to build AI agents in order to have a real impact.

The Mosaic Agent Bricks platform automates the optimization of agents using a series of innovations supported by research. Among the main innovations are the integration of Tao (Adaptive optimization of testing time), which provides a new approach to AI adjustment without the need for labeled data. Mosaic agent bricks also generate synthetic data specific to the field, creates benchmarks sensitive to tasks and optimizes the balance of quality at a cost without manual intervention.

Basically, the objective of the new platform is to resolve a problem that data users have had with the development efforts of existing AI agents.

“They flew blindly, they had no way of assessing these agents,” said Databricks technology technology director Hanlin Tang. “Most of them counted on a kind of manual monitoring of the manual atmosphere to see if the agent sounds good enough, but that does not give them the confidence necessary to enter the production.”

Research innovation on the production scale of corporate AI

Tang was previously co-founder and CTO of Mosaic, who was acquired by Databricks In 2023 for $ 1.3 billion.

At Mosaic, a large part of research innovation did not necessarily have an immediate business impact. All this changed after the acquisition.

“The great moment of the bulb for me was when we launched our product for the first time on Databricks, and instantly, overnight, we had, like thousands of business customers who use it,” said Tang.

On the other hand, before the acquisition, Mosaic would spend months trying to obtain a handful of companies to try products. The integration of the mosaic in Databricks has given Mosaic research team to large -scale business problems and has revealed new areas to be explored.

This business contact has revealed new research opportunities.

“It is only when you have contacts with corporate customers, you work deep with them, that you really discover interesting research problems to continue,” said Tang. “Agent bricks … in some ways are a kind of evolution of everything we work with Mosaic now that we are all entirely, entirely in brick.”

Resolve the agentic AI assessment crisis

Business teams face an expensive test and error optimization process. Without a reference of task or test data specific to the domain, each adjustment of the agent becomes an expensive riddle game. Quality drift, cost exceeding and missed deadlines follow.

Bricks agent automates the entire optimization pipeline. The platform takes a high-level task description and corporate data. It automatically manages the rest.

First, it generates specific assessments and LLM judges. Then it creates synthetic data that reflect customers data. Finally, he is looking for optimization techniques to find the best configuration.

“The customer describes the problem at a high level and he does not fall into the low level details, because we take care of them,” said Tang. “The system generates synthetic data and builds personalized LLM judges specific to each task.”

The platform offers four agent configurations:

• Information extraction: Converts documents (PDF, e-mails) into structured data. A use case could be retail organizations that use it to extract the details of the supplier’s PDF product, even with complex formatting.
• Knowledge assistant: Provides specific responses and cited from business data. For example, manufacturing technicians can obtain instant responses from maintenance manuals without digging in binders.
• LLM Personalized: Manages text transformation tasks (summary, classification). For example, health care organizations can personalize models that summarize patients for clinical workflows.
• Multi-agent supervisor: Orchestra Several agents for complex workflows. An example of use cases is financial services companies that can coordinate intention detection agents, documents recovery and compliance checks.

The agents are great, but don’t forget the data

The construction and evaluation of agents are an essential element of the preparation for the company of AI, but it is not the only necessary part.

Databricks positions the bricks of the mosaic agent as the consumption layer AI seated at the top of his unified database. At the Data + AI summit, Databricks also announced the general availability of his Lakeflow data engineering Platform, which was presented for the first time in 2024.

Lakeflow solves the data preparation challenge. It unifies three critical data of data engineering which previously required separate tools. Ingestion manages structured and not structured data in data data. The transformation provides effective data cleaning, reshaping and preparation. The orchestration manages production workflows and planning.

The connection of the workflow is direct: Lakeflow prepares corporate data by ingestion and unified transformation, then the brick agent builds Optimized AI agents on prepared data.

“We help bring the data into the platform, then you can do ML, Bi and AI analyzes,” said Bilal Aslam, principal director of product management at Databricks, in Venturebeat.

By going beyond data ingestion, the Mosaic agent’s bricks also benefit from the governance characteristics of the Unity catalog of the Databricks. This includes access controls and monitoring the data line. This integration guarantees that the behavior of the agent respects the governance of corporate data without additional configuration.

Learning agents of human feedback eliminates a rapid stuffing

One of the current approaches to guide AI agents today is to use a system prompt. Tang has referred to the practice of “farce invites” where users are pushing all kinds of advice in an invite in the hope that the agent will follow him.

Agent Bricks presents a new concept called – the learner agent of human feedback. This feature automatically adjusts system components according to natural language advice. He solves what Tang calls the problem of rapid stuffing. According to Tang, the rapid filling approach often fails because agent systems have several components that require an adjustment.

Learning agents of human feedback is a system that automatically interprets advice in natural language and adjusts the components of the appropriate system. The approach reflects learning to strengthen human feedback (Rlhf) But works at the level of the agent’s system rather than in individual model weights.

The system manages two basic challenges. First, natural language advice can be vague. For example, what does “respect your brand’s voice” means? Second, agent systems contain many configuration points. The teams find it difficult to identify the components that must be adjusted.

The system eliminates assumptions on the components of the agent who need adjustment for specific behavior changes.

“We believe that agents will help more adjustable agents,” said Tang.

Technical advantages compared to existing executives

There is no shortage of executives and agent AI development tools on the market today. Among the growing list of supplier options are tools of Lubricole,, Microsoft And Google.

Tang argued that what makes the bricks of mosaic agent different is optimization. Rather than requiring manual configuration and adjustment, agent bricks automatically integrate several research techniques: TAO, learning in context, rapid optimization and fine adjustment.

Regarding agent’s communications to the agent, there are some options on the market today, including Google Agent protocol2age. According to Tang, Databricks currently explores various agent protocols and has not engaged in a single standard.

Currently, agent Bricks manages agent agent communication through two main methods:

1. Expose agents as termination criteria that can be wrapped in different protocols.
2. Use of a multi-agent supervisor that is aware of MCP (Model Context Protocol).

Strategic implications for business decision -makers

For companies seeking to open the way to AI, it is essential to have the right technologies in place to assess quality and efficiency.

The deployment of agents without evaluation will not lead to an optimal result and will also not have agents without solid database. When you consider agent development technologies, it is essential to have appropriate mechanisms to assess the best options.

The approach of the learning agent of human feedback is also remarkable for corporate decision -makers because it helps to guide agentics in the best result.

For companies that seek to carry out in the deployment of AI agents, this development means that the evaluation infrastructure is no longer a blocking factor. Organizations can focus resources on identifying use cases and data preparation rather than creating optimization frameworks.