Quantcast
Channel: Joab Jackson, Author at The New Stack
Viewing all articles
Browse latest Browse all 697

Kraft Trims Snowflake Bill with AI App to Fine-Tune Queries

$
0
0

How do you make the most of your Snowflake deployments?

By optimizing the heck out of query executions and data warehouse sizes, according to a case study presented at Snowflake Summit 2025 by Kraft Heinz lead for analytics Karen Blomberg and Infosys senior technology architect Sudhir Nune.

Blomberg, working with Infosys, was able to save $56,000 annually on its Snowflake bills just by looking for inefficient practices within its Snowflake data operations. It identified high-cost queries, improper timeouts, unnecessary backups and even excessive view-to-table conversions, and about a dozen other data anti-patterns.

Then it built an app to automate the process.

 

A Continual Cleanup

Kraft Heinz is one of the largest food and beverage companies in the world, with over $26 billion in annual sales. For its Snowflake deployment, it uses about 67,000 credits a month. Kraft maintains about 800TB of storage across 350 warehouses, with 400,000 tables.

The company doubled down on its data-driven pipelines in 2020, looking for ways that data could provide more value for the company. Three years later, the food company execs started asking “What is the cost of those data products that we built so far? And, you know, do we have any opportunities to improve how we’re storing those data products and loading those data products,” Blomberg said.

Indeed, the company did have a lot of ad-hoc warehouses that could probably be right-sized. Queries could be fine-tuned to not use so many resources.

In the beginning of 2024, the company began an initiative that identified misaligned data warehouses and alerted the respective business owners to make the appropriate adjustments.

The project was such a success, Kraft looked at ways it could be automated.

It would be necessary.

Blomberg knew that Snowflake would continue to introduce new services, and Kraft’s onw data warehouses and workflows themselves would be in a constant state of flux.

So a regular cleanup of how Kraft used Snowflake would be a continual task.

“We really need to make this a sustainable, repeatable process,” she said.

An App for That

With Infosys and Snowflake, the company built an internal app to identify un-optimal and mistuned data warehouses. The app runs on an quarterly analysis and sends the results to the business owners.

The app was built entirely as a Snowflake Native App. This approach would be easier than using a third-party service, Blomberg said, given that Snowflake had all the functionality and data needed, as well as the built-in security and governance controls Kraft was already using.

The app draws from all the operational data Snowflake provides to all its customers. Flagging a table for storage optimization might involve looking at how many users are accessing the table, how frequently it is being loaded, how much data is there, how frequently the data is being deleted, Nune said.

The dev team also created a set of metadata to explain the business purposes for each resource.

The project used Snowflake’s Streamlit, a cloud-hosted version of the open source Python Streamlit library for building data apps.

Snowflake also provides the ability to build models, which use Snowflake operational data to make recommendations.

The recommendations come from two pools, Nune explained. One is deterministic, which are best practices already known by Kraft’s engineers, and used for identifying sub-par performance. The other is model-driven, a set of ML-based best practices Kraft would like its systems to adhere to.

The company now has an in-house user-design team to make the app more actionable to the line-of-business owners.

“We ensure that visibility provides a lot of information. Without the visibility, a lot of companies or teams will lose the confirmation of what they need to act on,” Nune said.

The dashboards provide cost and usage tracking and what has changed over the past few months.

The app can be used to assign the issue, and track its results, through integrations with Jira, ServiceNow and other API-based systems.

Once the issue is resolved, it can even offer an estimation of how much money was saved.

It rolls up nicely to management as well.

“We have a single pane of glass,” she said. “It shows what the utilization is for our Snowflake environment. It shows what opportunities are identified and the expected savings you could have if you address those opportunities. And then it finally will show you the opportunities that have been addressed already, and what were the savings related to those opportunities.”

Savings Galore

The app inspects consumption patterns on compute, storage and miscellaneous services, using a filter of 19 types of “opportunity cuts” where efficiencies could be made, along with explanations of how the issue could be resolved.

 

Types of cost opportunities

All sorts of Snowflake resources where you could save company money.

 

“Every small query adds up the cost,” Nune said. A single query may only cost a credit, but if it runs every 15 or 20 minutes, can add to a hefty bill.

One high-profile “opportunity,” as Nune calls them, are view-to-table conversions, which are expensive ways (in Snowflake land) of making data more readily available.

Backups are another area ripe for savings, Nune said.

Every organization has unnecessary backup tables. They may have been created as an app is updated, and the devs want to back up the data as well. First, this is redundant because of Snowflake’s own Time-Travel feature, Nune said. But then the app design team forgets about the backup tables altogether.

For these cases, the app will monitor the usage of the table, to determine how often it is consulted, if at all. If it is a dead table, it is flagged for removal. It can even be configured to drop the tables automatically if not used periodically.

Thus far, the app has identified over 100TB of redundant storage throughout Kraft’s accounts.

Snowflake Summit 2025 presentations can be viewed after free registration.

The post Kraft Trims Snowflake Bill with AI App to Fine-Tune Queries appeared first on The New Stack.

The food giant has developed an app with Infosys that can monitor for sub-optimal data warehouse deployments and mis-tuned queries.

Viewing all articles
Browse latest Browse all 697

Trending Articles