As Head of Core Services at Snowflake, Carl Perry brings with him the experience of an end user. Prior to Snowflake, he worked as general manager for Square’s developer platform, and there he learned the value of Snowflake’s value proposition.
We interviewed Perry at this year’s Snowflake Summit, which showed the company doubling down on its message that enterprises should get their data correct if they want to do proper AI. We spoke about the emerging role of AI in the enterprise, the challenge of data migration and how AI can help there, as well as the importance of proper data governance.
Prior to Snowflake and Square, Perry also worked as a group product manager for Microsoft’s Power BI team and as an engineering manager at AWS’s S3.
The interview has been edited for length and clarity.
What’s different about AI? How is this different than previous disruptive technologies such as XML or cloud services?
XML was just a better way to represent data, but it didn’t fundamentally change the systems that were being used to interchange the data. It was super valuable. Don’t get me wrong. You know, using binary or EDI probably wasn’t the best choice, but XML was much better.
Cloud services were groundbreaking. I worked in Amazon Web Services on S3, and it was a reimagining of what computers already did. So now there are aspects that make it more scalable. You know, you’ve got an entire data center as your computer, and you can scale out to massive sizes. And so there are new types of things that you can build on top of, so it’s reimagining what’s on my laptop or on my desktop into something that you put into the cloud.
I think where AI starts to tip over into being different is that there are things that you can do now that you would never have been able to do with the previous systems. AI has pivoted into doing things that, frankly, you never thought would be possible, or were so expensive to do that it wasn’t a viable option until now.
During his keynote at Snowflake Summit, OpenAI CEO Sam Altman said that AI is ready for the enterprise, where it may not have been even a year ago. Do you see that with Snowflake’s customer base?
It’s almost a night-and-day difference. And it’s funny. I was watching what Sam said. I think that there have been immense investments in the frontier models to drive better accuracy. I was talking to one of my old co-workers who’s here at the conference, and he worked on early versions of ChatGPT, like 2.0, and he was saying the hallucinations there were crazy, and he thought it would never work. Now he sees what’s happening. He’s like, ‘You know, I was wrong. It’s working.’
I think that the level of focus that the frontier model builders have brought to accuracy has truly changed the game. I really do now, I think. I think you don’t want to blindly trust every single one of these models and AI agents that you’re building to go run your business, yet, right? And so that’s where I think it’s the responsibility of the teams to do a number of proofs of concepts, but validate the accuracy and correctness of the answers and potentially the suggestions that these agents are taking before letting them run wild. And that will give you a better sense of if your data and your company are ready to start using AI in its core production systems.
Have you seen any cases from this, your show, or just in general, of AI success?
I work closely with the J.P. Morgan team, and they have a product called Fusion. They’ve integrated chatbot experiences into their data. J.P. Morgan has one of the largest data footprints in the world. And it may be one of the largest buyers of data in the world. So the amount of data it has is massive, and what they do is they take all this data — its own proprietary data and data it purchased, and packaged it up into Fusion.
This is about building the foundation you need, and validating what’s happening. The team was able to see dramatic improvements in terms of the insights that their customers are able to get out of the questions they ask in natural language.
Now for the enterprise, what’s the delta between where their data needs to be, where they could really benefit something like Snowflake Cortex [Snowflake’s AI platform], and where they are at currently?
Customers are in a diversity of places on their journey, depending on their workloads. We have a ton of customers that we do migrations with. They’re on prem and they’re looking to move to the cloud to get the benefits of everything that the cloud and Snowflake can offer. So what we do there is focus how can we accelerate their transition to the cloud to start getting value out of all of the data there.
So we do a bunch of work around building tooling and improving our product to make migrations easier. We had a bunch of announcements around SnowConvert AI and leveraging AI to do full end-to-end migrations to dramatically reduce the time there.
Customers need to do the migration, but the migration itself doesn’t add a ton of value. When you’re on the system where you can apply all the innovation that’s in the cloud and in Snowflake to the data — that’s where the true business value starts to accrue.
AI is the killer app for migration …
It is one of them, absolutely. It’s about taking the time it takes to do manual testing, to writing and figuring out what tests you write to having them automatically generated based on the characteristics and understanding of the system you’re coming from to the system you’re moving to.
It’s funny: In migrations, you think you’re about 70% of the way done, and you start doing significant testing, and you realize you’re only 10% of the way through, because you’re finding so many issues.
We know this in software development, that the sooner you can pull forward the testing that you need to do and the verification, the better off you are. With the AI verification capabilities we’ve we’ve enabled in SnowConvert AI, you can start that journey much sooner.
There’s a lot of manual work that goes into migrations today. AI does an amazing job of taking things that are manual and driving automation with it. And I think that’s like the key thing with migrations. There’s a lot of manual work that gets done when we talk to customers, be it small migrations to the largest migrations in the world.
So this manual work can be done, where you write scripts and use bespoke tools to do that sort of thing, but it’s still better to use AI instead?
Think about it as an AI problem, exactly.
Tools will definitely work, but there are aspects that are special to a customer’s system — the configuration, the things that are running on top of it, that AI is able to understand and discover far more quickly. I think tooling always helps accelerate these things. But what we found with migrations is there’s a whole bunch of unknowns that actually get found later, because most of [the original data-gathering process] is manual. So let’s, let’s apply automation today. And the best tool we have to build automation faster and more quickly, that’s more accurate, is AI.
One of the themes to this year’s conference is the power of the unified data platform. What are the advantages of this approach to the customer?
Actually, the only reason that we’ve built the product this way is because this is what our customers tell us.
The enterprise may have 20 different systems that are about storing data, managing data, and dealing with the complexity of having to deal with data spread across different data systems, managing permissions, managing governance — all of this is incredibly difficult.
Everybody thought when the cloud came out that this would solve the problem, but it actually made an explosion of services. Anybody can drop a credit card in, sign up and start using a service, and bam! You have an isolated data source that’s not connected to anything. IT organizations have realized this complexity, and they’re trying to manage it more effectively.
The cloud providers and other folks are trying to build overlays on top of this. But again, if it’s not built directly into the system, there’s always going to be holes and seams and problems that that customer ends up having to do and manage themselves.
So you know, from day one, [Snowflake co-founders] Benoit Dageville and Thierry Cruanes, when talking to customers, were like, ‘We need to build a single system.’
Having worked at Oracle and seeing what it takes to manage these complex systems that get actually deployed into enterprise, they’re like, ‘That’s part of our job. We will go and take care of the hard things so our customers can actually focus.’
We took a very different approach than everybody else, and it’s really hard to do this from an engineering perspective and from a product perspective, but I can tell you that every single customer I talk to, probably the most valuable thing that Snowflake provides is a single product that has a single governance, single security, single privacy, single compliance perimeter.
We hear from customers on a daily basis how to improve. We see what problems are coming in the future, and we continue to focus on building this single, unified data platform that makes it easier to get value out of your data.
I’ve seen cases of Google Spreadsheets or Excel files run amok. Where is this? Where is that? You don’t know where anything is. There’s a lot of good knowledge here, but there’s no way to find it in today’s enterprise …
This is where our customers live every day. You know, companies build amazing technologies, but they actually have to figure out where all the [systems] are that use the data, and that the right people are using them, and that the people aren’t using the data in a way that’s not intended. This gets just incredibly complex.
Well, here’s the single system, a unified data platform, that has AI built into it to enable you to get more value out of your data. And customers are like, ‘This is exactly what we want.’
Early on, Benoit and Thierry realized that part of our core job is to take care of the hard problems that our customers have so that they then can go focus on the things that truly matter. It is critical for an enterprise to make sure they have a strong security, compliance, privacy and governance posture, but it’s really hard to do that. So we’ll just go do that for you.
Do you have any advice for helping along employees who are tied to their spreadsheet? How do you get buy-in for moving to a centralized system such as Snowflake’s?
There are two things you need to do. One, there needs to be a team that’s explaining to their co-workers why it’s valuable to have something that helps manage these things for them.
You know, everybody loves their little tool. So I think you need to, inside the enterprise, understand what’s important to that person about these tools that they’re using, and what tools can you bring to bear to help them?
The second thing is to find advocates who are actually influential within each organization. They may not like your approach in the moment, to be clear, but those are the people that we found give the best feedback. If you listen to these people, and you listen to enough of them — it doesn’t take that many of them — you’ll see what the themes are. And as long as you focus on solving those themes — not maybe the way they want you to — but when you start to actually solve these problems for them, they are happy to take on other tools, other approaches, other frameworks, to get their job done.
I would imagine that AI is making the role of governance more essential.
You’re 100% right. What it does is it reinforces the criticality and importance of things that aren’t sexy: governance, security, privacy, compliance. The harder it is to manage those things, the harder it becomes for AI to be truly valuable for your entire organization.
That’s where Snowflake really shines. You have a single location that manages all the governance, security, privacy and compliance needs for your enterprise, and with all the data you have available to Snowflake, then you can be assured because you have that strong data foundation for building AI experiences on top of that that can be trustworthy. You’re guaranteed, because you have the right policies and permissions in place.
And I think it’s really important, because if you don’t have that foundation in place, and you don’t have confidence that you have the right permissions, governance, privacy policies in place, you’re probably gonna have discomfort at exposing this data.
The post Q&A: Snowflake Analytics Chief on Centralizing Data for AI appeared first on The New Stack.
Carl Perry, head of core services at Snowflake, discusses the advances AI has made in the past year, and how AI relies on a trustworthy data platform.