One of the dirty little secrets about large language models (LLMs) is that they can’t easily interpret structured information captured in data warehouses and databases (or even CSV files, really).
Yes, all that work to create schemas and structured datasets to reflect the business processes — all for naught, when it comes to powering an LLM at any rate.
Now Kumo, a San Francisco startup, is moving beyond the LLMs to offer greater discoverability in structured and semi-structured data, the mainstay of the enterprise.
The company has advised a new model, built on a new machine learning (ML) technique called Relational Graph Transformers, that “brings the same elements to enterprise data that LLMs brought to text,” said Vanja Josifovski, Kumo CEO, in an interview at Snowflake Summit earlier this month.
Kumo works as a Snowflake-native app, so it can run against your Snowflake data tables. You can also upload your structured data for the company’s cloud service to analyze.
The model was designed to save time in spotting trends within structured datasets, typically a job still done largely by hand by data scientists. And, thanks to graph-based neural nets, it doesn’t need to know semantic context, or even schema annotations of the data itself.
This approach can cut the time of identifying a pattern twentyfold or more, compared with using a feature store or even building your own model.
And it doesn’t even require an LLM.
The Problem With Structured Data
“The reality is that the most valuable data for the vast majority of the companies is stored in a structure,” Josifovski said. “That’s where the signal is.”
Database tables are comprised of rows, each a row a record. Each column is for a specified attribute, and one column is the primary key, used for indexing. Foreign keys link to other tables. The schema may be documented, or it may not. In either case, it wasn’t connected to the data itself.
At the time, breaking apart data in multiple tables was very efficient for a number of reasons: to keep down storage costs, speed retrieval times and so on. But it did ML no favors.
“That chasm is still there, 20 years later, for a variety of things,” he said.
Josifovski started his research at IBM, and then went on to residencies with Yahoo and then Google, and was at Google around the time his colleagues released the famous attention paper of 2015, which set the groundwork for ChatGPT by focusing attention on words to predict the next words.
This works great for unstructured text, or most of the text just lying around the internet.
“A data warehouses and a graph is the same thing for us.”
— Vanja Josifovski, Kumo CEO
Understanding a database table is far trickier for an LLM, because the information it would need to assemble some sort of context for the data is scattered across different tables, he explained.
“Trying to work on the semantic level is unreliable,” he said.
A column name may have no bearing, for various reasons, on what is in the column itself. Josifovski also worked at one point as the CTO for Airbnb. There, he recalled, they had thousands of columns in a database, and they all had only a very slight variation in their often-inscrutable take names. “You have to know exactly which variation is, to be able to produce the right analytics,” he said.
The problem is, after considerable investment, data warehouses just give you the picture of what is happening, but you still have to identify the patterns. This is the work of many data scientists. This is why many organizations still don’t get an ROI from a data warehouse: the work doesn’t scale.
“You have to bring the data to a single model, and that model is task-specific,” he explained.
Relational Graph Transformers
Text and images are simple for LLMs to process. Text is strings and images are matrices, Josifovski said.
Until recently, image classifiers still required manual intervention to identify the edges. A few years later, convolutional neural networks automated the task, basically by running a sliding window over the image. And text could be deciphered in a similar way.
And this is basically the way Kumo’s approach works. It uses a technique called Relational Graph Transformers.
“We found a way to learn straight from the raw relational data, [across] multiple different tables, without having to produce features or training sets,” Josifovski said. It works in a similar way that computer vision learns from raw pixels.
How does it work? Think of your data warehouse as a graph.
“Every row is a node, every foreign primary key connection is a link,” he explained. A data warehouse of one billion rows is a billion-node graph, which then can be analyzed for emerging patterns.
“Instead of attending over words, we attend over nodes in the graph,” Josifovski said.
What Can Relational Graph Transformers Predict?
With Relational Graph Transformers, the human is no longer needed to identify a signal.
For instance, an e-commerce firm could predict churn using a column of all the products previously purchased. The neural network does that without the need for any sort of semantic context whatsoever.
“And with that, we cut the human out of the loop. There’s no need to somebody manually figure out where the signal is,” he said.
To flush out some generic patterns, the company pre-trained a model on a billion-node data warehouse, stuffed to the brim with synthetic data from a variety of public datasets.
Adding a helping hand was an open source component called PyTorch Geometric, which writes and trains graph neural networks.
From there, the model came to recognize a number of generic patterns, which the company has sussed out how to map these signals to predictive outcomes.
In preliminary tests at any rate, “the results of these predictions are better than when a skilled PhD spends time with traditional methods to produce models,” Josifovski asserted.
“We did not expect this to work this well,” he admitted.
The company is previewing the technology on its website. You can upload a dataset and then ask predictive questions, in a SQL-like language, where instead of writing a SELECT statement, you write a PREDICT statement.
Scaling Graphs Is Hard
The chief challenge with graphs is that they are difficult to scale. Unlike documents, there are no natural scaling techniques like sharding. (And though it borrows ideas from graph computing, Relational Graph Transformers do not require a graph database, which would be too slow for the job, Josifovski noted.)
Kumo has some secret sauce to scale the graphs, and also quite a lot of sweat equity into setting up a scalable platform.
The company counts among its customers Reddit, DoorDash and even Snowflake itself. Another group is self-starters, firms without a lot of data science muscle who nonetheless want to use the predictive capabilities.
The post Kumo Surfaces Structured Data Patterns Generative AI Misses appeared first on The New Stack.
Kumo's Relational Graph Transformers can surface insights from relational databases that LLMs (and data scientists) miss. The secret sauce? Graph computing.