The folks at StackLok have developed a command line utility, called ToolHive, to securely manage the Multiple Context Protocol (MCP) servers you may be running in-house, by using Kubernetes and containers.
These days, everyone is building MCPs, which allow large language model (LLM)-based applications to access data and services from applications.
Setting up multiple MCP servers, however, can lead to the usual headaches that always come with managing multiple copies of software.
“Setting up and maintaining MCP servers involves navigating a fast-changing landscape with pretty limited tooling, documentation and support,” wrote Craig McLuckie, one of the co-creators of Kubernetes, and CEO and co-founder of StackLok a developer-centric software supply chain service provider, in a blog post introducing ToolHive.
ToolHive is an open source project, under an Apache 2.0 license.
MCP Deployments
“Developers may encounter complexities in installing and configuring these servers, as well as managing updates and ensuring compatibility across different versions,” McLuckie continued.
Each MCP server relies on external packages, such as those from the Python or Node.js ecosystems, which much be maintained and present when a new server instance is rolled out.
If you want users to log in, then there must be secrets management going on as well. It’s also a good security idea (if not an organizational mandate) for strong isolation across multiple servers. Users must be limited to only those servers they should have access to.
In short, if MCP servers aren’t uniformly deployed, then the resulting variety will be a headache to manage.
Enter Kubernetes
Fortunately, McLuckie pointed out, we already have an entire stack of tools for deploying and managing applications at scale. Kubernetes, and its associated collection of supporting technologies, makes a natural choice for managing MCP servers.
The “containerisation and orchestration capabilities provide a strong foundation for isolating and managing MCP instances,” wrote Chris Burns, a site reliability engineer for Stacklok, in another blog post introducing ToolHive.
“Kubernetes’ built-in features, such as role-based access control (RBAC), network policies, and secrets management, address the security concerns that deter enterprises,” he wrote.
“Furthermore, the Kubernetes ecosystem, including tools for monitoring, logging, and automated deployment, enables a comprehensive and secure operational environment for MCP servers.”
ToolHive can be the connective tissue between Kubernetes and the world of AI agents.
Written in Go, ToolHive is packaged as a single binary and can be run from the command line. The team working on it has already committed 26 releases on GitHub, where the project has earned 192 stars.
To run a collection MCP servers, ToolHive uses Kubernetes’ StatefulSets, a workload API object used to manage stateful applications. Applications are defined in a YAML manifest, and housed in Open Container Initiative-formatted containers for uniformity’s sake. They can be easily provisioned once the StatefulSet and pod are up and running.
MCP Popularity
AI service provider Anthropic introduced MCP as an open standard last November as a way to connect AI assistants to external sources of content.
Though the initial release was geared to interfacing its LLM-based service, Claude, Anthropic touted MCP as an open standard, and the emerging agentic AI community quickly ran with the idea.
Earlier this week, Anthropic demonstrated how MCPs could be used in a much larger context. The company introduced for its own users a new feature called Integration that uses MCPs to fuse third-party services into its own Claude Desktop.
Initially, 10 commercial services have been integrated into the desktop: Atlassian’s Jira and Confluence, Zapier, Cloudflare, Intercom, Asana, Square, Sentry, PayPal, Linear, and Plaid.
The post ToolHive Simplifies MCP Server Orchestration with Kubernetes appeared first on The New Stack.
ToolHive utilizes Kubernetes' StatefulSets to streamline deployment, updates, security and overall operational environment for MCPs.