CAST AI, the leading Kubernetes automation platform, today announced the launch of AI Enabler, an innovative optimization tool that streamlines the deployment of LLMs and drastically reduces operational expenses. AI Enabler leverages CAST AI’s industry-leading Kubernetes infrastructure optimization capabilities to intelligently route queries to the most optimal and cost-effective LLMs, whether they’re open-source or commercial, ensuring that the more reliable and cost-effective models are used, without impact on quality or performance.
The universe of available LLMs is rapidly expanding. Up until now, infrastructure teams (including AIOps, MLOps, and DevOps) have used time-intensive, manual approaches to identifying the most optimal and cost-effective LLM for their queries. The end result is that best-fit model assumptions are often made without the needed rigor and diligence. Teams typically default to the latest and greatest LLM, which is costly and has created a substantial barrier to AI adoption.
“With the increasing availability of LLMs, choosing the right one for your use case, and doing so cost-effectively, has become a real challenge,” said Laurent Gil, co-founder and CPO at CAST AI. “AI Enabler removes that complexity by automatically routing queries to the most efficient models and providing detailed cost insights, helping businesses fully leverage AI at a fraction of the cost. This automated approach allows organizations to scale generative AI solutions across their operations without sacrificing cost efficiency.”
“Our customers have been asking for a way to harness the power of LLMs without the prohibitive costs of the most popular models,” said Laurent Gil, Co-Founder and CPO at CAST AI.“With automated model selection and the ability to launch models locally on spot GPUs, we’ve made large-scale LLM deployment feasible for companies who need real-time insights without the high price tag.”
With detailed reporting and real-time cost comparisons, AI Enabler provides visibility into LLM expenses and highlights savings over traditional models. Leveraging CAST AI’s intelligent LLM router, users can direct queries to the most cost-effective, workload-specific LLM, minimizing resource costs while maintaining high performance. The CAST AI Playground (a testing resource to compare LLM performance and cost) further empowers users to benchmark and customize configurations for optimal LLM selection with zero code adjustments.
If you’re interested in learning more about AI Enabler, sign up for a free trial or book a live demo session.
