• Up to 50% reduction in costs and carbon footprint

How it works

CogCache leverages a cognitive caching mechanism to enhance the performance of AI-generated content. CogCache stores all LLM inputs and outputs in a cache and when similar or identical requests are made in the future, it can quickly retrieve the stored response instead of generating a new one each time, reducing the load on the AI model and speeding up the response time significantly—from the typical few seconds required to generate responses to milliseconds.

Two flavours of CogCache

1. CogCache with Caching only

You can purchase a caching only plan and you get all the benefits of CogCache, but without the LLMs. CogCache will require to bring your own Azure OpenAI deployment.

2. CogCache with Caching and Azure OpenAI LLMs

This is the recommended way to purchase CogCache. It provides the additional cost reductions of our discounted Azure OpenAI PTU-level deployments - with deeper discounts as your monthly generated token usage grows. No upfront commitment, only pay for what you use.

NOTE: Both flavours benefit the Cognitive Caching technology which allows you to view and manage the responses of LLMs and the LLM observability features which makes it easy to debug LLM based applications.

Getting started with CogCache is easy

Simply switch your code endpoints to your CogCache with the supplied key and that’s it. Your implementation doesn’t change but is faster, safer and more cost-effective.