Azure API Management's GenAI Gateway with Andrei Kamenev
How do you manage APIs to GenAI, and how can GenAI help with API management? Carl and Richard chat with Andrei Kamenev about the latest features coming to Azure API Management. On the one hand, there are Copilot tools to help craft and understand APIM policies, which can get very complex. Then, there is the provisioning of access to GenAI-related APIs like the Azure OpenAI service, which utilize tokens - and those tokens mean money, so they need to be controlled. The GenAI Gateway provides the ability to rate-limit token issuing and all the other capabilities you expect from APIM. Prompt caching is in preview and can decrease the cost of repeated use of the same prompts. Many of the features are new, and more are coming!
Guests:
Andrei Kamenev
From 2016 to 2024, Andrei worked at Microsoft in various architect roles in Europe helping customers to bring their applications to Azure. Now he works as a product manager at Azure API Management.
Links:
- FluentValidation https://github.com/FluentValidation/FluentValidation
- Azure API Management https://azure.microsoft.com/products/api-management
- Azure GenAI Gateway Announcement https://techcommunity.microsoft.com/t5/azure-integration-services-blog/introducing-genai-gateway-capabilities-in-azure-api-management/ba-p/4146525
- Azure OpenAI Service https://azure.microsoft.com/products/ai-services/openai-service
- Azure AI Gateway Samples https://github.com/Azure-Samples/AI-Gateway
- Azure AI SDK https://learn.microsoft.com/azure/ai-studio/how-to/develop/sdk-overview?WT.mc_id=DT-MVP-10953