Agent Factory: 12-Wk Proof of Concept

Business challenges addressed

The rise of Generative AI has created a new set of needs and priorities for companies across industries, and one of the common need is to manage a series of complex conversational agents to provide assistance on several business aspects. A centralised platform, capable of ensuring scalability, providing through shared development standards, and optimising time while also offering the necessary flexibility to quickly adapt to new scenarios and operational needs, has a key importance

Cluster Reply, part of Reply Group, has created the Agent Factory, a Generative AI Platform to centralize and standardize the management of Generative AI use cases. The solution is a based on three main components:

• the Core Platform, a platform that centralizes the management and monitoring of use cases, providing a single access point for developers and monitoring,
• the Template, standardized code libraries, which simplify and accelerate the development of new use cases.
• the Agents Portal, a web based application that provides use cases access to internal/external end-users This solution accelerates the adoption of AI within the company, reducing the time needed to integrate and develop new applications while ensuring compliance with common guidelines.

Key benefits

• Reduction of Time-to-Market: Standardisation and modularisation speed up the development and release of new use cases.
• Simplified maintenance and control: The modular structure makes managing and updating the platform simpler and less costly.
• Scalability and flexibility: The platform and templates easily adapt to new needs, ensuring continuous growth and the possibility of integrating future use cases.
• Governance & Compliance Integrated: The built-in mechanisms ensure a safe and compliant environment in the use of GenAI.
• Guided AI adoption: Thanks to centralized governance and standardized processes, the adoption of AI spreads uniformly within the company.

Cluster Reply Agent Factory

By combining large language models (LLMs), Retrieval Augmented Generation (RAG) with company data sources and advanced analytics, Cluster Reply Agent Factory is a comprehensive platform built using cloud technology Microsoft Azure and based on microservices and modules dedicated to specific functions, It includes:

• Core Platform a microservices layer with different modules to govern and streamline Generative AI use cases, provides a module to make language models accessible (Azure OpenAI), an AI guard-railing module to provide safety barriers to filter harmful content (Azure AI Content Safety), a control module to centrally manage the models used (Azure SQL DB + Azure Kubernetes Service), a monitoring system for observability (Azure Monitor + PowerBI), and an auditing module to archive all conversations and all inputs given to the language models (Azure Storage + Azure Kubernetes Service), a service to monitor and evaluate Use Case performances (Azure AI Foundry).
• API layer that provides access to all the components of the Core platform implemented using Azure API Management.
• Templates and standardized code libraries to manage functionalities such as document ingestion, processing and indexing (Azure Kubernetes Service + Azure AI Search), as well as orchestrating use cases (Semantic Kernel + Azure Kubernetes Service), managing the history of past chats and user feedback (Azure Kubernetes Service + Azure Cosmos DB), and integrating with Microsoft Teams leveraging Azure AI Bot Service. These libraries, acting as templates, easily enable users activate new use cases with single-agent and multi-agent systems
• Agents Portal a web application hosted on Azure Kubernetes Service and integrated with Azure Entra ID for internal/external users authentication, that provide access to all use cases managed in the platform depending on user profiles with a common and highly customizable user interface

What’s included

Proof of Concept timeline:

• Use cases identification (1-2 weeks, optional): Workshops with business and IT stakeholders to identify and select the more valuable use case.
• Use case discovery (1-2 weeks): analyze selected use case in detail, available data and expected benefits and metrics for success.
• Implementation (6-8 weeks): PoC Environment preparation and use case implementation.
• Test and tuning (1-2 weeks): use case testing and fine tuning, benefits and metrics monitoring.
• Use case(s) scaling (next weeks): roadmap for Generative AI scaling in the company key business areas.