Job Description:
We are looking for a hands-on and technically strong Technical Lead to drive the design, development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.
Responsibilities:
· Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
· Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
· Must have experience with Agentic AI frameworks like LangGraph, CrewAI, AutoGen etc.
· Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals.
· Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
· Must know how to measure and benchmark the performance of the Agents.
· Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
· Understanding and experience in LLMOps and AgentOps with multi-agent coordination and debugging.
· Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.
Qualifications:
· Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 8+ years of experience, with at least 1.5 years of hand-on in GenAI.
· Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
· Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
· Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
· Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.