Thursday 

Room 3 

15:00 - 16:00 

(UTC+11

Talk (60 min)

Production-Grade LLM Architecture: Lessons from Processing 50 Million AI Requests

Most AI talks show you how to call an API. This one shows you how to build production systems that don't fall over when real users arrive. Over 18 months, we scaled from "ChatGPT prototype" to a production AI platform processing 50 million LLM requests monthly.

AI

We learned that LLMs aren't just APIs with fancy responses—they're distributed systems with non-deterministic failure modes, unpredictable costs, and reliability challenges that break traditional architectural patterns. This talk is a technical deep-dive into building AI infrastructure that survives production.

Vaishnavi Gudur

Vaishnavi Gudur is a Senior Software Engineer at Microsoft and AI Frontier Ambassador who transforms bleeding-edge AI research into production systems that serve millions. At Microsoft, she architects security and compliance solutions for Teams, safeguarding 145M+ users worldwide while navigating the complex intersection of scalability, security, and user experience.What sets Vaishnavi apart is her unique position at the intersection of industry and academia. She's published research in IEEE conferences, contributed book chapters to Springer Nature and Wiley on AI-driven compliance and security, and serves as Lead Editor for an upcoming Springer volume on trustworthy AI systems. Her work spans autonomous cybersecurity systems, AI-powered defect detection, and ethical frameworks for responsible AI—always with an eye toward real-world implementation.As a sought-after speaker, she's delivered keynotes and technical sessions at SecurityWeek's AI Security Summit, IEEE Cloud Summit, and ODSC. She serves on program committees for AIES and ICAIC, and judges competitions including the Globee Awards. Her talks are known for combining rigorous technical depth with honest war stories from production—no hype, just practical patterns that survive contact with real users.Nominated for AI & Data Science Leader of the Year at the Women in Tech Global Awards 2025, Vaishnavi is passionate about building the next generation of ethical, scalable AI systems and mentoring engineers navigating the rapidly evolving AI landscape.