PostQodeJul 2025 – Present

Eva — agentic shopping assistant

What golden eval suite gates Eva tool-calling releases?

Eva is a production shopping agent Rupam Das built at PostQode: Azure OpenAI Responses API with SSE, 21 RBAC-gated tool groups (about 15 writes), PII scrubbing, citations, and a 2,000+ line golden-eval suite that is a CI merge gate.

Built an SSE-streaming, tool-calling agent on the Azure OpenAI Responses API with 21 RBAC-gated tool groups (~15 write tools), retry/circuit-breaker resilience, PII scrubbing, KB-grounded citations, and a 2,000+ line golden-eval suite in CI.

Published 2025-07-01 · Updated 2026-08-30

The constraint

Shopping agents fail in production when tools are over-privileged, PII leaks into prompts, or quality is vibe-checked after deploy.

Architecture

Azure OpenAI Responses API with SSE streaming. Tool groups sit behind RBAC, including about 15 write tools. Retry and circuit-breaker wrappers bound upstream failure. PII is scrubbed before the model. Knowledge-base answers carry citations. A 2,000+ line golden-eval suite is a merge gate, not a dashboard after the fact.

Azure OpenAI Responses APISSERBAC tool groupsCircuit breakersPII scrubbingKB citationsGolden evals

Sources / stack

Azure OpenAI Responses API
SSE
RBAC tool groups
Circuit breakers
PII scrubbing
KB citations
Golden evals

Agent control plane

01Interface
- SSE stream
- Responses API
02Policy
- 21 RBAC tool groups
- PII scrubbing
03Grounding
- KB citations
- Retry / circuit breaker
04Quality
- Golden evals
- CI merge gate

What we measured

RBAC-gated tool groups

~15 write tools never run without authorization

Golden eval suite

2,000+lines

Merge gate on every CI run

Result

Production agent with real users; write tools behind RBAC; evals blocking regressions every CI run.