Przejdź do treści
Global sound
The Tech Trek

The Tech Trek

AI Observability: Measuring Quality, Risk, and ROI

· 25 min · Odcinek 712

Notatki do odcinka

An AI agent can be fast, available, and still get the answer wrong. That changes what engineering teams need to measure.

Jeetendra Soneja, VP of Engineering, AI Platform at iHerb, joins The Tech Trek to discuss why AI observability goes beyond traditional software monitoring. Teams need to understand not just latency and errors, but also response quality, groundedness, cost, and whether an agent completes its intended task.

We also explore how to measure AI ROI without relying on misleading productivity metrics. Jeetendra shares his perspective on risk based guardrails, the skills engineers need, and why observability should begin during development rather than waiting until production.

Key Takeaways

Monitor agent quality and task completion alongside latency, availability, and cost.
Build evaluations around the specific business problem the AI system is solving.
Connect AI spending to business outcomes, not just token usage or code output.
Start with basic observability, then expand controls as AI adoption grows.

Episode Highlights

01:00 Why AI observability requires more than traditional monitoring.
06:00 The metrics that matter for cost, quality, and reliability.
08:00 How to measure AI ROI beyond productivity claims.
11:00 Setting risk tolerances and choosing appropriate guardrails.
16:00 The engineering skills becoming more valuable with AI.
22:00 Proving value before scaling AI systems.

One Line That Stuck

"It's one thing to get it done, it's another thing to get it done economically."

Follow The Tech Trek for more conversations about building and operating modern technology teams.

Strona: podcasters.spotify.com

Na żywo, ostatnia godzina: 14 591 utworów na 6 413 stacjach w 146 krajach

Odkrywaj świat przez radio

Więcej sposobów na odkrywanie (9)

Usłyszałeś coś fajnego? Zapisz to albo otwórz od razu w Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Rede São FranciscoSłuchaj dalej