Evaluating agents is leaving the SDK and moving into the tracing layer

Stripping evaluation from the framework

The main hurdle in deploying an agent now lies in proving that it works. When every new framework like LangGraph, LlamaIndex or the OpenAI Agents SDK brings its own evaluation method, teams are forced to lock into one vendor's tooling instead of running cross-framework comparisons. Amazon Bedrock AgentCore Evaluations, released by AWS, aims to solve this by stripping evaluation out of the framework entirely.^[1]

Instead of looking at the framework the application uses, the service reads OpenTelemetry spans sent to CloudWatch. It reconstructs the agent's run from the top-level request down to each model step and tool invocation. It then applies built-in checks for goal-success rate or correctness.^[1]

The new standard in the tracing layer

Evaluation moves out of the developer tooling and into the tracing layer; whether running in a continuous integration pipeline or monitoring production traffic, the mechanism remains the same.^[1]

The model layer is commoditizing, and value is moving up. When tracing and evaluation can be consumed like a standard API, orchestration becomes routine, and value shifts to the layer that audits the system. Developers gain an independent evaluation ground where they can swap out the underlying agent, rather than being locked into increasingly rigid frameworks.^[1]