dutchman labs - Eval Studio

Most AI agents today are shipped without real testing. Teams rely on manual prompts, spot checks, or hope things work in production. Eval Studio changes that. Eval Studio is a CLI-first tool that lets you run evaluations directly on your agent from your own codebase. What it does: Detects your agent automatically Generates evaluation datasets based on your agent’s logic Runs tests locally Surfaces failures and behavioral gaps Exports results to JSON, CSV, or pytest