ASSERT: Turning Written Intent into Executable Evals
🧭 ASSERT is an open-source framework that converts natural-language behavior specifications into executable evaluation pipelines, generating test scenarios, datasets, metrics, and scorecards for models, agents, or applications. The pipeline systematizes intent into a concept spec, produces an editable behavior taxonomy, generates stratified test cases, records full inference traces, and scores each trace with rationales and policy citations. Internal validation showed ASSERT improves coverage, surfaces distinct failure patterns, and yields judge agreement with humans in most cases, while SME review confirmed alignment and credibility.
