CTI-REALM: Benchmark for End-to-End Detection Rules
🔍 Microsoft introduces CTI-REALM, an open-source benchmark that evaluates AI agents on end-to-end detection engineering by turning real-world cyber threat intelligence into validated detections. The benchmark places agents in realistic, tool-rich environments where they must read CTI reports, explore telemetry, iterate on KQL queries, and produce Sigma rules and KQL-based logic scored against ground truth across Linux, AKS, and Azure. CTI-REALM's checkpoint-based scoring surfaces whether failures arise from CTI comprehension, technique mapping, data-source selection, or query construction, helping teams decide where human oversight and guardrails are required.
