Multi-agent evaluation
Register multiple agents and execute repeatable test suites against each target.
mate365
Multi-Agent Testing Environment — AI Agent Quality Testing Platform
Project Overview
mate connects to AI agents, runs automated evaluation suites against them, tracks quality over time, and red-teams them for adversarial vulnerabilities. It supports Microsoft Copilot Studio, Azure AI Foundry*, generic HTTP agents*, and Parloa* out of the box, with a modular architecture for custom connectors, judges, and red-team providers.
* Roadmap items
At a Glance
8
Attack Categories
4
Connector Types
4
Judge Modes
API
OpenAPI + Scalar
Pluggable judge modules
Combine deterministic rubrics, LLM scoring, CopilotStudioJudge Mode with rubrics and hybrid evaluation strategies.
Red teaming (roadmap)
Probe for jailbreaks, prompt injection, hallucination, privacy leak, and more.
Docker and Azure ready
Run locally with Docker Compose or deploy with Bicep to Azure Container Apps.
H
About the Author
Created by Holger Imbery. Connect on GitHub or LinkedIn to learn more.
R
Latest Release
Open the newest release package and changelog directly on GitHub.
Explore the project
Use the repository for source code and releases, or open the wiki for setup and architecture docs.