ifixai-ai/iFixAi

iFixAi places your agent in a simulated business environment, issuing various requests it might encounter in a day, including those designed to induce it to overstep its responsibilities. A model from a different vendor then acts as a judge, scoring each item individually. The results provide pass/fail status and evidence for each check, along with A-to-F grades calculated across five dimensions: fabrication, manipulation, deception, unpredictability, and opacity. If any of the three hard baseline criteria are not met, the maximum possible grade is D; reports are available in both JSON and Markdown formats.

iFixAi places your agent in a simulated business environment, issuing various requests it might encounter in a day, including those designed to induce it to overstep its responsibilities. A model from a different vendor then acts as a judge, scoring each item individually. The results provide pass/fail status and evidence for each check, along with A-to-F grades calculated across five dimensions: fabrication, manipulation, deception, unpredictability, and opacity. If any of the three hard baseline criteria are not met, the maximum possible grade is D; reports are available in both JSON and Markdown formats.

Why it matters Issues such as agents overstepping permissions, fabricating information, or being induced into unsafe behavior can be detected before real users encounter them, and the tool can be integrated into CI pipelines to enforce score thresholds.

Stars 44708 (+236 / 24h)

Reasons

  • 236 new stars in 24 hours (indicating attention only)
  • Provides plugins for Claude Code and Codex, along with skills for various editors, allowing coding agents to read your code and automatically generate profiles for the agents under test

Usage

  • After running pip install ifixai, try it out with the built-in demo agent (no API Key required); for formal audits, first run ifixai setup, then add --dry-run to see the number of tests and judge calls

Risks

  • Probe requests are real requests; the README recommends running against a test environment rather than production
  • Judge models are billed per call; scoring criteria are defined by the project maintainers, so conclusions are for reference only

Open

Trendshift(GitHub Trending 快照)

Explore the latest AI Radar updates