Claude Skill
suyoumo/ClawProBench
ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.
Overview
Repository
Install this Skill
pip install uvRegistry
Summary
ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime, featuring deterministic grading and repeated-trial reliability.
ClawProBench 是一个以实时执行为优先的基准测试框架，用于在 OpenClaw 运行时环境中评估 LLM 代理，具备确定性评分和重复试验可靠性。
Key features
- Live-first benchmark harness for LLM agents
- Runs inside the OpenClaw runtime environment
- Deterministic grading for consistent evaluation
- Repeated-trial reliability measurement
- Supports leaderboard-style comparisons
Use cases
- Evaluating LLM agent performance in live environments
- Comparing agent reliability across repeated trials
- Benchmarking agents for OpenClaw-based applications
- Building reproducible agent evaluation pipelines
- Tracking agent improvements via leaderboard metrics
README excerpt
<div align="center"> <img src="docs/assets/openclawprobench-logo.svg" width="160" alt="ClawProBench Logo"> # ClawProBench [](#benchmark-profiles) [](#benchmark-profiles) [](#benchmark-profiles) [](#quick-start) [](LICENSE) > Transparent live-first benchmark harness for evaluating model capability inside the OpenClaw runtime. <br> > 102 active scenarios, 162 catalog scenarios, deterministic grading, and OpenClaw-native coverage. </div> <p> <a href="README.zh-CN.md"><strong>简体中文 README</strong></a> </p> ClawProBench focuses on real OpenClaw execution with deterministic grading, structured reports, and benchmark-profile selection. The default ranking path is the `core` profile; broader active coverage remains available through `intelligence`, `coverage`, `native`, and `full`. The current worktree inventory reports `102` active scenarios and `162` total catalog scenarios (`60` incubating) via `python3 run.py inventory --json` and `python3 run.py inventory --benchmark-status all --json`. ## Leaderboard Browse the public leaderboard and benchmark cases at **[suyoumo.github.io/bench](https://suyoumo.github.io/bench/)**. [](https://suyoumo.github.io/bench/) [](https://suyoumo.github.io/bench/modelpk/) ## Get Involved We sincerely thank friends from Kimi and Qwen for their helpful feedback and improvement suggestions