Claude Skill
pinchbench/skill
PinchBench is a benchmarking system for evaluating LLM models as OpenClaw coding agents. Built by the team at kilo.ai.
Overview
Repository
Install this Skill
git clone https://github.com/pinchbench/skill.gitRegistry
Summary
PinchBench is a benchmarking system designed to evaluate LLM models as OpenClaw coding agents, built by the team at kilo.ai.
PinchBench是一个用于评估LLM模型作为OpenClaw编码代理的基准测试系统。由https://kilo.ai的人类团队用🦀制作。
Key features
- Evaluates LLM models as coding agents
- OpenClaw-based benchmarking framework
- Built by the team at kilo.ai
- Python-based implementation
Use cases
- Benchmarking LLM coding agent performance
- Comparing different LLM models for coding tasks
- Research on agent-based code generation
README excerpt
# 🦀 PinchBench **Real-world benchmarks for AI coding agents** [](https://pinchbench.com) [](LICENSE) <!-- task-count-badge --><!-- /task-count-badge --> > **Note:** This repository contains the benchmark skill/tasks. It is NOT the source of official leaderboard results. To add models to the official results, modify [pinchbench/scripts/default-models.yml](https://github.com/pinchbench/scripts/blob/main/default-models.yml). PinchBench measures how well LLM models perform as the brain of an [OpenClaw](https://github.com/openclaw/openclaw) agent. Instead of synthetic tests, we throw real tasks at agents: scheduling meetings, writing code, triaging email, researching topics, and managing files. Results are collected on a public leaderboard at **[pinchbench.com](https://pinchbench.com)**.  ## Why PinchBench? Most LLM benchmarks test isolated capabilities. PinchBench tests what actually matters for coding agents: - **Tool usage** — Can the model call the right tools with the right parameters? - **Multi-step reasoning** — Can it chain together actions to complete complex tasks? - **Real-world messiness** — Can it handle ambiguous instructions and incomplete information? - **Practical outcomes** — Did it actually create the file, send the email, or schedule the meeting? ## Quick Start ```bash # Clone the skill git clone https://github.com/pinchbench/skill.git cd skill # Run benchmarks with your model of choice ./scripts/run.sh --model openrouter/anthropic/claude-sonnet-4 # Or run specific tasks ./scripts/run.sh --model openrouter/openai/gpt-4o --suite t
Topics
No topics yet.