LiveBench: A Challenging, Contamination-Free LLM Benchmark
🧪 Evaluation, Observability & Safety
Model evaluation, prompt testing, observability, guardrails and red teaming
All 1040🧠 Large Language Models 24🤖 AI Agents 159🧩 App Development Platforms 23📚 RAG & Knowledge Bases 47💬 Chat UIs & Clients 41🖥️ Local Inference & Deployment 87👨💻 AI Coding 97🛠️ Frameworks & SDKs 46🔌 MCP & Tool Ecosystem 36🎨 Image Generation 26🎬 Video Generation 17🎙️ Speech & Audio 45👁️ Computer Vision 26📄 Document Parsing & Scraping 25🗄️ Vector Databases 25🏋️ Training, Fine-Tuning & MLOps 71🧪 Evaluation, Observability & Safety 79🚪 API Gateways & Proxies 12🔎 Search & Deep Research 16⚙️ Deep Learning Libraries 57🏷️ Data & Annotation 32🔬 Industry & Science AI 33⚡ Productivity Tools 16
Trained models & code to predict toxic comments on all 3 Jigsaw Toxic Comment Challenges. Built using ⚡ Pytorch Lightning and 🤗 Transformers.
Readymade evaluators for your LLM apps
The nnsight package enables interpreting and manipulating the internals of deep learned models.
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
Cut AI context cost without trusting the compressor. Every reduction is reversible, byte-exact recoverable, and carries an auditable receipt. Local-first, works through proxy, MCP, SDK, or agent wrapper.
Microsoft’s generated AI risk identification open source automation framework.