NVIDIA 官方大模型推理优化库，在 GPU 上提供极致吞吐与低延迟。
🖥️ Inferencia y despliegue local
Ollama, llama.cpp, vLLM: inferencia de modelos en local y en producción
Todo 1045🧠 Modelos de lenguaje 24🤖 Agentes de IA 162🧩 Plataformas de desarrollo de apps 22📚 RAG y bases de conocimiento 48💬 Interfaces de chat y clientes 40🖥️ Inferencia y despliegue local 88👨💻 Programación con IA 97🛠️ Frameworks y SDK 47🔌 MCP y ecosistema de herramientas 36🎨 Generación de imágenes 26🎬 Generación de vídeo 17🎙️ Voz y audio 45👁️ Visión por computador 26📄 Análisis de documentos y scraping 25🗄️ Bases de datos vectoriales 25🏋️ Entrenamiento, ajuste fino y MLOps 71🧪 Evaluación, observabilidad y seguridad 79🚪 Pasarelas y proxies de API 12🔎 Búsqueda e investigación profunda 15⚙️ Bibliotecas de aprendizaje profundo 58🏷️ Datos y anotación 32🔬 IA para industria y ciencia 34⚡ Herramientas de productividad 16
Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Open Machine Learning Compiler Framework
以 OpenAI 兼容接口运行任意开源大模型，并可一键部署到云端。
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
单文件即可运行的 GGUF 模型工具，自带 Web 界面，适合角色扮演与写作。
NVIDIA 的通用模型推理服务器，支持多框架、多模型并发部署。
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
Hugging Face 的生产级大模型推理服务，支持张量并行与连续批处理。
llama.cpp 的 Python 绑定，提供 OpenAI 兼容的 Web 服务。
High-speed Large Language Model Serving for Local Deployment
统一部署与服务各类开源大模型、嵌入与多模态模型，一行代码替换 OpenAI。
🤖 AI Gateway | AI Native API Gateway
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets
在 Intel CPU / GPU / NPU 上加速运行与微调大模型。
Accessible large language models via k-bit quantization for PyTorch.
A Datacenter Scale Distributed Inference Serving Framework
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Fast, flexible LLM inference
在 Mac 上用 MLX 运行与微调大模型的工具包。
A framework for efficient model inference with omni-modality models
Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.