Google's end-to-end machine learning platform with a complete ecosystem.
tiktoken
OpenAI 模型使用的快速 BPE 分词器。
- Category
- ⚙️ Deep Learning Libraries
- Repository
- openai/tiktoken
- Stars
- ★ 19,489
- Forks
- 1,650
- Language
- Python
- License
- MIT
- Last updated
- 2026-08-17
- Visits
- 0
Tokenization
Similar projects
A unified interface to tens of thousands of pre-trained models covering text, images, audio and multimodal tasks.
A deep-learning framework with dynamic computation graphs, the mainstream choice in both research and industry.
The classic Python machine-learning library: simple and efficient.
多后端（JAX / TF / PyTorch）的深度学习 API。
最简单快速的中型 GPT 训练与微调仓库。