DeepSeek open-sources the DeepInfer inference framework: How does it keep KV cache for three days and reduce speculative-decoding verification by 30%?
Background: DeepSeek is a Chinese AI company known for large language models. Model announcements and community claims require separate verification.
- DeepSeek
- A Chinese AI company that develops large language models and offers a chat service and developer API. Reference ↗