A small open engine gets most of its light from a large prompt cache, because the prompt cache does most of the work. Frosted panels cover some parts of the prompt cache. These panels show the parts that DeepSeek kept.
DeepSeek released its AI engine, not its prompt cache
The cache stores the AI's work on earlier prompts, so 95 of every 100 tokens cost less
On 2026-10-20, DeepSeek released DeepInfer, the inference engine that runs its own API, the service that apps use to call its AI models. An inference engine is the software that runs a trained AI model and returns its answers.
DeepSeek also published its production numbers. One cluster, a large group of AI chips, serves 155T tokens per day. A token is a small part of a word. At off-peak prices, each chip earns $38 per hour.
One headline that many people shared said that this cluster earns $22M per day. Our estimate is about $2.2M per day. One number explains the 10x gap. The same number explains why the most valuable part of DeepSeek's service is not in the release.
By our calculation, 95 of every 100 tokens of all types are cache hits.
DeepSeek finds 96 of every 100 input tokens in its cache.
The daily volume is 200x the volume in Feb 2025.
DeepSeek gave the source numbers on Zhihu, a Chinese question-and-answer website, and in its Open Infra Day 6 report. No outside group audited them.
DeepSeek released the engine behind its own API
In April 2025, DeepSeek said that it would not release its inference engine. DeepSeek's April 2025 note gave three reasons: an old copy of vLLM, close links to internal tools and a small team. vLLM is a popular open-source inference engine.
The release has the DeepInfer repository, a deployment Q&A and a Zhihu post with production numbers.
DeepInfer runs one model, DeepSeek-V4.1-Flash, on Nvidia Blackwell GPUs and Huawei Ascend 950 NPUs. GPUs and NPUs are two types of AI chip. Each request has two steps. In prefill, the model reads all the input tokens at one time. In decode, the model writes the answer, one token at a time. DeepSeek runs prefill and decode on separate groups of chips, with about 1 prefill chip for each decode chip.
Our estimate is about $2.2M per day, not $22M
DeepSeek's prices are public, so its revenue number is easy to check. A cache hit is an input token that DeepSeek already processed and stored for an earlier request. Cache-miss input tokens are input tokens that are not in the cache. At off-peak prices, DeepSeek's API price page gives $0.003 per 1M cache hits, $0.15 per 1M cache-miss input tokens and $0.60 per 1M output tokens.
We call 1 prefill chip plus 1 decode chip a chip pair. In 1 hour, a chip pair earns this revenue:
The prefill chip reads 1.44M cache hits per second, which earn $15.55.
It also reads 60K cache-miss input tokens per second, which earn $32.40.
The decode chip writes 14K output tokens per second, which earn $30.24.
Our total is $78.19, or $39.10 per chip-hour. That is less than 3% above DeepSeek's $38.
The next numbers are our estimate, not DeepSeek numbers:
One chip pair processes 1.514M tokens per second, or about 130.8B tokens per day.
155T tokens per day divided by 130.8B gives about 1,185 chip pairs. Our estimate of the cluster is about 2,370 chips.
2,370 chips at $38 per chip-hour for 24 hours give our revenue estimate: about $2.2M per day, or $0.79B per year.
Latent Space's analysis of DeepInfer estimated about 2,400 GPUs, which agrees with our estimate. The $22M headline multiplied all 155T tokens by one average price of $0.14 per 1M. It did not use the $0.003 price for cache hits, which our calculation puts at 95% of all tokens. That is the one number from the start of this issue.
By our estimate, cache hits in Oct 2026 are 95% of DeepSeek's tokens, but they earn only 20% of the revenue. Cache-miss input is 4% of the tokens and earns 41%. All prices are off-peak prices (Oct 2026).
If you estimate the cost of an API, calculate each token type separately. What makes 95% cache hits possible? Most of the answer is not in the engine code.
The prompt cache and the model design keep DeepSeek's price low
The simple explanation for a low price is fast engine code. DeepSeek's own numbers show two other causes: the prompt cache and the model design.
The KV cache is the data that a model makes when it reads input tokens. It works like the notes that a reader keeps, so the reader does not read the same chapter again. DeepSeek keeps this data, so it does not process cache hits again. In Feb 2025, DeepSeek's cache hit rate for input was 56.3%. In Oct 2026, it is 96%. Because the output is small, our calculation gives 95% cache hits for all tokens.
The KV cache store is the prompt cache in our title. It keeps the KV cache on about 100 PiB (more than 100 million GB) of SSD. It runs on 3FS, a file system that DeepSeek released in Feb 2025. Any prefill chip can read the stored start of any prompt, so users do not need a fixed chip.
The model design also decreases the work for each token. DeepSeek-V4.1-Flash is a mixture of experts (MoE) model. An MoE model has many small expert networks and uses only a few of them for each token. The main network has 552B parameters, the learned numbers in the model. With its lookup tables, the total is 763.2B. Only 8B are active per token in prefill, and 16B in decode.
The model also makes the cache small. The DeepSeek-V4.1-Flash model card gives 890 bytes of main KV cache per token, about 1/4 of the amount for the earlier V4-Flash model. DeepSeek also says that one design choice nearly halves the total prefill work. A different model cannot get these gains from engine code.
The engine code is not the main cause of the low price. The prompt cache and the model design are the main causes.
At this scale, low-cost inference is a cache problem as much as a compute problem. Before you tune engine code, measure your cache hit rate.
A confidence score decides which draft tokens to check
The best idea in the released code does one small job well. Speculative decoding uses a small draft model to write several tokens ahead. The main model then checks all of them in one step.
DeepSeek's draft model is DSpark, and it writes 5 draft tokens for each request. A confidence head, a small extra layer, gives each draft token a score: the chance that the main model keeps it.
The scheduler is the part of DeepInfer that plans each step. It sorts all draft tokens in the batch (the requests in one step) by score. It adds tokens one at a time and stops at the best ratio of kept tokens to step time. This logic is about 300 lines in csrc/engine/mtp_varlen.hpp.
DeepInfer scores every draft token, sorts all of them and checks only the best ones.
Against a fixed check of 5 tokens, the scheduler checks about 30% fewer tokens. The main model keeps almost the same number of tokens per step: 3.20 against 3.29. DeepSeek estimates a gain of 5.4% at its production load of about 60 requests per GPU. The gain is 11.0% at 96 requests, and the result is 1.5% worse at 16.
Use confidence scores only if your batches are large.
DeepSeek released the engine and kept the service
DeepSeek's post on X said "fully open source," but the README file lists three parts that are not in the release. They are the prompt cache, the split into separate prefill and decode services, and the load balancers. A load balancer sends each request to a chip that has capacity available. DeepInfer also has no HTTP server, the part that gets requests from the web. Its only entry point is an offline function, LLM.generate.
recipes changeDeepSeek released DeepSeek Recipe and DeepInfer for prefill and decode. It kept the HTTP server, the load balancers, the KV transfer and the KV cache store, which is the prompt cache.
DeepSeek Recipe changes chat messages into tokens. The KV transfer sends the KV cache from the prefill chips to the decode chips.
The Q&A says that the kept parts connect closely to DeepSeek's internal infrastructure, so their code is "not yet fully open-sourced." The Zhihu post describes their design. DeepSeek released 3FS, but not the prompt cache that runs on 3FS.
The Q&A also lists three smaller gaps:
A kernel is a small program that runs on the chip. The public FlashMLA kernel does not have one hand-tuned optimization, so it is 5 to 10% slower in decode.
The internal draft model is a little newer.
DeepSeek says that the end-to-end performance of the public and internal versions is at most 5% apart.
Use DeepInfer as a reference design, not as a server that you can install.
Critics give four reasons to be careful with these numbers
Part of the decode chart is a model, not a measurement. The chart assumes that the main model keeps 75% of 3 draft tokens, which is 2.25 tokens. With the 1 token that the main model writes itself, this gives 3.25 tokens per step. The script multiplies measured steps per second by this assumed number. All test requests also share one prefix in the cache.
vLLM says that it already matches DeepInfer. The vLLM team's post on X reports decode speed within 6% of DeepInfer on Nvidia GB200 GPUs, because vLLM already uses DeepSeek's kernels. A competitor made this claim.
Outside measurements show lower speed. On 2026-10-22, OpenRouter's provider speeds showed a median of 111 tokens per second for DeepSeek's own API. Two other providers, Together and Baseten, showed 261 and 235. OpenRouter uses a different method and load.
No outside group audited the production numbers. Yuchen Jin said that DeepSeek did not name the chip that earns $38 per hour. Online margin estimates of 74% to 96% do not include SSD, network or staff costs.
Our view is that the price math is correct, but the speed charts show DeepSeek's best case. Do not quote the decode chart as a measured result.
The code supports Huawei chips, but the evidence uses Nvidia chips
Every TileLang kernel in DeepInfer has two versions, one for Nvidia and one for Huawei Ascend. TileLang is a language for kernels. SiliconANGLE's report on the release says that the release came one day after a US Homeland Security directive against DeepSeek products. TechNode's summary of a Bloomberg report says that DeepSeek plans at least 160,000 Ascend 950DT chips for inference.
But all published benchmarks ran on Nvidia GPUs. Deterministic output, where the same input always gives the same output, works only on Nvidia.
The release is a message to two groups. Chinese customers see a path to Huawei chips. Rival labs see real revenue per chip, which closed labs do not publish. If you plan to use Ascend chips, wait for public benchmarks.
Pick one step for this week
If you call the DeepSeek API, keep the start of each prompt the same across calls. A cache hit costs 50x less than cache-miss input.
If you build an inference engine, read
csrc/engine/mtp_varlen.hpp. Test it at your real batch size, because the result was negative at 16 requests per GPU.Before you quote a number, read
doc/inference_deployment_qa.md.
DeepSeek gave the public its engine, its kernels and its real numbers. It kept the prompt cache, which makes 96% of its input cost 50x less. One question stays open. Is the 96% hit rate a result of DeepSeek's design, or a result of the traffic of its users? If design is the cause, the most valuable code is still private.
Reply to this email with your cache hit rate and your type of traffic.