Local: a GPU in every city.
Private inference keeps the plaintext on the customer’s machine and trades a round trip with the GPU every layer. Distance to the GPU is the whole latency budget.
GPUs sit in the regions the centralized provider built. For most customers the nearest is about 20 milliseconds away: roughly one second per token on a private lane. Usable for batch work, not for conversation.
Any GPU anywhere can join and serve the nearest customer, the way ride-hailing dispatches the nearest car. A miner in the same city is about 3 milliseconds away: interactive speed. More miners means a nearer GPU for everyone, and more competition for every job.
A centralized provider’s capacity is capital expenditure sited where the provider built. It cannot put a GPU in every city, and it cannot let strangers serve its customers, because its customers trust it rather than the strangers. Permissionless supply is the only way to be everywhere.
Blindproof