memory
How is Apple able to fit such a model on an iPhone without any adverse impact on the device?
What else should be on the list?
You can probably guess what he blames for the excessive workload
In the previous post on 500 virtual linux devices on ARM64 I hinted that I expected serious improvements if we got KVM working. Well. We’re there. Let’s see wha…
Recurrent Neural Networks (RNNs) are a class of neural networks designed to handle sequential data. Unlike standa
Reading Jia-Wei Hong and H. T. Kung, I/O Complexity: The Red-Blue Pebble Game , STOC 1981, pp. 326–333. Original paper Imagine an infinitely fast processor whos…
Reading Grzegorz Kwasniewski et al., Red-blue pebbling revisited: near optimal parallel matrix-matrix multiplication (2019, arXiv v3). Original paper Suppose a …
Reading Barna Saha and Christopher Ye, The I/O Complexity of Attention, or How Optimal is Flash Attention? (2024). Original paper We often measure an algorithm’…
大模型推理的核心计算几乎全是矩阵运算(GEMM / GEMV),尤其是 Attention 和 MoE/FFN。Thor 的这些加速器不是“多了几个核”,而是专门针对“大矩阵 + 频繁数据搬运 + 低精度格式”这条路径做的硬件优化。GB10(sm_121)在这些路径上支持不完整,所以同样代码在 Thor 上能走完全不同…





