📰 2026-06-18 · 每日技术速递
今日精选 6 篇深度技术文章,覆盖 MoE 量化、AI 加速器评测、边缘推理部署、Transformer 算子加速、解耦推理博弈分析、LLM 在线策略蒸馏。
1. MoE 多模态大模型混合精度量化:MODE 框架
来源:arXiv · cs.LG
链接:https://arxiv.org/abs/2606.17118
标签:MoE量化 · 混合精度 · 多模态LLM · 内存压缩 · PTQ
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods, expert-level mixed-precision quantization has proven effective for MoE-LLMs, yet suffers notable degradation on MoE-MLLMs due to two overlooked biases in expert importa
核心要点:
- 针对 MoE-MLLM 的 GPU 内存瓶颈,提出模态分解的专家级混合精度量化策略
- 将视觉 token 与语言 token 分通路处理,为不同模态分配不同比特宽度
- 在保持精度的前提下显著降低显存占用,推动 MoE 模型低成本部署
2. 新型 AI 加速器上的 LLM 推理评测:Prefill/Decode 感知框架
来源:arXiv · cs.AR
链接:https://arxiv.org/abs/2606.17104
标签:LLM推理 · AI加速器 · Prefill · Decode · 系统评测
As large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings, inference efficiency has become a central systems challenge. While GPUs dominate current deployments, a growing number of AI accelerators claim advantages for LLM inference, yet it remains unclear under which conditions such accelerators outperform GP
核心要点:
- 系统性评测 GPU 以外新型 AI 加速器在 LLM 推理中的表现
- 分离 Prefill(计算密集)与 Decode(带宽密集)阶段的评测指标
- 为硬件选型提供量化依据,揭示不同加速器的适用场景差异
3. 从压缩到部署:极低功耗嵌入式设备上的实时 FastGRNN
来源:arXiv · cs.AR
链接:https://arxiv.org/abs/2606.17249
标签:模型压缩 · 边缘推理 · FastGRNN · 能效优化 · 嵌入式部署
The dominant trajectory of modern machine learning has been to scale up: larger models, larger accelerators, larger memory budgets. Yet a multi-year global semiconductor supply constraint and the growing energy and carbon cost of always-online inference expose the fragility of this trajectory and motivate the opposite direction: refactoring AI and
核心要点:
- 在半导体供应约束和碳成本压力下,探索”缩减而非扩大”的 ML 路径
- FastGRNN 在超低资源约束设备(微控制器级别)上实现实时能效推理
- 完整覆盖从模型压缩到硬件部署的全链路优化方案
4. MIVE:Softmax/LayerNorm/RMSNorm 加速的极简整数向量引擎
来源:arXiv · cs.AR
链接:https://arxiv.org/abs/2606.17781
标签:硬件加速器 · 整数运算 · Softmax · LayerNorm · LLM推理
The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints. Although matrix multiplications dominate the overall computational workload, non-linear vector normalization operations, such as LayerNorm, RMSNorm and Softmax can become
核心要点:
- 矩阵乘法之外,Softmax/LayerNorm 等非线性算子成为 Transformer 推理新瓶颈
- MIVE 使用纯整数向量引擎高效支持上述算子,规避浮点硬件开销
- 面积和功耗极小,可作为专用加速器的轻量协处理模块
5. 解耦推理中的混乱代价:Prefill/Decode 分离的博弈论分析
来源:arXiv · cs.AR
链接:https://arxiv.org/abs/2606.17081
标签:解耦推理 · 分布式推理 · 博弈论 · GPU调度 · Prefill/Decode
Disaggregated inference architectures physically separate prefill and decode phases onto distinct GPU pools, creating competing “agents” that share a fixed hardware budget. We provide, to our knowledge, the first formal game-theoretic analysis of this architecture, using NVIDIA Dynamo as a concrete case study. We model disaggregated serving as thre
核心要点:
- 首次用博弈论形式化分析 Prefill/Decode 分离架构中的资源竞争问题
- 将 prefill 和 decode 池建模为竞争固定 GPU budget 的”博弈方”
- 量化”混乱代价”(Price of Anarchy),为集群调度策略提供理论指导
6. PowerOPD:有界幂变换稳定 LLM 在线策略蒸馏
来源:arXiv · cs.LG
链接:https://arxiv.org/abs/2606.17199
标签:知识蒸馏 · 在线策略 · LLM训练 · KL散度 · 训练稳定性
Standard on-policy distillation (OPD) for large language models estimates the reverse-KL objective using student-sampled tokens, yielding an unbiased single-sample Monte Carlo estimator that avoids vocabulary-wide computation. However, we show that this estimator suffers from severe training pathologies in practice: sample inefficiency, unstable ge
核心要点:
- 标准 OPD 的单样本蒙特卡洛估计存在严重训练病态:梯度爆炸/消失
- 引入有界幂变换对 reverse-KL 目标进行正则化,从根本上消除数值不稳定
- 在多个 LLM 蒸馏基准上显著提升训练稳定性和最终模型质量
📊 今日速览
| # | 标题摘要 | 来源 | 方向 |
|---|---|---|---|
| 1 | MoE 混合精度量化 MODE | arXiv cs.LG | 量化/压缩 |
| 2 | LLM 推理 AI 加速器评测框架 | arXiv cs.AR | 推理系统 |
| 3 | FastGRNN 极低功耗实时部署 | arXiv cs.AR | 边缘推理 |
| 4 | MIVE 整数向量引擎加速 | arXiv cs.AR | 硬件加速 |
| 5 | 解耦推理的博弈论分析 | arXiv cs.AR | 分布式推理 |
| 6 | PowerOPD 稳定蒸馏训练 | arXiv cs.LG | 模型蒸馏 |
自动生成 · 2026-06-18 · jeffinchen daily tech reading list
评论