How Apple fits a LLM on iPhone with surgical efficiency

9 pointsposted 15 hours ago
by lroe
(youtube.com)

1 Comments

paxy

12 hours ago

TL;DR

- Quantization

- KV caching

- LoRA

- Sparse MoE model

- Dedicated AI chip

- Offload to cloud

Nothing really groundbreaking.