Perhaps you are looking for larger savings but I would count the RWKV line of LLM architectures in this category: https://wiki.rwkv.com/basic/architecture.html