The Htop for LLM Inference

4 pointsposted 12 hours ago
by helasaoudi

2 Comments

rbanffy

11 hours ago

This is super cool. Would be nice if it could be adapted to other architectures besides Nvidia. How is it getting its data?

BTW, exporting the data to Prometheus would be a killer feature.