An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU