bfeynman
12 hours ago
probably all AI slop but I find it hilarious in the blog post they actually posture like they would know how to fine tune a model to sound like them given that what they actually did is something that you could one shot with claude if you knew what you were doing.
dandinu
34 minutes ago
The method I used in the article is real and is pretty standard. Also, I do a decent amount of distillation and tinkering with weights for work, so I can assure you I did try that before resorting to good 'ol RAG.
Overall, even with a finetuning-as-a-serice like Tinker (the one from Thinking Machines) which is pretty cheap, the economics didn't work out that well.
Also, you probably one-shot this with Claude, I agree. But, you need to have an expensive Max subscription, which not everyone is willing to shell out 200 bucks for, just to have some weekend fun.
philipswood
3 hours ago
Fine tuning a model isn't that hard and the tradeoff he described is real.
I was on the fine tuning team of a multi-team hackathon to make a specialized chatbot once a few years ago and despite working technically well our output had very little impact on end to end output.