measuring that part is hard IMHO (without the whole context)

I think in the same vein, the first token generated is also not as important as the final answer, without the full context it generally gives no signal.

To measure the intelligence of a string of text, the context in which it has been generated is far more important.

I do think coming up with better ways to measure intelligence in any part of the answer O(10 tokens), O(20 tokens)... would be useful