blululu
8 hours ago
This is an interesting piece and it is kind of an outstanding puzzle, but I feel like it was also indicative of the zeitgeist. The 60s were a more optimistic time for physics. We had succeeded in Quantum Electrodynamics, Quantum Chromodynamics were underway. The standard model was being developed. To say nothing of the radical success of earlier advances applying fundamental insights to industry. We had not yet spent 40 years trying to develop a theory of quantum gravity. Or gotten into the immense computational challenges of solid state physics. We are now sifting through a huge number theories which all describe the universe reasonably well. To a reasonable approximation anything can be modeled mathematically and it can be modeled in multiple ways. But that doesn't really tell us too much. Plenty of mathematical frameworks could be described as universal function approximators. And an SVM and a Transformer NN can, in principle, both model language to an arbitrary degree of accuracy. One is computationally tractable and achieves good results. But it feels realistic to say that that neither model captures to actual structure of language as it is generated in the human brain. This is all to say that this essay might be reading too much into the success of various mathematical models.