> "Second, there is no guarantee that a given neural network can be approximated by DISCOVER"

They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.

The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.

So, this paper will not necessarily lead to any computation savings.

But I am at page 6. ;)

Their representation is a key-value store that they embed via a tensor product to reconstruct the vectors the model produces. The catch is that it works best if the key-value mapping is task-specific. So if you already know how to solve the task the model is performing and can transform the input into a data structure that makes it easy to just read off the answer, you can probably also recover the model output.

That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.

The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.

It's a fascinating emergent behaviour but also one that could have been predicted?

I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?

I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.

> Is evaluating these closed-form representations more computationally efficient?

No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.

Inference is just tokens transformed through a fixed crystalline structure, no? You already could put that on a chip. There's no particular reason it couldn't be represented as some set of symbolic equations instead of a layered process... it's just another kind of quantization.

When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage required in exchange for more generalized computing, lessening the need for so much GPU in a lot of cases.

I don't see why a model couldn't be represented that way. After all, if you just polled the output of a model, you could evolve genetic algorithms to predict it with fairly high accuracy in a limited domain. Take that out to the Nth degree and you're basically just unspooling the model into a giant set of equations.

So, basically you're saying: I have only skimmed the article, similar approaches had issues in the past, and my own method from my forthcoming paper is better.

I mean, no disrespect, but that's the core of your argument, yes?

> diminishing these things as 'Next token predictors' seems absurdly reductive.

This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.

Regardless, if all you want is a next-token selector, save your money and roll a die.

it is not. in this case these researchers have lost their way, with how symbolics entered the conversation. to remind, it is via wanting to prove that ai vs traditional program is understanding deeper. well guess what, ai is not understanding your input better, its your mind playing tricks with language. For analogy, digital world is not real in a physical sense. if output is food, and input is ingredients, then symbolic programs care about macro slicing dicing stacking them, while ai is micro level spice & heat that doesn't especially conscious to the ingredients, just that chemistry appears magical. hey! we eat our information food tho.

as long as it is honest, everything is ok.

I haven't read this in depth yet, though I plan to. If this general line of research is interesting to you, I'd recommend checking out some of the lines of research it touches upon--they're really rich and fascinating, and some are pretty approachable mathematically even if ML research papers aren't usually your thing. The related works section here seems pretty well stocked, but mechanistic interpretability is a pretty interesting peephole into this general vein: https://transformer-circuits.pub/

distillation is now illegal tho

yep

Imagine a box of balls. They have size, weight, colour, density… etc. These properties, each a measure, are dimensions and they are orthogonal to each other. Taken together are multi-dimensional.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

>The human mind cannot comprehend the capacity of massively multidimensional space.

That is why the scam works, because investors are humans...

Post the actual articles that you have in mind. What I've seen is vague slop.

A good example is https://richardaragon.substack.com/p/a-universal-prime-funct... describing a supposed "universal prime function" which is simply a finite approximation using a sum of 50 sines (each applied to a linear term plus a sine-log offset). The 53 parameters are fitted to the first 10^3 or so prime numbers. This is followed by the *absolutely ridiculous* claim that if the function approximates the first 10^3 primes well, it must also fit the remaining prime numbers (of which there are infinitely more than 10^3000000000) equally well.

Then they suggest "A formal proof connecting this function to the RH would involve the following steps" using this great discovery: "1. Correspondence with the Explicit Formula: Demonstrate that the oscillatory correction term in our function corresponds to the sum over zeta zeros in the explicit formula for ψ(x) or π(x). 2. Error Bound: Prove that the error in the prime counting function derived from our function is bounded by O(√x log x). 3. Contradiction: Show that if any non-trivial zero were to lie off the critical line ℜ(s) = 1/2, the error would exceed the bound, leading to a contradiction."

This isn't even midwit math.

It's the kind of naive ideas I had as a high schooler, who was good at high school math and who knew how to code functions and plots in Mathematica, but who had no understanding of higher math. This kind of naive approach to RH signals that one doesn't even understand the problem.

Cypher says that, and he's clearly referring to a blonde, a brunette, and a redhead.

People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.

yep

I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.