We haven't done concrete measurements yet, but my intuition is that the quality of Lacuna proposals is a function of density of topics / research directions. Lacuna is most dense around AI topics; for neighboring ones (e.g. bio, physics), I'd join you in tempering expectations.

The one thing I could recommend however is rigging up the mcp (https://github.com/tiptreesystems/lacuna-research-mcp) to your code agent and trawling the database for ideas worth thinking about.

> I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.

Yes, I think this can work. We use a similar idea to categorize what papers have been influential or foundational -- look at the ones that have changed the trajectory (in time) of a research direction significantly.

> big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach

I feel like when it comes to "Alien Spaces" or new approaches, there's no other way than to babble a lot of stuff that doesn't move the needle much but then leads through the implicit to language and communication mechanisms to a spike out of the local minimum. Didn't the ancient guard have hundreds more notes than ended up in their useful collections?

Nowadays you have so much research that nobody really cares to just ramble with attempts and approaches ... and instead skims the work that others certainly tried already ... distancing themselves from the possibility of new approaches ...

Thanks for checking it out! We're working on a collaborative discussion / editing feature mediated by Althea (our AI agent). That should provide a way for domain experts like yourself to contribute to improving it. Hopefully we'll have that soon!

The problem of course, is how to measure the "movement of the needle." It might help to weight papers and research in the corpus based on some notion of "significance". One metric I know people have used for this is based on citations, on the reasonable premise that influential work gets cited much more. IIRC the original PageRank paper was inspired by work on citation analysis as well.

> It will never, ever, come up with Velcro. If a concept does not exist on the Internet, or the corpus of training material, LLM’s will not invent it.

So this guy made up an imaginary test, provided a completely unjustified result to it- which casually confirms his ideas- and he's so proud he was right! What a good test he invented!

I think if an LLM had read any discussions on how best to remove burrs from pet coats then it might eventually get to Velcro - I believe the inventor of Velcro got the idea from burrs?

ya - I think my intuitions around velcro are firmly intertwined with my meat-space actuators and perceptual equipment.

In other words, I experience high-resolution multi-modal qualia all day and it's really noisy and I think a lot of new ideas and concepts come from obsessively exploring a small subset of those details. Maybe that's just me posthoc rationalizing my phd, though.

That's a bold claim. It was almost certainly true for models in 2023, when it was written. I think that the recent spate of mathematical discoveries at least significantly narrow the implications for even current models (one can argue that they are only re-mixing existing mathematical concepts to solve problems that humans initially generated, but in my opinion, at some point you are really narrowing what it means for an idea to be "novel"). And I would seriously consider taking odds against it for models within the next 3 years (since that's how long it's been since it was published).

We are basically already at the point where if what current frontier models can do isn't truly "novel", then the vast majority of humans have never had a novel idea in their lives. My three years from now guess is that if whatever it is that frontier models can do at that point isn't "novel", then no human ever will have had a novel idea.

LLMs assign non-zero probability to every output (unless you you sampling modifiers like top-k), so if you sampled enough times the model would definitely come up with velcro. It's the same as the proverbial monkey with a typewriter.

The real question is whether it would happen in a practical timeframe.

This is the same thing as "but AI can only repeat what it already saw" but the goalpost moved again.

It's ironic looking at a recent post on the same blog and immediately noticing the claudisms https://blog.karliner.net/posts/thingstudio-common-ground/

edit: clarification: the content itself is fine, in fact I'm curious!

Thank you!

One part of the explanation is that we focus primarily on AI research, and a substantial chunk of physicists gravitate (heh) towards AI research, so we end up capturing more of their work.

Another part of the explanation could be that (AI +) physics papers are just easier to access, so we end up processing more of those.

Great question - the "universe" of venues we address is centered on peer-reviewed machine learning like ICML, ICLR, NeurIPS, COLM because (a) we have expertise in this area and so can therefore judge the quality and (b) it's all open access.

Therefore, the further you get away from the topic of ML, the less dense our coverage. Frankly, it's a tough to editorial choice to determine which research directions and therefore which proposals to include. ML is quite multi-disciplinary. And people who have a few papers in ML but many in (e.g.) medicine get upset when their profile is sparse. But we can't possibly produce good proposals in medicine due to its tendency to be closed-access.

Thanks! I wish we could take full credit for the paper name, but it's based on a line of work from James Evans (Chicago-based computational social scientist) investigating "Alien artificial intelligences" https://x.com/profjamesevans/highlights

Yes, vocab drift matters a lot. Most of the pipeline is basically compressing out any of the weird vocab/language stuff, so it doesn't have numbers and proper nouns and is in a sufficiently abstract concept space to cluster nicely. We use a density-based clustering method (HDBSCAN) which when you set min_samples=1 works as an online clustering (i.e., incrementally adding new things is the same as if you re-ran the clustering in batch mode).

It cost ballpark $250k

Oof - it's not like goodfire - we have quantitative results and a clearly defined method!

But yes, we might be bad at marketing. Internally the term we use more often is "concept elements".

The intuition is that if you take a complex treatment of a topic (like an academic paper), you should be able to break it down into some more abstract self-standing ideas. Then, critically, these units are composable and easily embeddable (i.e. they don't have proper names and numbers).

Here's a couple examples of these idea atoms pulled from our paper https://lacuna.tiptreesystems.com/work/the-alien-space-of-sc...: "Scientific ideation often conflates the logical coherence of a research direction with its likelihood of being discovered by the current community. This oversight makes it difficult to identify research paths that are scientifically sound yet unlikely to be discovered due to the existing distribution of expertise and collaboration networks."

Another example: "A generative model estimates the plausibility of concept combinations by treating conceptual units as tokens and calculating their length-normalized log-likelihood. By training on random orderings of these units, the model learns to identify which components are likely to co-occur in a valid research contribution regardless of their sequence."

> I made it to the first figure in the paper before I raised an eyebrow at this term.

I came up with that name, and fair eyebrow. Think of an "atom" as a unit of a working vocabulary (a "token"), not a Platonic form. It's a way of compressing the corpus (paraphrases of the same idea across papers get merged into one symbol, a bit like a tokenizer's vocabulary).

We also talk about some limitations of this framing in page 11 (towards the top).

This site crashed my browser although I also wonder what exactly constitutes those arbitrarily picked "idea atom" sets.

I'd say science is somewhat fractal in this sense. A research area contains topics, those topics have sub-topics, those sub-topics have competing lines that "talk" to each other, etc. So if it works in core ML that means it's working for RL and deep learning and computer vision and NLP -- which is kind of cross-domain.

That said - we believe the method does need to have sufficient data to model the conceptual density. So it'll not work in closed-science areas.

One other thing I will share: we have been working on a way to enable verified academics with sufficient topical domain expertise to engage in small group discussions that update these documents. You can think of it like Althea (our agent) being an area chair and moderating a discussion to determine how to update pages. The reason we think this will work is that you can have an anonymous discussion with the AI system (and be blunt), but we can weigh your expertise when updating the documents.

Sure! My personal favorite I found at COLM a couple weeks ago: https://lacuna.tiptreesystems.com/hypothesis/passing-uncerta...

The idea is to allocate academic reviewer time based on the estimated quality of a submitted paper. But critically, you should maintain uncertainty bounds in your estimate. It's kind of like an UCB (upper confidence bound) bandit approach but inspired by cascaded speech translation (where you really should not throw away information at any point in your pipeline).

We have generated a bunch here: https://lacuna.tiptreesystems.com/explore?view=hypotheses

Recommendation: rig up the mcp and have claude or codex trawl the corpus for the kinds of ideas you might care about.

``` claude mcp add --scope user lacuna-research -- uvx lacuna-research-mcp ```

or

``` codex mcp add lacuna-research -- uvx lacuna-research-mcp ```

[dead]

  > Bio is one of our sparsest regions 

It does seem like one of the best uses of AI -- digging through existing literature to make novel connections/ideas.

Curious if you've heard of Don Swanson's "Public Knowledge Problem" paper from the 1980's, which was roughly this line of thought?

I think the way computers developed is kind of an interesting example of this in the real world. Babbage basically realized the idea of a modern computer, but tried to build a too-complicated 'supercomputer' (40 decimal digit precision!) because he didn't know how incredibly useful a much simpler, more limited computer could be. We had to wait another 75 years and spend an incredible amount of money to develop relay and vacuum-tube supercomputers, before realizing that the idea could be scaled down, and something like a PDP-8 / PLC / microcontroller was incredibly useful for industrial control and, with sufficient cleverness, even do a lot of scientific computing.

I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.

Didn't that sort of happen to Heisenberg and Bohr? Einstein refused to accept it though he knew they were right (cuz the math worked).