I built a similar idea as a hobby project to see what was possible with harnesses. My idea was give a model a chat input, but outputs can be data models (sqlite), and panels that are custom react components that can be generated and mounted on the fly by the model.
The basic idea in todo list form was:
"Let's build a todo model to manage my daily tasks, then build a task list panel where i see today's todos."
It got more interesting from there as I kept building ontop of it. For example, I added data hooks (trigger something when data changes, e.g. custom code or even webhooks) and it felt like an AI native Airtable.
Everything persists and becomes the UI of the app itself.
Yeah I know people who think that chat is the last UI we'll ever need, but I really think there's so much that can be done with malleable interfaces.
This is really cool! I'm glad there are others that had the same idea!
I prototyped mine early last year and models weren't quite where they are today. I'll try to find some time to update the code and release it.
Hey OP I made something very similar called slices-ide [0]
It’s a self modifying editor with embedded agents. Honestly that description doesn’t do it justice, it was an experiment in making an architecture to use the strengths of my adhd when coding.
You can chat with agents inside the editor itself and ask them to modify ANYTHING in the IDE itself, including adding custom panels, changing the theme, rewriting logic
Hell, if they wanted to they can brick the entire editor too or remove agents entirely lol. There is a global undo / redo and timeline so as long as the timeline isn’t bricked you can step back to an earlier point in time
The editor has a visualisation panel that shows all data flowing around the ide itself
You might enjoy it, i think it’s cool and it’s very very experimental
[0] - https://github.com/con-dog/slices-demo#slice-ide---an-experi...
The demo and videos are really cool! Just a few guardrails might make this a lot more usable as an actual IDE.
In my opinion it can be more usefol if I can add it with an mcp server, so I can use with my agent and test it in real workflows to see if it can add power and some prons to my agent
I believe currently it would work if you've already set up the MCPs via Claude Code, because it's running off your local Claude Code under the hood. But I'll look into surfacing the MCPs like in the Claude app.
The project seems to be a great initiative however does it have checkpoints? I find that feature a must with agentic workflow for very obvious reasons.
By checkpoints do you mean versioning and rollback?
Yes versioning and rollback to return to a previous state
this is very easy (and fun) to do in emacs with gptel as well
you can do a lot of very neat stuff when your llm has full access to a lisp repl that can self modify itself / the ui its running in
Thanks for sharing emacs + gptel! I'll find some time to try it out
Sounds cool, can you explain a bit more how it works?
It has a chat interface, but also lets you split the window into different sub-windows that can open different interfaces, like a Jupyter notebook (with a working kernel), a code editor, or a file browser. You can also ask the agent to create custom interfaces, like a protein structure visualizer, or an SQLite viewer, or just a simple place to manage your citations.
Under the hood it's running on your local Claude Code instance, I'll probably be adding support for other APIs through Open Router soon.
letting the agent build its own panes is a neat idea. do people keep the layouts it creates?
Yup, the layouts and panes are persistent and also have state.
Hi HN, I built Panel because research work means switching all the time between a chat with an agent, the papers I'm reading, the notebook I'm running and the data files I'm looking at.
Panel is a local web app with dockable panes made with dockview. You get chat, files, PDFs, markdown and Jupyter notebooks.
The agent can also create custom panes on request, like a protein structure viewer or an SQLite browser.
There's also an early "Module" system. Modules are like Skills, but with typed inputs, outputs and intermediates, so a long job shows its progress and its results can feed the next step. Right now the working one is a literature review, with an early version for hypothesis generation.
How it's built: a Vue + dockview frontend and a Python (FastAPI) sidecar that drives the agent backends. Everything runs on your machine.
Honest limits for now:
- Full support is Claude Code only, so you need it installed and signed in. Codex and other harnesses are not supported yet.
- Still very rough around the edges, I've been adding to it as I use it myself. Feedback welcome!
MIT licensed.
This is cool! We have a similar approach in Nimbalyst... agent makes extensions that are human editable and agent editable ... includes Jupyter, markdown, csv, excalidraw, code, pdfs, browser. Its so nice not to have to app switch all the time. MIT as well. Maybe we should join forces?
I’m not familiar with this concept. So the purpose is to have a chat llm spawn new independent activity windows all in the same chat context?
Actually it's something like a researcher-focused app that allows you to open Jupyter notebooks, code snippets, PDFs, right next to the chat window pane.
Let's say you're working on protein design and you want to view the 3D structure of the protein, you can have the agent build a pane for that in the app, and you can ask the agent to help customize it as well.
Look into openrouter so you aren’t tied to just Claude or codex
I will! Thanks for the rec