29 minutes ago
"For AI agents". I understand why everything needs to be marketed in this way, but it's just ... an easy-to-generate language for expressing charts. That's impressive! That's useful.
10 minutes ago
Isn't this literally made for AI agents to be accessed through an MCP server? Seems to me the AI agents part of the marketing is important.
an hour ago
Is there a specific explanation about how this is better or different than vega itself? https://vega.github.io/vega/docs/specification/
My understanding is that Vega was already an expressive DSL for visualizations and its probably already well spread through LLM training data.
an hour ago
Vega was a high-level language in the past for human, but now they can be a bit too low-level for AI agents! AI agents have to write a lot of low-level params just to make charts looking good, and the result is that programs are hard to write reliably for AI agents.
Flint is a higher-level abstraction, with simpler much shorter spec, and the compiler derives low-level decisions so that charts are looking good.
So: flint lets agent write short program that achieving good looking charts that had to be done with lengthy program in the past.
14 minutes ago
I'm sorry, but as someone who creates data visualisation as a big part of my job, I wouldn't say the charts on the website look good. Most aren't awful either, but by no means are they an improvement over what I'd get by telling any coding agent to make a chart with Vega-Lite or Observable Plot, and probably worse than if I had some decent instructions/skills.
I don't quite get what the goal of this is other than abstracting away a little bit of the complexity at the expense of flexibility. To me, the promise of LLMs is the opposite, I can get flexibility and customisation without the cost of complexity.
a minute ago
Some composite charts are quite annoying to be generated well (like bullet, waterfall etc), their Vega-Lite equivalent can be quite long if just starting from scratch.
The intention here is that Flint is a simpler abstraction to get basic setups right and any followup edits can be done on top of the first compiled outputs (thus not limiting expressiveness). It also makes it easier for user to manipulate (like swapping axes, click to change something, which can be very hard if LLM generates a complex chart spec upfront).
But for many basic stuff your intuition is completely right.
42 minutes ago
> simple chart specs can be reliable, but generated charts are often of low quality due to reliance on system defaults; - complex chart specs with explicit details can produce good-looking charts, but they are verbose and agents can struggle with reliability
N of only a few of us working on an analytics agent, I don't think we've been finding this to be the case. We've been impressed with just how good LLMs (even smaller open weight models) are at using Python and R for visualization. Often any shortcomings go away if we iterate a bit to about ambiguity. Are there any threads of research that could better support this claim or highlight where issues might be?
4 minutes ago
we are considering also reliability, interactivity besides expressiveness. Simpler spec with good expressiveness comes handy when you want the agent to be reliably for non-expert users and with small models.
25 minutes ago
A simpler spec can be used by a simpler agent. So, maybe that's the use-case here... use by smaller/cheaper agents that run in parallel as opposed to large models running one visualization at a time.
Or at least, maybe that's the idea?
IME, Claude and ChatGPT do just fine generating ggplot models, but extensive customization can get a bit hairy.
30 minutes ago
There’s an emerging pattern in agentic systems and this project is a great example.
A deterministic layer like a compiler or generator of code with some kind of IR that the LLM generates and feeds it with.
I feel we will be seeing this more and more in the near future.
19 minutes ago
A well designed intermediary enables both validation and control over the output independent of the AI. This changes the interaction model between human and AI from delegation to collaboration.
27 minutes ago
When I first saw Claude generating PPT decks by writing Python code instead of making the XML directly, it was sort of an "aha moment" for me. This seems to be the path for many things. It also feels slightly limiting, and like a hack LONG term, but 100% correct approach for a while.
2 hours ago
> requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler
Isnt graphviz there for the same reason?
Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!
an hour ago
In fact, Json as a common language for human in visualization has been around for a while! The benefit of declarative grammar is that users can effective manipulate specs through UI (drag and drop, clicks).
Btw, Flint is intentionally designed to allow agent skip low-level params like scale, axe, zero, step size etc (which are extremely crucial for "GOOD-looking") and they are dynamically optimized by the compiler. So AI agents can have a easier time.
an hour ago
> Json as a common language for human in visualization has been around for a while
Plant, Mermaid, Graphviz are all declarative textual representations designed for human authoring, JSON is made for tools. Its not a criticism just a statement that if interop across agent and human was intended this is not the simplest option.
an hour ago
right, in fact many small models still struggle with following json, some new forms are also needed
2 hours ago
Project page: https://microsoft.github.io/flint-chart/
25 minutes ago
> Flint is built by Microsoft Research in collaboration with the IDEAS Lab, Renmin University of China.
Interesting to see a Chinese University collab for once.
an hour ago
This is cool to see from a research team. A few weeks ago I was exploring a similar idea with ntcharts, where a user or LLM can specify a chart in a Golang or JSON object...
and then that spec would be rendered either to a Bubble TUI via NTCharts or to HTML/SVG via ECharts. That Echarts HTML could be naturally served by a Golang http service.
But Flint goes much deeper with semantic layers and settings optimizations. Perhaps a NTChart, or whatever terminal chart, could be a rendering target? I'll add it to the list to explore...
https://github.com/NimbleMarkets/ntcharts/blob/spec/spec/REA...
an hour ago
This is fun! We started thinking it would just be an engineering task in the beginning, but doing a solid intermediate language turned out to be a research project (the paper will be out soon).
Also, I find NTChart very fun, maybe we should add NT chart to the list of compilation backend for Flint so it works in the library. Putting a reminder here: https://github.com/microsoft/flint-chart/issues/45
an hour ago
This is pretty crazy, literally built something almost exactly like this for a project I'm working on (a local-first AI agent that does work on folders while you sleep). Basically going from JSON "Lego blocks" to full reports (including charting, though a subset of what Flint offers). And with post-generation validation and retry steps.
Functions extremely well and the result is a very clear (and consitent) human-readable "output layer." Cool idea, fun to see people converging on similar concepts in the space.
an hour ago
That's awesome!
I find that besides training better models, designing new language for agents is also a super viable paths to improve their performance!
2 hours ago
The charts are very nice, and I think the visualisation layer for LLMs is a very interesting problem.
I’ve been building https://smalldocs.org for this exact reason. It’s an office suite for AI agents - but my main use case is giving a cli based LLM the canvas to express itself - charts, mermaid diagrams, etc. I’ve extended it a bit further to be a format for all types of work so the agent can embed slides and spreadsheets in a document.
Sample document: https://smalldocs.org/blogs/what-is-a-smalldoc
2 hours ago
> mermaid diagrams
I'm terrible at diagrams, so I gave GPT very generic descriptions of one of our project, to convert in to that mermaid style, then for Lucid I pasted it in there, and had a visualization of what I needed. Worked out nicely.
an hour ago
Yep I find them to be very useful for explaining a system.
They can do a lot of cool things! Mermaid gallery here: https://smalldocs.org/s/xZrc-lNW1kbXpoIuU3l_ky#k=-0ehGe2B-hR...
an hour ago
interesting how you don't discuss literally anything about the project actually posted and spam your thing. Not pointing you out, seen many other comments like this on HN but always felt a bit weird about them
13 minutes ago
Fair point. Sorry
18 minutes ago
an hour ago
Forget AI agents, this DSL is better even for humans. Cool project!
an hour ago
For AI agents and Humans :)
2 hours ago
https://github.com/scicloj/kindly
but enterprise
2 hours ago
It compiles into Echarts, but echarts already has a JSON co figuration spec
2 hours ago
It's more like a simple high-level spec to make it easier. The idea is that you don't have to fill position / axes details just to make the chart work. The compiler has a bit of magic of using semantic types to optimize what parameters will be set in ECharts.
In some composite chart examples, the good-looking echart spec is like 5x longer than the simple Flint one!