I’m working in biotech. I think creating “curated” materialized views with dbt + datawarehouse or duck db + using webgl with d3 will be best long term solution https://blog.scottlogic.com/2020/05/01/rendering-one-million... .

Matplotlib works well for static plots. Altair and others freeze at around 4000 data points, which is crazy. Streamlit + matplotlib is impossible to maintain but is quick to get up and running.

I think https://github.com/uwdata/mosaic is really promising here. See the example https://idl.uw.edu/mosaic/examples/linear-regression.html where the user can recalculate a linear regression based on their selection.

You'd still need to implement any custom selection widgets, data transformations (like other statistical tests) etc. still missing, but i like the technical design to build on top off. It uses https://github.com/observablehq/plot under the hood, which aims to have just as flexible a grammar as ggplot (already quite capable) but with interactive features (built by the creator of d3 and uses it under its hood).

Igor Pro is nice but the underlying language which grew out of a set of macros is pretty rough to work with (by default works via side effects and mutation of global state, though there are ways to contain them). However, I know people who have built some nice GUIs on top of it. While it does have HDF5 I/O integration, there's no memory mapping and it will choke on larger data sets from what I recall.

I've heard promising things about Makie [1] in Julia; there is also capability to build a dashboard called Genie [2] (and a commercial dashboard builder [3]) though not sure if Makie and Genie play nicely together at the moment.

[1] https://docs.makie.org/ [2] https://genieframework.com/ [3] https://info.juliahub.com/blog/create-low-code-apps-on-julia...

I agree with cactusfrog that d3 is a step in the right direction for dataviz. It really is a swamp out there, with twenty or so different ways to do similar things but usually not quite what you want (or at least not as easy as you want it to be)! I've been researching dataviz out of curiosity and annoyance with the current state of dataviz software for a while before this post popped up and would be interested in integrating a lot of the past decade-or-so of open-source work in this space to a tool for full stack data viz. I'd love to work with as many engineers who do data viz as I can! You can get in touch with me via email at puterproblems [at] proton.me if you're interested (and to find out more about me as I'm aware this is a brand new account :D).

Doesn't R require coding too? And my recollection of Igor Pro was from long ago (on a 68k Mac), but it also required coding in its bespoke scripting language. In fact I walked away from it for that exact reason... I wasn't going to spend brain cells on somebody's proprietary language.

I think that for things like dashboards, we're still stuck between "code" and "no code" tools. I don't know of a happy medium.

Vega/Altair is a declarative grammar, so I wouldn't dump it in with the others. It's also convenient because it reduces to json (easy to store) and has TypeScript libraries for native presentation in a browser.

Additionally, Seaborn (https://seaborn.pydata.org/) is a great mention for people that want to use Matplotlib with better default aesthetics, amongst other conveniences:

"Seaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics."

SciencePlots is another option: https://github.com/garrettj403/SciencePlots

It probably depends on your interests but I'm excited about the efforts using GPU rendering for interactivity (it's related to vispy but I forget the latest status of whether vispy is the current focus... I think the last I remember is someone was prototyping a vulkan-based system).

Why on earth would you want to do exploratory data analysis in a compiled language

Is ggplot2 still using hyphens for minus signs? A pet peeve of mine.

plotnine is a really good grammar of graphics at this point.

I second this! Nicolas and I have been working hard lately to try to advance the field of scientific visualization in Python — stay tuned!

Very nice guy too! Met him at a workshop recently and got a hardcopy of the book

Looks good, thanks

I find the same thing about the official docs. I can't really put my finger on the reason, except that it might be due to the vastness of MPL and the need for the docs to auto-update with each revision cycle.

When I do find useful docs, they're usually in the form of an example that I can use as a starting point. Referring to the docs comes later if needed. The consequence is that I probably use only 1% of MPL's capabilities, but that's already more than I could ever have imagined.

I've only been using Google Copilot for a few weeks, so it's too soon to know if that's how I'll deal with the situation in the longer term.

TikZ has a steep learning curve and felt so convoluted when I tried to use it.

How do you use those outside of Latex? Lots of figures are coming from Matlab or Python, where Latex is a non-starter. Is there a cookbook to pollinate?

You can use matplotlib's pgf backend for direct use in latex. That said, not sure I agree that's better for scientific viz. Perhaps in the past. Also tikz/pgf is much more fiddly and has it's limitations too (imho).

Matplotlib is very easy to customize via “style sheets”, so you can make it match any style you want. Note that you can even turn on the “usetex” option and provide a preamble to compile all text/equations via TeX.

Interactivity is not desirable in all situations. For static figures, matplotlib is still king.

Plotly is only useful if you need to display low hundreds of data points. For anything more, it becomes so slow it's quite literally unusable.

Can plotly generate publication ready vector graphics with Latex texts?

plotly has had ten years to take over the world?