Release of Polars 2.0

249 pointsposted 5 hours ago
by simicd
(pola.rs)

40 Comments

gozzoo

3 hours ago

I'm not following the trends closely, but has Polars become a full replacement for Pandas? Are there use cases where one is better suited than the other?

desipenguin

3 hours ago

From recent Python Bytes podcast (https://pythonbytes.fm/episodes/show/496/a-lake-house-in-sea...)

> 1 Billion Row Challenge benchmark: Pandas took 4m28s vs. Polars 5.04s and DuckDB 5.19s — DuckDB also used 19x less memory

Python Vs Rust : In terms for speed - No comparison

(The above episode transcript has a link to blog post titled "Pandas should go extinct" )

esco2292

3 hours ago

Polars is effectively a full replacement for Pandas for 99.9% of all cases. The only exception I'm really aware of is if you're working with geospatial data, as there isn't yet a "Geopolars" equivalent of the commonly used "Geopandas". However, Geopolars is still in active development and should eventually be production ready.

adeptima

3 hours ago

it’s on the correct path. i use rust for geo spatial and the gap with c, c++ closing rapidly or negligible in most cases

from https://github.com/pola-rs/geopolars/tree/main

Comparison with GeoPandas

Imitation is the sincerest form of flattery! GeoPandas — and its underlying libraries of shapely and GEOS — is an incredible production-ready tool.

GeoPolars is nowhere near the functionality or stability of GeoPandas, but competition is good and, due to its pure-Rust core, GeoPolars will be much easier to use in WebAssembly.

niltecedu

2 hours ago

Yes and no, its not replacing the reason why pandas was popular ie data scientists, but it a full replacement of its pipeline usage, And I would saw also beating out spark

392

3 hours ago

my understanding is Polars is faster, scales better without using external solutions, better API, +Rust. Pandas wins if you want to use what the vast majority of folks are using and have used in the past. Probably has a more complete set of helpers / recipes for the little things you bump into when using it thoroughly, but in the age of LLMs, I think that's minor.

vovavili

3 hours ago

>Pandas wins if you want to use what the vast majority of folks are using

Vast majority of skilled developers are now using Polars, unless they are constrained by lack of Narwhals support in their third-party library of choice (e.g. Great Expectations, SHAP). That's the more important trend to follow.

derriz

2 hours ago

It’s the API that gave me the push to leave Pandas. 10 or more years of occasional Pandas use and I still had to google for any non-trivial queries.

In that regard, I’m still waiting for a credible jq replacement…

boltzmann64

37 minutes ago

Learn SQL and interface with Duck. You will be 100x faster than Pandas/Polaris duo at fraction of memory. Also SQL is supported literally everywhere with a much more capable than Pandas API. Duck outputs to a Pandas Dataframe, but just treat that like a dictionary. Do all your processing, filtering and aggregation in Duck.

Also, try fx.wtf as a replacement for jq. it comes with a in-built tui viewer that supports vi-keybindings. Ecmascript is built into fx.wtf so you can query the JSON with JS notation (where JSON was born). You can use any JS functions including map/reduce/filter or perform any kind of transformation instead of learning jq dsl that you will forget tomorrow.

mhh__

an hour ago

Avoiding pandas developers is a great reason to use polars imo

seemaze

an hour ago

It has been for me. I greatly prefer the API, it fits my mental model much better. Give it a try!

bmitc

2 hours ago

There are awkward things. For example, if you ingest a nanosecond resolution timestamp, there's no way to re-export that out of the Polars dataframe with nanosecond resolution.

therno

3 hours ago

I use Polars 2.0(rc) to (pre)calculate billions of weather scores on https://therno.com and it has been a lifesaver

Happy that I can upgrade to 2.0 final tonight.

niltecedu

2 hours ago

A bit surprised about the datafusion results from the post, I have tried it time and time again, but datafusion has always been the leading/trading blowers with polars for our workfloads with duckdb being vastly slower.

f311a

2 hours ago

There are some benchmarks for the previous version https://benchmark.clickhouse.com/#system=+ti%20rud|Dusa,s|PD...

orlp

an hour ago

Doing the benchmarks for 2.0 on the large AWS metal machines at small data sizes (SF=10) really opened my eyes that we have some low-hanging fruit in Polars when it comes to optimizing our constant overhead for smaller queries.

For example our join currently does a full partition into T partitions, for each of the T threads. Overall we create T^2 partitions, which on a 192-core machine is non-trivial. Great if you have a ton of data to feed that with, but if you 'only' have a few dozen million rows it becomes rather small. This is the primary reason we saw in the benchmarks that Polars pinned to 32 threads beats 192 thread Polars at SF=10.

I'll be working on improving that soon. I expect that to have a big impact on SF=10, and a decent impact on ClickBench, which sits between SF=10 and SF=100 in terms of rows.

sgarland

4 hours ago

TIL that Polars supports SQL. Amazing.

brap

an hour ago

At what point can we say that Polars is basically an in-memory database? (Genuine question)

dist-epoch

an hour ago

It did since the first releases, but it was limited.

Now it seems they want to go head to head with DuckDB.

efromvt

an hour ago

Love competition in the local data SQL space, makes everyone better

Centigonal

38 minutes ago

out of core sounds awesome! biggest thing that forced me to switch from pandas/polars to other solutions back in the day.

dkgs

4 hours ago

A coincidence with the fact duckdb is supposed to release 2.0.0 very soon? :)

orlp

4 hours ago

Actually yes. We had already been planning to do 2.0 for a long time. We originally said we'd move on from 1.x quickly when released 1.0 but ended up staying at 1.x much longer than intended.

From a quick check our first PRs were merged to the 2.0 branch in June:

    2026-06-17T21:27:51Z #27993 chore: Stop coercing `pl.col(...)` to selector ...
    2026-06-18T14:19:26Z #27996 chore!: Replace multi-seed hash API with a single seed
    2026-06-19T07:05:11Z #27991 chore(python!): Remove `Expr.flatten` function

m00dy

4 hours ago

Are they competing for something?

vindex10

3 hours ago

> first class SQL support, which together with the performance improvements has Polars leading DataFusion and DuckDB in TPC-H and TPC-DS1 benchmarks,

Apparently about something ))

hnd9q09qk4

4 hours ago

Thing I care about most is whether the old eager-vs-lazy footguns got cleaned up. Half my bugs were a stray collect() in a loop killing the query plan.

jt-s

26 minutes ago

Although I find pandas a bit aggravating in many ways, for myself and my equally idiotic laboratory scientist pals, seems that it is the default way you might interface with other libraries like SciPy (i.e. they expect things as NumPy arrays or pandas dataframes). Is this a real issue or will most things happily accept a polars dataframe? We don’t work with such large datasets that speed is likely a huge concern tbh.

dartharva

3 hours ago

Been using polars for over a year now, it is fantastic.

Vaslo

3 hours ago

My team is all moving over to polars and DuckDB

matthewpick

2 hours ago

What was your previous setup / stack?

tonyhart7

3 hours ago

finally long time coming

cant wait to upgrade my Quant trading bot

m00dy

3 hours ago

I'm also using it for BlockRotate, it's time to upgrade.

tonyhart7

8 minutes ago

I mainly trade xauusd and fx

more predictable

mrtimo

2 hours ago

I can use pandas to clean a dataset, but each cleaning task is usually one line of code. OTOH, With DuckDB with one SQL statement I can replace 40+ lines of polars/pandas. You may reply, SQL isn't as easy to understand! Fair point, it's a declarative language... which is why I use Malloy. Malloy is to TypeScript as Javascript is to SQL. Malloy is much easier to read and write (just as TypeScript is) because it has a built in semantic model -- all the joins, measures, and dimensions are done in one place.

Here is an example [1] of visualizing college football games. Here are all the queries, and semantic model that power all the visualizations [2] Here is the AI generated typescript/react that does the visualizations [3]. The Malloy ecosystem has Malloyyo and Publisher which are replacements for PowerBI and Tableau and Looker. Here is another example for visualizing global trade [4].

[1] - https://mrtimo.github.io/cfb-games/games-2026.html?week=Week... [2] - https://github.com/mrtimo/cfb-games/blob/main/drives.malloy [3] - https://github.com/mrtimo/cfb-games/blob/main/dashboards/gam... [4] - https://tradeexplorer.org/