Announcing Polars 2.1

By Polars Team on Tue, 27 Oct 2026

Polars 2.1 is out. This post gives an overview of the highlights grouped by area. Everything else is in the release notes.

Upgrade with:

pip install -U polars

Before you upgrade

Two changes may need your attention:

#30071 – Disallow mixed string <=> numeric arithmetic in eager. pl.Series([1, 2]) + "4" used to return ["14", "24"] and now raises, as it already did in lazy
#29888 – Drop support for Python 3.10

Out-of-core

In 2.0 we enabled spilling to disk by default, but the most common reason for running out of memory, a high-cardinality group-by, couldn’t spill yet. Now it can.

#29911 – Run the streaming group-by out of core, with shuffles
#30282 – Evaluate the streaming window node (over) out of core
#30104 – Add an out-of-memory kill threshold (POLARS_OOMKILL_THRESHOLD_FRACTION)
#29259 – Add a reverse node to the streaming engine
#29792 – Add query-level IO active time and thread count metrics

To see it working, give Polars a small memory budget and turn on the spill metrics:

import os

os.environ["POLARS_OOC_MEMORY_BUDGET_MB"] = "200"
os.environ["POLARS_OOC_LOG_METRICS"] = "1"

import polars as pl

n = 30_000_000
lf = pl.LazyFrame({
    "k": pl.int_range(n, eager=True).shuffle(seed=0),
    "v": pl.int_range(n, eager=True),
})

lf.group_by("k").agg(pl.col("v").sum(), pl.col("v").n_unique()).collect()

spill_stats(group-by): ... spill(succ=100.0%, n=312) ...
prefetch_stats(group-by): file_mb(written=132.8, read=132.8) ...

The state for 30 million groups doesn’t fit in 200 MB, so the engine writes part of it to disk and reads it back to finish the query.

SQL

Since 2.0 we treat SQL as a first-class citizen, and this release adds a lot of it.

#30188 – Support recursive CTEs (WITH RECURSIVE)
#30256 – Support CREATE [OR REPLACE] TEMP TABLE
#29834 – Add ASOF JOIN ... MATCH_CONDITION (...)
#30131 – Support correlated subqueries in JOIN ... ON and correlated scalar subqueries without an aggregate
#30141 – Support uncorrelated subqueries in JOIN ... ON
#29912 – Add generate_series and range
#29967 – Add REGEXP_REPLACE and DATE_TRUNC
#30144 – Support CURRENT_DATE, CURRENT_TIME and CURRENT_TIMESTAMP
#29899 – Support time-zone-aware timestamps
#29951 – Support STRUCT and MAP type names
#29963 – Support qualified table names
#29954 – Add a read_ndjson table function with named options
#30047 – Accept a list of paths in the read_* table functions

Iceberg

You can now manage Iceberg tables end to end from Polars: create them, write to them, query them and drop them.

#30113 – Attach Iceberg catalogs to SQLContext via register_catalog
#30260 – Support CREATE TABLE with column definitions in attached catalogs
#30247 – Support DROP TABLE [PURGE] on tables of attached catalogs
#29909 – Add overwrite_filter and dynamic partition overwrite to sink_iceberg
#29908 – Add create_if_not_exists to sink_iceberg
#30087 – Add create_namespace_if_not_exists to sink_iceberg
#30091 – Support sinking to Iceberg tables with a bucket partitioned field
#30032 – Add pl.delete_iceberg
#29753 – Add a Rust resolver for scan_iceberg

Parquet

#29606 – Read encrypted Parquet files
#29905 – Write encrypted Parquet files
#29987 – Add experimental GeoParquet read and write support

Smaller additions

#30169 – Support full joins in join_where
#30138 – Add Expr.truncdiv for division rounding toward zero
#30232 – Add conversion to and from masked numpy arrays
#30133 – Allow reinterpret between Int128 and Decimal
#29424 – Add sampling to the KLL approx-quantile sketch

Encryption, register_catalog, delete_iceberg and masked numpy conversion are still unstable, so their API may change in a later release.

Performance

This release has close to 90 performance PRs. These are the ones you are most likely to notice:

Out-of-core

#30296 – Make file readers wait for spills while over the memory budget
#30294 – Wait for spills while over the out-of-core memory budget
#30280 – Return freed memory to the OS right away while out-of-core data is spilled
#30237 – Set the default spill compression level to 1
#30292 – Lower the default out-of-core prefetch fraction to 0.5

Optimizer

#29852 – Track sortedness through joins
#29860 – Rewrite sort().first() to min() and single-key sort_by().first() to min_by()
#30056 – Merge union inputs that scan files of the same kind into one multi-file scan
#30007 – Push predicates through order-keeping columns
#30123 – Push as-of join key filters to the right input

Parquet

#29953 – Use the Parquet string prefilter for literal str.contains and SQL LIKE
#30006 – Skip Parquet row groups with the running top-k threshold
#29882 – Filter Categorical and Enum Parquet columns while decoding
#30245 – Speed up writing nested Parquet columns

Cloud and Iceberg

#29863 – Reduce requests and raise concurrency for cloud CSV scans
#29857 – Reduce requests and raise concurrency for cloud NDJSON scans
#29798 – Use Iceberg statistics
#29790 – Reuse the Iceberg catalog and cache REST catalog scans

Kernels and construction

#29813 – Huge speedup creating wide Struct data
#29739 – Huge reduction in from_rows and from_dicts memory usage during nested value conversion
#30281 – Aggregate unique and n_unique in the streaming group-by itself
#29849 – Speed up arg_min and arg_max, especially on data with nulls

Bug fixes

Some of the bug fixes in this release:

Wrong results

#29887 – Fix incorrect filter expression rewrites
#29793 – Don’t push filters into a cache past a projection
#30199 – Correct unsigned Parquet statistics merging
#29868 – Write valid Parquet statistics and column index for NaN and nested values
#30290 – Compare Array(Struct) rows field by field
#30060 – Correct rolling window when offset is not the negation of period
#30068 – Don’t silently wrap when casting far future dates
#30069 – Truncate when casting Decimal to integer
#30200 – Fix wrong JSON output for Struct and Array inside list
#29965 – Fix wrong results and name clashes for computed SQL GROUP BY keys
#29823 – Fix wrong results in SQL REPLACE and ARRAY_AGG, join_where on decimals and sorted group-by keys

Hangs and panics

#29824 – Fix lost wakeup in the async executor when waking an already-scheduled task
#29819 – Fix livelock in TaskParker on very low thread counts
#29804 – Fix panic in when/then that masks a multi-chunk struct column
#29850 – Fix row-estimation panic for aggregations over an empty scan
#29918 – Raise instead of panicking on out-of-range datetimes in to_list
#29945 – Raise instead of panicking when a sink is used as input of another operation

Other fixes

#29828 – Call statx via raw syscall so wheels load on glibc < 2.28
#30266 – Respect memory limits of nested cgroups
#30065 – Disallow scanning and sinking to the same file in one query
#30001 – Fix reading Parquet files with empty row groups
#30204 – Raise on Delta tables with column mapping

Last words

Thanks to everyone who contributed code or filed an issue for this release. If something breaks after upgrading, please let us know.

Announcing Polars 2.1

Before you upgrade

Out-of-core

SQL

Iceberg

Parquet

Smaller additions

Performance

Out-of-core

Optimizer

Parquet

Cloud and Iceberg

Kernels and construction

Bug fixes

Wrong results

Hangs and panics

Other fixes

Last words

Let's keep in touch