LiquidCache
VLDB 2026

LiquidCache

Efficient Pushdown Caching for Cloud-Native Data Analytics
Xiangpeng Hao  ·  Andrew Lamb (InfluxData)  ·  Yibo Wu  ·  Andrea Arpaci-Dusseau*  ·  Remzi Arpaci-Dusseau*
University of Wisconsin–Madison
* joint senior authors
Supported by InfluxData Bauplan Spiral
Takeaway
Cache a better format
than the storage format.
Object store CSV JSON Parquet MCAP Cache Liquid Liquid Liquid Liquid Compute DataFusion any engine transcode once serve every query CSV Liquid JSON Liquid Parquet Liquid MCAP Liquid

10× cheaper

Storage keeps its stable format; the cache serves one that’s 10× cheaper to query.

Zero overhead

Transcoding uses idle cache CPUs and hides behind the cache-miss I/O.

2 / 16
Background · 1 / 2

Cloud-native analytics already reads remote data through a cache

Object store Parquet Cache tier Parquet the same bytes, closer Compute BI SQL ML 100–200 ms $ per GET ~1 ms Standard practice: Alluxio, Databricks disk cache, and in-house caching tiers.
3 / 16
Background · 2 / 2

LiquidCache answers in a better format
than it fetches

Object store Parquet unchanged LiquidCache Parquet Liquid transcodes on first access, keeps only the better format, evaluates filters in place Compute DataFusion any engine unchanged on miss · 100–200 ms matching rows · ~1 ms
4 / 16
Cache a better format than the storage format —
1

Why is a different format 10× cheaper?

2

Why is it zero overhead?

5 / 16
Part 1 · Where the 10× comes from · 1 / 5

Writers pick formats for the archive

The archive needs
  • Stability over decades
  • Universal compatibility
  • Compression
  • Open governance
Today’s hardware wants
  • Cheap decoding
  • SIMD-friendly layouts
  • Random access
  • Freedom to evolve
What lands in the lake
CSV JSON Nimble in-house
6 / 16
Part 1 · Where the 10× comes from · 2 / 5

File formats are not created equal: each new generation is faster to query

CSV · JSON row-oriented text Parquet · ORC columnar Vortex · Liquid decoding-first design + column pruning + compression, statistics + SIMD encodings + random access faster to query 10× lower decoding CPU, same compression as Parquet
7 / 16
Part 1 · Where the 10× comes from · 3 / 5

Switching formats takes a decade

Option A

Re-encode the whole lake

  • petabyte-scale ETL, paid up front
  • two copies of everything
  • converts data no query reads
Option B

Ask every writer to switch

  • thousands of writers to convince
  • breaks the oldest consumer
  • churn vs. archival stability
IN PRACTICE New formats keep arriving; production data stays in years-old Parquet — held back by governance, not performance.
8 / 16
Part 1 · Where the 10× comes from · 4 / 5

Keep the archive as is —
cache the better format instead

Object store CSV JSON Parquet MCAP all unchanged Cache server Liquid our format ephemeral Compute DataFusion any engine nothing to adopt transcode serve fast Whatever the writers chose goes in; one format we control comes out. Liquid filters strings without fully decoding them; it owes no compatibility.
9 / 16
Part 1 · Where the 10× comes from · 5 / 5

The whole system gets cheaper

10×
lower cache CPU per query
not end-to-end latency
4×
smaller than Arrow in memory
vs. Arrow, still query-ready
100×
lower network traffic
filters run in the cache
ClickBench and TPC-H on Apache DataFusion, against non-pushdown caches.
CloudLab 6525: 16 cores / 32 threads, 128 GB RAM, 10 Gbps network. Five runs per query, last three averaged.
10 / 16
Cache a better format than the storage format —
1

Why is a different format 10× cheaper?

2

Why is it zero overhead?

11 / 16
Part 2 · Why it’s zero overhead · 1 / 4

Disaggregation already made the cache mandatory

Object store Cache Compute on miss 100–200 ms ~1 ms The server, its DRAM, and its network are already paid for — whatever format the cache happens to store.
12 / 16
Part 2 · Why it’s zero overhead · 2 / 4

Cache servers bundle idle CPUs

A cache server network serving reads DRAM · SSD holding data CPU idle spend the idle cycles on transcoding The cores come with the box — transcoding is what they are good for. Measured on a conventional byte cache.
13 / 16
Part 2 · Why it’s zero overhead · 3 / 4

Only the data that queries touch is ever transcoded

DATA LAKE — PARQUET written once, read rarely on miss LIQUIDCACHE — LIQUID hot subset, transcoded No eager ETL — the cold data is never converted at all. >50% of smart-tiered Azure capacity sits ≥30 days untouched — Azure, 2026. ~80% of Redshift CTAS tables feed exactly one query — VLDB 2024.
14 / 16
Part 2 · Why it’s zero overhead · 4 / 4

Transcoding hides behind the miss I/O

Parquet LiquidCache transcoding CPU · 2.2 s 0 10 20 cold-run time (s) Cold runs match Parquet everywhere but memory; warm runs 2× faster. ClickBench on CloudLab — 16 cores, 128 GB RAM, 10 Gbps network.
15 / 16
Conclusion
Cache a better format
than the storage format.

10× cheaper

Storage stays Parquet forever; the cached format cuts CPU by 10× and network by 100×.

Zero overhead

Idle CPUs, only the data queries touch, hidden behind the miss I/O.

github.com/XiangpengHao/liquid-cache
Techniques upstreamed to Apache Arrow, Parquet, and DataFusion.
InfluxData Bauplan Spiral
16 / 16