Algorithmic Breakthrough · Minimum Description Length (MDL)

Mathematical Compression.
100% Bit-Exact Fidelity.

Kolmos replaces dumb byte compression with intelligent mathematical explanations. Discovered formulas and prototype rows store data 1.66×–2.12× smaller than Parquet-zstd.

The MDL Optimization Objective
cost = bytes(program) + bytes(residual) + λ · decode_compute + μ · risk
Kolmos balances formula program size against residual delta bytes, verifying bit-exact restoration across all 18 golden test fixtures.
The Hierarchy of Explanation

The 3-Rung Explanation Ladder

As datasets accumulate, autonomous background workers elevate chunks up the compression ladder.

Rung 0

Dictionary & Bitpacking

Zstd + Per-Column Trained Dictionaries

Baseline literal compression. Trains custom dictionary models on high-cardinality strings and bitpacks uniform integer columns, achieving fast decompression throughput.

Efficiency: 1.2× – 1.4× over raw Parquet
Rung 1

Autonomous Formula AST Mining

Algebraic & Sequence Synthesis

Analyzes column vectors for deterministic mathematical relationships: linear relations (y = mx + c), arithmetic progressions (id = start + i), timestamp offsets, and string patterns.

Efficiency: 2.0× – 5.0× compression gain
Rung 2

Prototype Row Clustering

Bounded k-Medoids Clustering

Identifies representative prototype rows across multi-dimensional table clusters. Records are encoded as compact residual bit-deltas against cluster medoids.

Efficiency: Up to 10× on repetitive schemas

Physical Storage Benchmark Comparison

Tested on 100 GB real-world analytical schema (ecommerce events & financial telemetry).

Storage Format
Disk Footprint
Compression Ratio
Fidelity Guarantee
Uncompressed Postgres / MySQL100.0 GB1.0× BaselineLossless
Parquet (Snappy)38.2 GB2.6× ReductionLossless
Parquet (Zstandard Level 7)24.5 GB4.1× ReductionLossless
Kolmos KSF (Explanation Ladder)12.8 GB7.8× Reduction (1.9× smaller than Parquet-zstd)100% Bit-Exact Verified