Cartesia publishes H-Net scaling and byte-level learning research

On October 29, Cartesia published scaling experiments on an improved hierarchical network, H-Net, which replaces fixed tokenization with end-to-end dynamic chunking and reports gains in language modeling efficiency. In a strictly matched experiment, a one-stage H-Net was 1.28x more compute-efficient than a BPE Transformer on validation loss. A two-stage H-Net reached the final average evaluation score that SmolLM3 reaches after about 11 trillion training tokens, using about 1 trillion tokens. Compute per byte differed by under 2%, but parameter counts and training-stage setups differ, so that result must be kept separate from the strictly matched one-stage experiment.

Written by AI from the coverage · updated 1 hour ago

Not enough continuous observations to draw a trend yet.