Table of Contents
Fast data algorithms
From: https://jolynch.github.io/posts/use_fast_data_algorithms/
| Use case | Bad Performance | Good Performance | Performance Gain |
|---|---|---|---|
| Trusted data hashing | md5, sha2, crc32 | xxhash | ~10x |
| Untrusted data hashing | md5, sha2, sha1 | blake3 | ~10x |
| Fast compression | snappy, gzip (zlib) | lz4 | 10x over gzip~2x over snappy |
| Good compression | gzip (zlib) | zstd | ~2-10x |
| Best compression | xz (lzma) | zstd -10+ | ~2-10x |