« Everyone Should Know SIMD »: Mitchell Hashimoto's reminder about the only optimization that still matters

Mitchell Hashimoto (founder of HashiCorp) signs an essay that presents a simple thesis: in the era of wide CPUs, not knowing SIMD means leaving 4× to 16× performance on the table.

The concrete case

Mitchell Hashimoto - the co-founder of HashiCorp who became a prolific individual contributor (he now works on Ghostty, his terminal emulator in Zig) - publishes a long post simply titled Everyone Should Know SIMD. The thesis can be summed up in one sentence: in a world where CPUs are stagnating in frequency but widening in vector registers, ignoring SIMD is leaving orders of magnitude of performance on the table.

SIMD (Single Instruction, Multiple Data) refers to CPU instructions that apply the same operation to multiple values simultaneously - adding eight float in one cycle, comparing sixteen bytes at once. On modern x86-64 CPUs, these are the SSE / AVX / AVX2 / AVX-512 instruction sets. On ARM (Apple Silicon, Neoverse servers), it's NEON and SVE.

Under the hood: why the topic is resurfacing

The debate is not new - SIMD has been around since MMX (1996). What has changed:

1. Register width: 128 bits (SSE, NEON) → 256 bits (AVX2) → 512 bits (AVX-512). An AVX-512 register holds 16 float32 or 64 int8.
2. Availability: AVX2 has been present on all consumer x86 CPUs since Haswell (2013). NEON is standard on all ARMv8 (iPhone 5S and later, all ARM servers).
3. CPU frequency is plateauing: the "free" gains from one generation to the next are diminishing. Vectorization, on the other hand, continues to pay off.
4. Auto-vectorizers have limits: compilers (LLVM, GCC) vectorize simple loops well, but often fail when there is a branch, pointer chasing, or a particular accumulation. The programmer has to do it manually.

What Hashimoto says

The central message: SIMD is not just for graphics engine coders or video codec developers. Any mass data manipulation - JSON parsers, full-text search engines, string comparisons, statistical calculations, image filters, hashes - benefits massively from going vectorial.

Tangible case studies:

• simdjson parses JSON at 3-5 GB/s on a recent CPU, largely thanks to SIMD.
• The sorting engines VqSort (Google) and ips4o use SIMD to beat std::sort by a factor of 3-5 on integer arrays.
• Columnar DBs (ClickHouse, DuckDB) are built around SIMD kernels.
• Parsing Protobuf and UTF-8 can be vectorized for a similar gain.

In other words: it's no longer a niche, it's a basic skill for anything related to performance at scale.

How to get started without drowning

SIMD by hand is notoriously painful: unreadable intrinsics, fragile portability between x86 and ARM. Three viable approaches today:

1. Highway (Google): a C++ library that abstracts SSE/AVX/NEON/SVE behind a common API, runtime dispatch according to the CPU.
2. std::simd (proposed for C++26): in the STL, portable, but still experimental.
3. Portable SIMD in Rust / Zig: both languages offer portable vectorial APIs (core::simd in Rust nightly, @Vector builtin in Zig).

For a "standard" application developer, entry through a portable library (Highway, std::simd, @Vector) avoids diving into low-level intrinsics while keeping most of the gain.

Our take

Hashimoto is right: the generation of developers trained on managed runtimes (JavaScript, Python, Go) often ignores that the CPU has had, for ten years, computing means that are never used by their code. This is not an accusation - it's a fact. And the return on investment, when you finally connect these means to a critical path, is spectacular.

Key takeaways

• SIMD is no longer an exotic optimization: it's the main source of remaining gains on CPUs.
• Portable libraries (Highway, std::simd, Zig @Vector) make the entry affordable.
• Anything that deals with data at scale (parsers, DBs, images, hashes) should be questioned from this angle.