Yeah that makes sense. I took the optimizations of my hamming distance library to a bit of an insane level. I wrote this about the first round https://emschwartz.me/unnecessary-optimization-in-rust-hammi... The second round brought the best result down to 0.8 nanoseconds per comparison on x86 with SIMD (for a batch of 1000).
Separately, I’m a huge fan of your writing about search! It’s been very helpful while I’ve been building https://scour.ing.
This works especially well if your embedding model was trained to perform well with quantized embeddings. Binary + hamming distance = incredibly fast.
This post is from 2024 but I wrote about using this technique in https://emschwartz.me/binary-vector-embeddings-are-so-cool/
Hamming w/xor+popcount is the only thing I can make numpy do faster than float32 dot products :)
int8s, float16s are all fairly slow. I suppose it’s because BLAS does float32/64 very fast.
Yeah that makes sense. I took the optimizations of my hamming distance library to a bit of an insane level. I wrote this about the first round https://emschwartz.me/unnecessary-optimization-in-rust-hammi... The second round brought the best result down to 0.8 nanoseconds per comparison on x86 with SIMD (for a batch of 1000).
Separately, I’m a huge fan of your writing about search! It’s been very helpful while I’ve been building https://scour.ing.
Thank you! <3