Trained KV cache bank turns any LLM into Jev like Model gambler-relay-us-west1.leo-fish.ts.net 1 point by faangguyindia 3 minutes ago
faangguyindia 1 minute ago Here you can see a 26B model hacked using a trained KV cache bank responding like Gemma, latency i am seeng <127ms
Here you can see a 26B model hacked using a trained KV cache bank responding like Gemma, latency i am seeng <127ms