Ask HN: How to scale long-lived connections with Kubernetes?

7 points by k888sss 2 months ago

My application serves users via long lived TCP connections and users are quite volatile. They come and go without any obvious patterns. And my current approach of scaling based on CPU/memory usage are too damn barbaric. I should not break active connections, but I also need to control the cost. On top my head, I can think of writing a custom middleware to manage connections and pods. Are there any ready-to-use solutions or approaches to tackle similar problems? Thanks

wmf 2 months ago

This kind of issue comes up in game development where I think the standard answer is a tier of proxies that can hold open the connection to the clients while reconnecting to different backends.

  • k888sss 2 months ago

    Unfortunately, client code is not upgradable, and I have to solve this issue from server side only.

    • neonlex 2 months ago

      I don’t understand your answer, that’s basically what wmf described. Put a proxy or load balancer in front of your service.

      • k888sss 2 months ago

        Thanks. I can't upgrade clients to add reconnect functionality, and I can only work on the server side to solve the scaling issue. I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

        • dominotw 2 months ago

          > I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

          thats exactly what the original suggestion seemed to be was.

          • k888sss 2 months ago

            Yupp. I think I misunderstood at first. Now I get it. Thanks all.