If you are interested in deployments of databases on top of cluster orchestrators, here are some interesting projects:
* Postgres automation for Kubernetes deployments https://github.com/sorintlab/stolon
* Automation for operating the Etcd cluster:https://github.com/coreos/etcd-operator
* Kubernetes-native deployment of Ceph: https://rook.io/
There are a number of projects and storage solutions that you can use depending on what is important. Using persistent volumes and external storage like EBS is one way. Using distributed file-systems with local or network attached storage like Portworx or Ceph is another. I’m not sure using NFS for the post in this example demonstrates what you would be doing in the real world however.
Thanks for the links. Do you know about a project that Targets PostgreSQL as a operator, like etcd?
This one: https://github.com/sorintlab/stolon
It is not called the operator, but in many ways it is similar to what Etcd Operator does
If your pattern requires you to connect to etcd, then you're not taking the best advantage of Kubernetes.
Yes, you can connect to an etcd instance outside of Kubernetes, or deploy etcd on Kubernetes. CoreOS developed the etcd operator for the purposes of self-hosting the system KV store, but you can of course run a "user" etcd.
A better pattern is relying on the objects provided by Kubernetes, which are themselves backed by a lock service (which happens to be etcd, but that's an implementation detail you shouldn't worry about.)
You could, for example, use an Annotation on an object to do leader election. Look at how HA masters are implemented on Kubernetes, for example:
https://github.com/kubernetes/kubernetes/blob/master/pkg/cli...
That's a great point. But doesn't this make it mandatory for the applications to use Go? It would have been more useful if there was a REST interface for leader election.
Not at all - it's just a language for which there is a Kubernetes client. The API is 100% REST. We have a Python library underway also: https://github.com/kubernetes-incubator/client-python
Right, but I couldn't find a REST API for leader election. If my app is in Go, I can use the leaderelection package here https://github.com/kubernetes/kubernetes/blob/master/pkg/cli.... But if my app is in Python, I won't be able to use that package. In that case I'd need to use etcd or zookeeper on top of k8s. Am I missing something?
You can reimplement that logic in Python. There's nothing magic about it: you can see in https://github.com/kubernetes/kubernetes/blob/master/pkg/cli... that it just uses a Kubernetes annotation.
There's some more information (and a sidecar pod prebaked) at http://blog.kubernetes.io/2016/01/simple-leader-election-wit...
Yes, one can reimplement it in Python, but don't forget:
// This implementation does not guarantee that only one client is acting as a // leader (a.k.a. fencing). A client observes timestamps captured locally to // infer the state of the leader election. Thus the implementation is tolerant // to arbitrary clock skew, but is not tolerant to arbitrary clock skew rate.
makes sense, this was written before TPR though, so there is a room for improvement.
Yes, we are currently developing a PostgreSQL operator for Kubernetes. We are heavy PostgreSQL users. Spilo[0] supports all kinds of slave configurations, backups, monitoring tools[1] and all advanced capabilities.
[0] https://github.com/zalando/spilo [1] https://github.com/zalando/PGObserver
Edit: https://github.com/zalando/patroni is a better link
Thank you! And Zalando as a bit company behind it is even better.
Are you going to use this as is in production, or do you have an internal version it?
We don't have a separate internal version. We are already running some hundreds PostgreSQL clusters in AWS + etcd with Spilo/Patroni. The Kubernetes variant is not used yet in production but we plan to migrate the first datasets soonish (probably some weeks).
This is great to hear, will definitely try it.