Honestly, I think what kills it is that it's not an Algol-descended language [1]. If Erlang was written with an Algol-esque syntax it would have taken off years ago. But instead it has this weird syntax, which it then doesn't really do that much with. What do I mean by that? Haskell has a radically different syntax, but it does things with that syntax and its pervasive currying to enable a powerful succinctness that one can not imagine being translated back into the Algol-esque framework. Lisp does things with its bizarre syntax, making it obvious how to write correct macros and being homoiconic, which translates poorly back into Algol-esque infix languages. It's almost inconceivable that one could translate a concatenative program back into Algol-esque syntax [2]. But Erlang really doesn't do anything that couldn't be in Algol-esque syntax. (Near existance proof: Go. Yes, there are significant differences, but the two are inter-transliterable to a much greater degree than any pairing of any of the previous sets of language families.)
If it had an Algol syntax, and performed the SSA transform behind the scene, it would probably be very, very big now.
Bear in mind as I say this that I'm not necessarily advocating for those changes. For instance, this would require some tweaks to the semantics of pattern matching, too, which aren't necessarily for the better... in the abstract. However, they probably would be for the better in terms of usage.
I'm pretty sure Go is going to eat Erlang. Erlang programmers will 100% absolutely correctly complain that OTP can't be translated without loss into Go, and almost nobody will care. Again, I'm not necessarily advocating for this, because the Erlang advocates will be right, you just can't quite get it fully expressed in Go and that saddens me, it's just what's going to happen, I think.
In fact I'm doing it myself; the Erlang core of my system is getting pulled out and replaced by Go for a variety of reasons, and one is despite the fact my team is fairly adventurous over all, we're still better off finding people to work on Go than Erlang. (In the next couple of months I hope to release my first release of "reign", "Rewrite Erlang In Go Nicely", which brings some of the Erlang stuff into Go for the purpose of porting existing programs. I've been pulled into other fire fighting so I'm not on it this second, but I'll be getting back to it soon. That implements Erlang-like mailboxes and network clustering, and I've got a supervisor tree implementation on deck for Github too. Subscribe to https://github.com/thejerf to see when those come out in the next couple of months.)
By the way, Erlang advocates, bear in mind that trying to argue me out of this position is a waste of time. I've been programming in Erlang for 7 years now. I get the syntax just fine, even if I still don't like it. The problem is that you have to argue the greater programming community out of this position, and I don't think you have, and I really doubt you can. For better or worse, being non-Algol seems to put a hard limit on your general-purpose programming acceptance. (In my opinion, that is for the worse, but here we are. Again, please don't mistake this opinion as celebration of any of these facts. My opinion is that Erlang deserves better. My belief is that it won't get it.)
[1]: That's pretty much every modern mainstream language today: C(/++/#), Java, Python, Javascript, etc. Not all those languages come from the same semantic heritage (scripting vs. conventional OO manifest types being one big example), but they come from the same syntactic heritage. Contrast with the ML family, the Lisp family, the Prolog family (which is pretty much just Erlang now), and the Forth family for different syntactic heritages.
[2]: http://evincarofautumn.blogspot.com.es/2012/02/why-concatena...
The syntax issue is why I think the Elixir programming language[1] has a good chance of improving the ecosystem around the Erlang VM. Although, most Erlang programmers, who i've spoken to, feel the syntax is a non-issue, so it is more likely to convert programmers who are new to, or curious about, Erlang.
It is also worth a note that Elixir adds language features, such as default UTF8 binary strings, real macros, Clojure-style protocols, tooling, etc, on top of the Ruby-style syntax.
[1] http://elixir-lang.org/
I think this is mostly right. Erlang is, roughly, in the Lisp, Forth, Smalltalk bin, not the C, Ruby, Go bin. Its advocates are going to be going on about how "Erlang got that right in 199something" for years, even when things like Go have got most of what it has.
I wrote about Erlang 7 years ago, and a lot of that still holds true, IMO:
http://journal.dedasys.com/2007/09/22/erlang/
That said, I lately helped start a pretty big project in Erlang and have been using it a lot lately. It has improved, and it's a fun language to use in a lot of ways.
Err, by his classification, Smalltalk is also in the C, Ruby, Go bin.
He divided by Algol-like syntax and heritage, not by mere popularity.
> I'm pretty sure Go is going to eat Erlang.
I wouldn't be surprised by this. Go's concurrency pattern is fantastic, but to match what Erlang does it needs to be easily extended beyond the running process. Can channels connect multiple "nodes" yet? That is, two running go programs, can they communicate via channels or do they have to use some other IPC mechanism? Across a network? That universal communication structure is (IMO) erlang's killer feature.
I do not have direct knowledge, but I'm pretty sure the problem with "network" channels in Go is the synchronous nature of channels. Even if we did have a perfect network, that works poorly across a network, and if the network starts degrading even a bit the sync channels just completely fall apart. The language treats them as a fast synchronization method. You can very nearly translate RPC to "channel" in this argument and it works: https://news.ycombinator.com/item?id=2318249 (Err, I did try to find a better explanation that wasn't mine, but Googling up an RPC criticism was hard, because saying "RPC" comes up with a pretty big search pool to be trying to find the word "bad" in...)
My reign library basically just ports over the Erlang idea; asynch messages, PID-like things that can be transmitted across the network and identify the same mailboxes within a cluster. It's meant to be a porting aid, not a solution you'd reach for in Go to create a brand new program. (If you do that, it's on you.) Clustering like this is nontrivial and the big reason I'm sitting on the code locally until clustering is at least working under some circumstances; without clustering support, the mailboxes are just an invitation to write nonidiomatic code for no gain.
> I wouldn't be surprised by this. Go's concurrency pattern is fantastic
Only for those not versed in java.util.concurrent, TPL, PPL, TBB, Cilk Plus.
java.util.concurrent doesn't have a convenient mechanism for "selecting" across available channels/queues without blocking. This is a key feature of erlang and go. There are ways to simulate it, but it's not the same thing. (I'd love to be wrong about this).
Yes, you need to peek() the queues, so it is not really being blocked on select like Go.
But then, one can use something like Akka, which I forgot to mention, and still be on the JVM.
>so it is not really being blocked on select like Go.
Select in Go only blocks if you don't feel out the "default:" case...
"Those not versed in Cilk Plus" is a pretty large set
Erlang scheduler time share each processes. So if you have an infinite forloop, it while stop for several ms for other processes to run.
Preemptive scheduler, I doubt Go have this, then again Go doesn't have a VM/OS system.
go 1.2 has preemptive scheduling http://golang.org/doc/go1.2#preemption
In fairness, it's worth pointing out that Go 1.2 has partial preemption. Preemption is still checked only on function entry. However, unlike something like Node, this is not built in the language semantics, it's a characteristic of the runtime, and I imagine over time this will be fixed.
>Preemption is still checked only on function entry.
And I believe also heap allocation... the two cases together make what is still "partial preemption" pretty damn effective.
Not knowing enough about Erlang/OTP or Go: Would it be possible for more of OTP to be replicated in a Go 2.0? What's missing? What would it take? What can never be replicated?
Erlang is like Lisp in that once you build a language up to a level of "reliable disturbed system language," you've basically re-implemented Erlang.
Is "disturbed system" intentional? Nice term for a bunch of processes which are expected to die occasionally.
You can recover a substantial portion, but there's a few things you can't quite get back. Since Go is a mutable-state language, and also a shared-state language (isolation is by convention, not by language design), you have to live with the consequences of that. In my supervisor tree implementation, the restart of a monitored goroutine is just to fire off a new goroutine on the exact same object again; if you still have bad state lying around and immediately crash again, well, too bad, the supervisor can't do anything about that. (Except not thrash the processor with endless restarts.) In contrast, when an Erlang supervisor restarts a process, it is guaranteed to be a fresh process with no shared state from the one that just crashed. It's much more likely to not immediately crash.
(I fiddled with having the user specify a closure that is supposed to "create" a new object for the restart, but then simplified that away when I realized that the closure itself could well have mutable state too. Better just to stick with the simpler interface and document the consequences of failing to clean out state at the beginning of your monitored code. On the plus side, so far the interface for the supervised-things is "Serve()" and "Stop()", and nothing else; hard to argue with that level of simplicity!)
You also won't ever be able to recover live code restarts; you can try to transfer all state to a new process, including file handles, but that's a lot of implementation work. (I'm stuck right now on the fact that my connections are encrypted, and the encryption code won't let me even get to the state, let alone transfer and reconstitute it somewhere else. Someone's got some code for moving file handles around on GitHub but I can't seem to Google it up.) Most people already live in a world where live restarts are difficult or impossible, though, so they won't miss that. Getting a REPL into a live Go server is impossible, at least with Go itself, and will also probably never be as clean as Erlang. (Go "2.0" could fix that with an official Go interpreter, which is the missing bit. I can handle the sockets, I can bind readline and give you a nicer REPL, but interpreting Go is a challenge. Progress is being made, I believe as part of the new compiler implementation, but I'm not sure anyone has this as an explicit goal.) Again, most people live in a world where they already don't have a REPL into their running code, and therefore won't miss this.
There's also some more subtle semantic differences, but I'm saving that for a blog post. (It gets long, especially if I first have to explain Erlang, which is what the blog post does.)
really looking forward to that blog post. I clicked to your github profile in your other comment upthread, but realized I was already following you there after seeing gomempool. :)
I'd have to say your idea of supervisors misses the point.
Supervised processes provide guarantees in their initialization phase, not a best effort. This means that they are always restarted to a known stable state. It's not just a question of retrying, it's a question of returning to a piece of data and environment that is reliable. I wrote on this more in details at http://ferd.ca/it-s-about-the-guarantees.html
If you reuse the same exact object again that can be modified, you lose these guarantees entirely, and it becomes a question of convention and attention to detail rather than something provided for you.
The REPL is a vital part of my every day Erlang experience. I can't imagine living without one when things get tricky in production and that pre-built tools aren't enough. Being able to poke around and inspect everything while it runs is great.
It is funny to me that Erlang combines unique tools for application stability with the ability to edit stuff online in production. I'm not saying I never launch a production Rails console, but seeing that in Erlang is a weird juxtaposition. I'd love to understand better how those fit together comfortably in an Erlanger's world view.
In practice, one will have to refrain from some of the crazier stuff when working interactively in the shell of a production system. However, it's extremely useful for debugging and minor adjustments by a skilled engineer. Things you can do include inspection of data structures, (very) careful tracing, loading of instrumented modules or minor corrections.
Since Erlang is dynamically typed, practically all data structures can be inspected at run-time, and one particularly nice thing about the Erlang shell is that it tends to be responsive even when the system is operating at near-maximum load.
"I'd have to say your idea of supervisors misses the point."
I think it's unfair to explain how I "missed the point" to me when I'm the one who brought it to your attention in the first place, when I called it out explicitly as a weakness, in the context of a post explaining how OTP can not be completely ported into Go without loss. I did after all explain in a parenthetical that I tried to fix it, realized the fix wasn't really a fix, and decided just to go simple and idiomatic after that.
The thing is, while it is sad that we lose that characteristic of supervisors, there are still other useful characteristics: Sensible restarts; naive restart code has a lot of pathological cases. I rather expect my library to ship with a couple at first too, but we (yay open source) can fix it in one place, once. Composition of supervisors is a great way to build applications, with self-contained pieces; the whole "hierarchy" still applies, though I find with the elimination of the restart strategies it gets a lot simpler, for better and for worse. It's still nice to be able to build code and easily slot it into a management system, even if it can't quite be as powerful.
A complete translation is impossible, but an idiomatic one may be, and I'm already finding it useful even in my little coding efforts.
As for the REPL... yes, that is an unmitigated loss, no question. I'm mitigating it in my personal app with a lot more flexible logging and some more external controls, but there is no true replacement for a REPL. Though... we may get a REPL someday, it just may be either a mere subdialect of Go or a separate scripting language. I've resisted putting a dependency on one of the very young Go scripting languages in my project, but theoretically in another year or two this might at least be mitigated.
Fair enough.
>Someone's got some code for moving file handles around on GitHub but I can't seem to Google it up:
https://github.com/sharvil/flingfd
Besides the issues that jerf mentioned, Erlang also has per-process garbage collection built right into the VM. Last time I checked Go's garbage collection was still stop-the-world, mark-and-sweep, making it less suitable for soft real-time systems than Erlang.
"Go 2.0" probably can't ever quite fix that, but there's some potential for improvement; one can imagine as an optimization the compiler being able to determine a particular goroutine really is isolated (no shared state with anything else at start time, communicates only via channels that are full copiers of state, etc.) and independently GC's them, and also excludes them from the "stop" in the "stop the world". But yes, it is stop-the-world, and I'm keeping a close eye on the GC performance in the app I'm transitioning. (So far the numbers are working out OK, but it's definitely in the range I have to watch out.)
Yeah I don't doubt there are some smart people out there that could make something like that happen (look at Azul and their pauseless GC for the JVM for example). However, it will be a tough fix when compared to the simple/pure approach Erlang takes.
For the life of me, I cannot understand why Oracle hasn't bought Azul and made it available to all, for free.
Azul only works well on multi-processor machines with LOTS of RAM, but when it does, it beats everything else out there by a mile. You'd think Oracle would want that.
The Go situation is changing:
runtime: concurrent GC sweep Moves sweep phase out of stoptheworld by adding background sweeper goroutine and lazy on-demand sweeping.
https://codereview.appspot.com/46430043/
I'm looking forward to see how reign turns out!