points by pron 7 years ago

The funny thing is that that post doesn't address any of the theoretical arguments made by Brooks. While certainly less rigorous than physics, his argument is analogous to a discussion of the speed-of-light limit, and that "Yes Silver Bullet's" argument is analogous to, "but what if we use a different kind of fuel in our rockets?" and then claiming that that fuel is the silver bullet with neither empirical nor theoretical evidence to support that claim.

Indeed, quite a few PL enthusiasts called Brooks's predictions overly pessimistic based on similar non-arguments back in the '80s (he lists some of them in his followup, No Silver Bullet, Refired), but reality proved his predictions to be overly optimistic. So he was right and they were wrong. The arguments in Yes Silver Bullet had already been put to the test failed. Is that reality a definitive proof that there is little accidental complexity left? Of course not, but it does raise the bar for those who claim there's a lot of it left, certainly well beyond simply asserting that that is the case or considering it a reasonable working hypothesis.

lliamander 7 years ago

So the author does conjecture that statically typed FP would represent a major improvement on accidental complexity. I agree that this is unlikely (though I haven't done enough of that myself to be sure) but most of the article seemed to be making an empirical claim about productivity improvements that have already been achieved.

Fred Brooks' central claim was that there would be no "order of magnitude" improvement by removing additional accidental complexity, and that real improvement would come from tackling essential complexity.

Whether Mark Seeman's empirical claim is true depends somewhat on whether we think the improvements were on the accidental or essential complexity, and whether any of them individually were an "order of magnitude" improvement.

What is accidental vs. essential complexity? According to Brooks, the essential complexity is formulating "complex conceptual structures" that make up the design of the software. In other words, figuring out what the software is supposed to do and how to do it.

Accidental complexity is concerned with "the mapping of [the design] onto machine languages within space and speed constraints". It seems to me that, according to Brooks, the biggest barriers for software development prior to 1986 was that programming languages were pretty horrible for expressing designs and that computers were too slow and limited to be able to effectively execute software (without seriously butchering the design).

His argument seems to be that, as of 1986, most languages were sufficiently high-level and computers (and PL implementations) sufficiently capable that most programmers could easily express their designs and expect them to be able to run efficiently enough. There might still be progress to be made there, but that it wouldn't be a massive improvement. Instead, most of the improvement would come from getting better at designing those "complex conceptual structures".

So, let's take a look at Seemann's list of improvements:

* WWW/Stack Overflow

* Automated Testing

* Git

* Garbage Collection

* Agile (the idea of iterative design/development rather than any process in particular)

Which of these tackle essential vs. accidental complexity (as Brooks defines the terms)? Agile development and automated testing would I think belong in the essential category (as Brooks himself refers to iterative design and tight feedback loops as methods for dealing with essential complexity). Garbage collection, on the other hand, seems like it dealt with accidental complexity.

I don't know about the others, but I would definitely say that garbage collection (especially the fast, low-latency GC we have now) does provide an order of magnitude improvement by reducing accidental complexity. Obviously, GC was invented before 1986, but didn't become widely adopted until (more than) 10 years later.

Will there be other such improvements? Maybe. Statically typed FP doesn't seem like a great candidate to me, at least not by itself. I do however think that Peter Van Roy's formulation of pure FP + deterministic concurrency + message passing + global transactional state may qualify. That's not one single advance, but it does represent (IMO) a order of magnitude improvement on the imperative + "threads and locks" programming everyone was doing 20 years ago.

Still I think Brooks was right in the general sense, being that we should put more emphasis on producing better designs.