This was a formative essay for me. When I look around the software industry, it's completely full of accidental complexity. Consider Kubernetes and the entire Cloud Native world - how much of that is actual business logic that your customers need? None of it.
The main areas of complexity I perceive today in building software are:
- "infrastructure complexity" (see above)
- "deployment complexity": continuous delivery and getting code into production
- API complexity: using 3rd party tools used to be function calls, now they're complex distributed systems
- code complexity: git, pull requests, setting up your dev environment, etc
I wrote more about this here [1], but I honestly can't believe how complex it is to create software today.
Very right! The frustrating thing is that it doesn't have to be this way. The "tower of babel" of languages/tools/technologies used in Web/Cloud/DevOps programming is daunting and i don't feel like it is really worth my time learning any of it unless and until i actually need it for a project.
He says BL is fundamentally wrong in his analysis, and that the overwhelming majority of time spent programming in 1987 turns out to be what BL calls "accident", and has been radically reduced by progress since.
There is still no "silver bullet", but there are many, many bullets with cumulative effect, so that many programming tasks that would have taken prohibitively long in 1987 can now be done on a whim.
Probably the biggest bullet is greater accessibility and power of libraries, coupled with languages better able to express and use them. When libraries are more useful, there is more motivation to make and publish them, and so there are more available to use.
The 2006 paper "Out of the Tar Pit" also disagrees with Brooks' estimate of total accidental complexity, explaining their position in terms of {essential,accidental} {state,control}. One of their arguments is that accidental (derived) state has a magnifying effect on complexity, so a first-order goal of any software architecture should be to reduce accidental state as far as possible -- and arguably, centralize the remaining essential state as much as possible.
It somewhat puts the old saying in another light: "There are two hard problems in computer science: cache invalidation and naming things." Cached state is, effectively, another name for derived state.
The basic principle of practical systems theory states, that you can’t remove complexity, you can merely transform it into a form of complexity you are more or less capable of handling. Libraries are hugely complex nowadays. Did a test a while back: from zero to a rudimentary web service with Spring on took ~2 days, with go (lacking any previous experience with the language) it was a few hours.
This was a formative essay for me. When I look around the software industry, it's completely full of accidental complexity. Consider Kubernetes and the entire Cloud Native world - how much of that is actual business logic that your customers need? None of it.
The main areas of complexity I perceive today in building software are:
- "infrastructure complexity" (see above)
- "deployment complexity": continuous delivery and getting code into production
- API complexity: using 3rd party tools used to be function calls, now they're complex distributed systems
- code complexity: git, pull requests, setting up your dev environment, etc
I wrote more about this here [1], but I honestly can't believe how complex it is to create software today.
[1] https://blog.darklang.com/what-is-dark/
Very right! The frustrating thing is that it doesn't have to be this way. The "tower of babel" of languages/tools/technologies used in Web/Cloud/DevOps programming is daunting and i don't feel like it is really worth my time learning any of it unless and until i actually need it for a project.
Dan Luu just wrote a blog post about this.
He says BL is fundamentally wrong in his analysis, and that the overwhelming majority of time spent programming in 1987 turns out to be what BL calls "accident", and has been radically reduced by progress since.
There is still no "silver bullet", but there are many, many bullets with cumulative effect, so that many programming tasks that would have taken prohibitively long in 1987 can now be done on a whim.
https://danluu.com/essential-complexity/
Probably the biggest bullet is greater accessibility and power of libraries, coupled with languages better able to express and use them. When libraries are more useful, there is more motivation to make and publish them, and so there are more available to use.
The 2006 paper "Out of the Tar Pit" also disagrees with Brooks' estimate of total accidental complexity, explaining their position in terms of {essential,accidental} {state,control}. One of their arguments is that accidental (derived) state has a magnifying effect on complexity, so a first-order goal of any software architecture should be to reduce accidental state as far as possible -- and arguably, centralize the remaining essential state as much as possible.
https://github.com/papers-we-love/papers-we-love/blob/master...
It somewhat puts the old saying in another light: "There are two hard problems in computer science: cache invalidation and naming things." Cached state is, effectively, another name for derived state.
You neglected to mention the third of the two hard problems: "off-by-one errors" :D
Discussed a month ago: https://news.ycombinator.com/item?id=25569148
The basic principle of practical systems theory states, that you can’t remove complexity, you can merely transform it into a form of complexity you are more or less capable of handling. Libraries are hugely complex nowadays. Did a test a while back: from zero to a rudimentary web service with Spring on took ~2 days, with go (lacking any previous experience with the language) it was a few hours.
If curious see also
2019 https://news.ycombinator.com/item?id=20818537
2017 https://news.ycombinator.com/item?id=15476733
2015 https://news.ycombinator.com/item?id=10306335
2011 (1 comment) https://news.ycombinator.com/item?id=3068513
Related from 2008 (by Brad Cox): https://news.ycombinator.com/item?id=239323
"These gains flow from the transformation of computer manufacture from an assembly industry into a process industry." [from the posted article]
Fred Brooks is a true genius to percolate a generation's worth of transformation into such a statement.