Great paper, a classic. I assigned this as reading to my students. A worthwhile exercise is to try to refute the core idea regarding "essential" vs "accidental" complexity. You may find you end up disagreeing with him! But that is why I like this paper so much. Agree or disagree, it makes you think, and it's worth thinking about.
>> The hardest single part of building software system is deciding precisely what to build. ... Therefore the most important function that software builders do for their clients is the iterative extraction and refinements of the product requirements. For the truth is, the clients do not know what they want.
What's the role of the product managers here. In common experience, how much of the above falls on the product managers vs. the software builders?
Thanks!
Maybe the fact that the author is Frederick P. Brooks should be mentioned. From the domain name I was expecting something written by Bret Victor
HN typically goes by original title with some exceptions. It's ok to have to click through to figure out what the title is about (or by whomst), it's less ok to puff up a title by adding things to it which is usually why people slap on the author.
npm is a good example of the "buy vs build" idea, although npm modules are mostly free of course. Why write code when you can just call someone else's? But then you get problems like the left-pad fiasco
I always understood "Buy vs Build" as "buying software is cheaper than building it yourself". In the case of NPM, using most of the third-party modules out there (buying) is more expensive than writing the code yourself (building).
Also there is probably a decent amount of “wont get fired for choosing X” where X is React, date-fns, jasmine etc. But those things have dependencies with dependencies and so on.
Using off the shelf tools was perhaps 50/50 when this was written. For a long time it was common for computers to come with company-specific OSes and programming languages. The high volume of the PC helped change that, but it was not overnight.
I always understood it more like using existing libraries when they exists. So platforms with a huge choice of libraries with good support is the rational decision (which may be python, jvm ecosystem or js)
Depends on what you mean by "good people". You don't have to hire only from the top 10%, whatever that may mean to you.
One of my main lessons from Brooks was always to put people in the roles that they are best suited for. In "The mythical man month" developers are not interchangeable. There are a number of different roles that needs to be filled, and different types of people are required to fill them.
Brooks wrote a retrospective on it 9 years later, _"No Silver Bullet" Refired_, which appears as a chapter in the 20th anniversary edition of _The Mythical Man-Month_. Of course that is still 25 years ago now, so perhaps we should have a retrospective of the retrospective :-)
Oops, yes. I couldn't quite remember the title (my copy of the book is at home), and googling found people claiming both Refired and Refined, so I made my best guess...
On the contrary, I found the juxtaposition of archaic vocabulary alongside propositions which were not only true then but are still true today to be the most compelling part of his argument.
(Brooks, who referenced Goldstine/von Neumann and even Aristotle, certainly would not mind if we enjoyed his article in its context)
For my own contribution, I think machine learning may be a good refutation of No Silver Bullet. It removes the need for the “essential complexity” of solving a problem. Like imagine trying to make GitHub copilot before machine learning. In Brooks’s world, the essential complexity of designing algorithms and data structures would indeed make this very difficult to do. But by using machine learning, you just let the neural net figure out the problem.
One thing he got very, very wrong: technological advances -- faster computers, better languages, better compilers, better editors -- have blown programmer productivity through the roof. Another is that "object-oriented" programming turned out not to be very important, and rather a distraction.
I think you're being unfair. Brooks did mention those kinds of advances, he just didn't consider them magical. For example, from Wikipedia's summary (which aligns with what I remember from my own reading):
> "While Brooks insists that there is no one silver bullet, he believes that a series of innovations attacking essential complexity could lead to significant improvements. One technology that had made significant improvement in the area of accidental complexity was the invention of high-level programming languages, such as Ada."
The core of "essential" vs "accidental" complexity remains very insightful. If, like Brooks argues, we are mostly hitting essential complexity now, no single programming language or IDE is going to provide a radically large improvement anymore.
"Productivity" is a tricky beast. What does it even mean? Programmers typing fast and getting code into production quickly? But what about the success and correctness of the systems they write? What about bugs and performance problems? What about incorrect or misunderstood requirements? Have we solved those?
> like Brooks argues, we are mostly hitting essential complexity now
I'm a bit skeptical about this. If you look at random backend code, you'll find lots of boilerplate, control structures, error handling, dealing with different data structures, performance optimizations. All of that I would consider random complexity. All of that is there only because computers can't figure these things out on their own.
Control structures as in "business logic" is essential complexity to me.
Random spaghetti code and boilerplate isn't, but Brooks' argument is that improving this, while definitely a good thing to strive for, won't result in a huge improvement in productivity, because the major roadblocks lie elsewhere.
I don't think he is arguing against minor wins, either. He just says we must focus on the major ones.
If you go into backend code from the typical place where "backend code" is a thing, it can be derived completely mechanically from the frontend. So all the work that goes into it is accidental, for a start.
Often that relation goes both ways, and the frontend code (that is even more complex) can be derived from the backend too.
That's boilerplate code and I agree it can be completely automated.
I would argue it's a very convenient but minor thing, and that it doesn't truly change the complexity of writing code. It just makes it less dreary. I would also argue it's a win that it can be automated, and I think Brooks would agree!
I don't think he was arguing small wins don't matter.
Boilerplate in itself is not complexity (eg import statements), it’s just tedious and unergonomic. In fact it can reduce complexity, again with example of imports being a better alternative than global symbols.
Coupled boilerplate on the other hand is complexity, where if you change x in front end you also must change corresponding x in backend.
I might agree that error handling is business logic in the broadest sense, but control structures certainly are not. Unless there is an explicit business requirement like "use a nested for loop to iterate through an employee array", control structures are just implementation details.
It is, exactly, the distinction between accidental and essential complexity that has not panned out.
Productivity is about getting solutions successfully deployed to production. Things that interfere with that may be "essential" or "accidental", but the distinction turns out not to matter much.
> Things that interfere with that may be "essential" or "accidental", but the distinction turns out not to matter much.
I obviously disagree. I think it matters a lot, especially because it allows us to detect that trap programmers and hackers often fall prey to: hyperfocusing on minor technical details because they are cool to think about without regards to the actual gains.
That's not to deny a lot of improvements in the past decades actually made programmers' quality of life tremendously better.
Dan Luu is a very interesting essayist and I find myself nodding in agreement with a lot of what he writes. He's made it his business to go against the grain of most "universally held truths" (a great chunk of his essays are about debunking commonly held beliefs about tech and science), but in doing so I think he tends to overstate his case. In this case, I think he is right but also overstating his case, and Brooks remains as relevant as ever. Luu's 2022 addendum is interesting, because it certainly cuts both ways -- not just in the way Luu would want!
Great paper, a classic. I assigned this as reading to my students. A worthwhile exercise is to try to refute the core idea regarding "essential" vs "accidental" complexity. You may find you end up disagreeing with him! But that is why I like this paper so much. Agree or disagree, it makes you think, and it's worth thinking about.
>> The hardest single part of building software system is deciding precisely what to build. ... Therefore the most important function that software builders do for their clients is the iterative extraction and refinements of the product requirements. For the truth is, the clients do not know what they want.
What's the role of the product managers here. In common experience, how much of the above falls on the product managers vs. the software builders?
A classic.
A few threads that have had traction in the past:
(2021) https://news.ycombinator.com/item?id=25926136
(2019) https://news.ycombinator.com/item?id=20818537
(2017) https://news.ycombinator.com/item?id=15476733
(2015) https://news.ycombinator.com/item?id=10306335
The Wikipedia article:
https://en.wikipedia.org/wiki/No_Silver_Bullet
Thanks! Macroexpanded:
No Silver Bullet: Essence and Accidents of Software Engineering (1987) - https://news.ycombinator.com/item?id=25926136 - Jan 2021 (9 comments)
No Silver Bullet (1986) [pdf] - https://news.ycombinator.com/item?id=20818537 - Aug 2019 (85 comments)
No Silver Bullet: Essence and Accidents of Software Engineering (1987) - https://news.ycombinator.com/item?id=15476733 - Oct 2017 (8 comments)
No Silver Bullet (1986) [pdf] - https://news.ycombinator.com/item?id=10306335 - Sept 2015 (34 comments)
No Silver Bullet: Essence and Accidents of Software Engineering (1987) - https://news.ycombinator.com/item?id=3068513 - Oct 2011 (1 comment)
"No Silver Bullet" Revisited - https://news.ycombinator.com/item?id=239323 - July 2008 (6 comments)
Interesting that Brad Cox's "revisited" article was the first one to get comments.
Thanks! Maybe the fact that the author is Frederick P. Brooks should be mentioned. From the domain name I was expecting something written by Bret Victor
HN typically goes by original title with some exceptions. It's ok to have to click through to figure out what the title is about (or by whomst), it's less ok to puff up a title by adding things to it which is usually why people slap on the author.
npm is a good example of the "buy vs build" idea, although npm modules are mostly free of course. Why write code when you can just call someone else's? But then you get problems like the left-pad fiasco
I always understood "Buy vs Build" as "buying software is cheaper than building it yourself". In the case of NPM, using most of the third-party modules out there (buying) is more expensive than writing the code yourself (building).
> using most of the third-party modules out there (buying) is more expensive than writing the code yourself (building).
This is probably true for "most of the modules" but not for "most of the modules that people actually use"
Yes.
I came from PHP, where everyone and their mom would reinvent the wheel.
Then I switched to Node.js, where no one would write their own code anymore.
The truth lies in the middle.
Can I write something like D3 until the dead line hits? Probably not.
Can I write something like left-pad until the dead line hits? Certainly.
Habits often mute common sense.
True. Though realistically, regarding left-pad, in other languages I would expect left-pad to be part of string formatting package.
There's a balance to be had for sure. If you bring in some module to a project it comes with costs of its own. That's not often well understood
Also there is probably a decent amount of “wont get fired for choosing X” where X is React, date-fns, jasmine etc. But those things have dependencies with dependencies and so on.
So in short: - Hire good people - Use off the shelf tools when it makes sense - Prototype rapidly - Develop systems iteratively
Using off the shelf tools was perhaps 50/50 when this was written. For a long time it was common for computers to come with company-specific OSes and programming languages. The high volume of the PC helped change that, but it was not overnight.
Yeah! And the whole thing about fourth generation languages fits in there too:
Using a modern high productivity like Python, Ruby, or Javascript will enable faster iteration and software growth.
Note: sometimes Rust or another lower level language will be more productive. Tool choice is hard and contextual.
I always understood it more like using existing libraries when they exists. So platforms with a huge choice of libraries with good support is the rational decision (which may be python, jvm ecosystem or js)
Higher productivity tools just shift the definition of what is complex.
Well sure, but that leaves behind a lot of opportunities for people who can’t or won’t pursue the complex cases.
Depends on what you mean by "good people". You don't have to hire only from the top 10%, whatever that may mean to you.
One of my main lessons from Brooks was always to put people in the roles that they are best suited for. In "The mythical man month" developers are not interchangeable. There are a number of different roles that needs to be filled, and different types of people are required to fill them.
This article could do with a modern retrospective and update. Who refers to "expert systems" now?
Brooks wrote a retrospective on it 9 years later, _"No Silver Bullet" Refired_, which appears as a chapter in the 20th anniversary edition of _The Mythical Man-Month_. Of course that is still 25 years ago now, so perhaps we should have a retrospective of the retrospective :-)
Refined rather than "Refired" ... though your version would make a very good title for the next retrospective IMO!
Oops, yes. I couldn't quite remember the title (my copy of the book is at home), and googling found people claiming both Refired and Refined, so I made my best guess...
On the contrary, I found the juxtaposition of archaic vocabulary alongside propositions which were not only true then but are still true today to be the most compelling part of his argument.
(Brooks, who referenced Goldstine/von Neumann and even Aristotle, certainly would not mind if we enjoyed his article in its context)
"ὥσπερ τὸ πῦρ καὶ ἐνθάδε καὶ ἐν Πέρσαις καίει"
But would those Persians know what a fire extinguisher is for today?
They still exist in rules engines. Plenty of bullshitty “AI” features are more like expert systems.
There is at least one that I know of:
https://danluu.com/essential-complexity/
For my own contribution, I think machine learning may be a good refutation of No Silver Bullet. It removes the need for the “essential complexity” of solving a problem. Like imagine trying to make GitHub copilot before machine learning. In Brooks’s world, the essential complexity of designing algorithms and data structures would indeed make this very difficult to do. But by using machine learning, you just let the neural net figure out the problem.
“and here it is!”
ML can help with managing complexity. It can also be a tool for creating additional complexity.
"Machine Learning: The High Interest Credit Card of Technical Debt" https://research.google/pubs/pub43146/
One thing he got very, very wrong: technological advances -- faster computers, better languages, better compilers, better editors -- have blown programmer productivity through the roof. Another is that "object-oriented" programming turned out not to be very important, and rather a distraction.
I think you're being unfair. Brooks did mention those kinds of advances, he just didn't consider them magical. For example, from Wikipedia's summary (which aligns with what I remember from my own reading):
> "While Brooks insists that there is no one silver bullet, he believes that a series of innovations attacking essential complexity could lead to significant improvements. One technology that had made significant improvement in the area of accidental complexity was the invention of high-level programming languages, such as Ada."
The core of "essential" vs "accidental" complexity remains very insightful. If, like Brooks argues, we are mostly hitting essential complexity now, no single programming language or IDE is going to provide a radically large improvement anymore.
"Productivity" is a tricky beast. What does it even mean? Programmers typing fast and getting code into production quickly? But what about the success and correctness of the systems they write? What about bugs and performance problems? What about incorrect or misunderstood requirements? Have we solved those?
> like Brooks argues, we are mostly hitting essential complexity now
I'm a bit skeptical about this. If you look at random backend code, you'll find lots of boilerplate, control structures, error handling, dealing with different data structures, performance optimizations. All of that I would consider random complexity. All of that is there only because computers can't figure these things out on their own.
Control structures as in "business logic" is essential complexity to me.
Random spaghetti code and boilerplate isn't, but Brooks' argument is that improving this, while definitely a good thing to strive for, won't result in a huge improvement in productivity, because the major roadblocks lie elsewhere.
I don't think he is arguing against minor wins, either. He just says we must focus on the major ones.
If you go into backend code from the typical place where "backend code" is a thing, it can be derived completely mechanically from the frontend. So all the work that goes into it is accidental, for a start.
Often that relation goes both ways, and the frontend code (that is even more complex) can be derived from the backend too.
That doesn't look very minor to me.
That's boilerplate code and I agree it can be completely automated.
I would argue it's a very convenient but minor thing, and that it doesn't truly change the complexity of writing code. It just makes it less dreary. I would also argue it's a win that it can be automated, and I think Brooks would agree!
I don't think he was arguing small wins don't matter.
Boilerplate in itself is not complexity (eg import statements), it’s just tedious and unergonomic. In fact it can reduce complexity, again with example of imports being a better alternative than global symbols.
Coupled boilerplate on the other hand is complexity, where if you change x in front end you also must change corresponding x in backend.
You're using a very specific notion of boilerplate and complexity. I mean, I've never heard of imports being described as boilerplate...
control structures and error handling sound like essential complexity to me, as that's effectively business logic / system requirements.
I might agree that error handling is business logic in the broadest sense, but control structures certainly are not. Unless there is an explicit business requirement like "use a nested for loop to iterate through an employee array", control structures are just implementation details.
The logic being expressed by the control structure is essential complexity though.
There are better and worse ways to go about it, i.e. "accidental" complexity, but the core logic is irreducible.
It is, exactly, the distinction between accidental and essential complexity that has not panned out.
Productivity is about getting solutions successfully deployed to production. Things that interfere with that may be "essential" or "accidental", but the distinction turns out not to matter much.
One of Dan Luu's better essays covered this.
> Things that interfere with that may be "essential" or "accidental", but the distinction turns out not to matter much.
I obviously disagree. I think it matters a lot, especially because it allows us to detect that trap programmers and hackers often fall prey to: hyperfocusing on minor technical details because they are cool to think about without regards to the actual gains.
That's not to deny a lot of improvements in the past decades actually made programmers' quality of life tremendously better.
Dan Luu is a very interesting essayist and I find myself nodding in agreement with a lot of what he writes. He's made it his business to go against the grain of most "universally held truths" (a great chunk of his essays are about debunking commonly held beliefs about tech and science), but in doing so I think he tends to overstate his case. In this case, I think he is right but also overstating his case, and Brooks remains as relevant as ever. Luu's 2022 addendum is interesting, because it certainly cuts both ways -- not just in the way Luu would want!