> A programming language is more scalable if an engineer unfamiliar with a code base written in it produces correct code more quickly.
> Scalability is often at odds with peak effectiveness, the maximum effectiveness of an engineer who is intimately familiar with the codebase...
I would think we would want to use the term "scalability" in this kind of context to say something more like "a programming language is more scalable if it makes it easier to have large numbers of engineers all working on features of a single app concurrently without stepping on each other's work (even if they have to go to great effort to learn how to do so)"?
(edit addition 25 minutes later:) As a maybe overly-trivial example, it can be really irritating to work with other developers in an object-oriented programming language that requires all the fields or methods of a class to be declared in one place, as you can end up with constant merge conflicts. If you can create a new file and declare fields and methods that extend an existing class -- preferably with the fields in that new file being scoped to only being accessible to methods in that new file -- that can be a massive help (even if it makes other specific aspects of understanding the functionality of the class a bit more difficult).
> a programming language is more scalable if it makes it easier to have large numbers of engineers all working on features of a single app concurrently without stepping on each other's work
This is optimizing for a wrong (minor) problem.
As the application grows, both accidental and essential complexity grows. More and more of engineers' time is spent analyzing existing code, and they typically spend less time writing new code. The limiting factor in large/old codebases (which is where the word "scalability" comes into play IMHO) is the inability of even experienced engineers to understand aspects of the codebase.
This is why I think e.g. Java has been so successful in the "big & complex" space. Java is annoying to write, but easy to read. I mean, some more conciseness would make it easier to read, but it's still quite OK. Very importantly, it's OK to read even if the dev writing it wasn't good / didn't observe best practices etc. I often have to deal with code written 20 years ago, and you can't change the past ...
Regarding the merge conflicts - they happen, most of the time causing only minor problems, sometimes it's a bigger deal. Some programming styles can help you avoid some minor merge conflicts, but I doubt they will be effective in avoiding the real troublesome merge conflicts (which go far beyond introducing a new field).
The other side of the coin for something like Clojure (lisps, really) allow tiny teams to manage large domains - so the team can't scale to a large number of programmers, but a 5-man army is viable because that is a small enough team to curate the individuals members for quality and they get a lot done.
But the author's rankings look extremely reasonable. All companies have management problems and some languages cope better with that than others. Another way of interpreting this is "What costs does the language impose on programmers doing something novel" - usually novel is a bad sign for long term success.
Is that a virtue of the language (Clojure/lisp/any other) or a virtue of the 5 experienced programmers?
I ask because I’ve never delved deep into any language of the lisp family but I have seen that small experienced teams tend to produce good work regardless of the stack they choose. My guess is the recipe for good outcomes is: experience with the chosen stack + experience working together + small-ish team.
The technical influences of language are, theoretically, very small. As far as I can tell anything can be built in anything (evidenced by the remarkable number of great applications written in JavaScript of all languages).
But Clojure is a language that supports [0] that sort of team better than Java or Python (I won't say Rust or Go because it isn't really a first choice for systems programming). The Clojure community also has a cultural expectation that teams will be small and highly skilled, for obvious reasons.
[0] I don't see how this sort of statement could ever be anything other than opinion, so I won't try to justify the assertion.
Reading the article it's hard to deduce what the author is talking about developing, what type of software, is it a web application, is it a systems application, is it some parallel processing application that requires multi-threaded code?! but when looking it appears he's talking about web application development because of the mention of Ruby on rails.... also the use of the word scalability to describe readability of code doesn't make a whole lot of sense. also, the reference of programming languages misses the most commonly used ones (php), which are used for mainly web development, and are primarily the easiest to develop.. obviously if we're talking about systems level application development then yeah rust and go makes sense but if we're talking about web application development then I think that they take a lot of time to develop an application in and then although it can be done it's not that popular compared to other ways of doing it.
Same thoughts...but the authors definition of "scalability" seems to relate to making employees disposable and enabling cross team collaboration.
Bu that definition Java is probably the best just due to it's presence in teaching and while I prefer python, better threading support ...
but I do look forward to the rise of LOGO!!!
? FORWARD 20
? RIGHT 90
? FORWARD 20
? HIDETURTLE
?
(Really items like the available talent pool and what languages align best with business needs are better metrics, I'll put on my flame proof pants and say language choice is an architecturally insignificant implementation detail outside of concerns like the above)
An experienced engineer will tell you: it doesn’t matter what software. It doesn’t even matter what kind of application. Actually, it doesn’t matter what it is at all.
Keeping things simple keeps them scalable. Cleverness and magic are antithetical to that goal.
> Scalability is often at odds with peak effectiveness
This is the astute point of this article. It transcends the domain at hand: interchangeable software developers. It basically hits any professional talent pool. It even applies to various tools, machines, products we use in every day life.
The question one has to ask is what price one is willing to pay for scalability. And whether you’re paying for something you really need or not.
I have programmed in many languages over a 2 decade long career. C / C++/ C# / Java / Scala / Python / Rust / Go / lisp / JavaScript, and probably more that I have forgotten.
I agree with the framing that scaling a language is about scaling to more engineers easily.
Having said that, you can write scalably in any language, and write unscalably elsewhere. This is a bigger factor on scalability overall by far, than the language itself.
For example, I can make unmaintainable macro magic in rust. I can avoid dynamic pitfalls by using pydantic effectively in python.
Anyway, my 2c is speed to delivery of features matters more than scalability as defined here. Again, that depends more on technical components that have been invested in, rather than the selected language.
> If I wanted to select viable candidates from that list, I would probably pick between Rust, Go, or Java/Kotlin...I arrive there with a simple heuristic, as all of these languages are statically typed and considered mainstream, almost boring.
The scroll bar appears to be missing from the article. In case the author is reading the comments: was this deliberate? If so, I'd be really interested to hear the thought process behind the design decision.
This is the secret to end all secrets: anything is scalable if you fucking keep it simple.
No magic. No configuration. No cleverness. No DRY. Will it be beautiful? Not at all. Look at LLMs. We can make things that talk like people using ugly fucking matrix math and recursion so long as we keep it simple. We can scale it to infinity if we keep it simple.
It doesn’t matter what it is. QBASIC, JavaScript, Rust, jet engines, pneumatic circuits, CPU cores, manufacturing facilities… the list goes on.
Keep it so that it can be reasoned through by a single person and you can scale it forever.
The problem is that a lot of simple processes combined are no longer simple. So you need patterns which are general enough to write less processes, without being so general that developers write processes which are unintuitive and therefore complex by themselves.
My opinion is that tooling (perhaps with or without LLMs) can achieve the best of both worlds, by creating a simple query/projection (something in between) of complex code. Then the developer can get and mutate whatever properties they want from the code, without seeing unrelated information, and seeing seemingly-not-but-actually-related information presented in a concrete (simple) way.
> A programming language is more scalable if an engineer unfamiliar with a code base written in it produces correct code more quickly.
> Scalability is often at odds with peak effectiveness, the maximum effectiveness of an engineer who is intimately familiar with the codebase...
I would think we would want to use the term "scalability" in this kind of context to say something more like "a programming language is more scalable if it makes it easier to have large numbers of engineers all working on features of a single app concurrently without stepping on each other's work (even if they have to go to great effort to learn how to do so)"?
(edit addition 25 minutes later:) As a maybe overly-trivial example, it can be really irritating to work with other developers in an object-oriented programming language that requires all the fields or methods of a class to be declared in one place, as you can end up with constant merge conflicts. If you can create a new file and declare fields and methods that extend an existing class -- preferably with the fields in that new file being scoped to only being accessible to methods in that new file -- that can be a massive help (even if it makes other specific aspects of understanding the functionality of the class a bit more difficult).
> a programming language is more scalable if it makes it easier to have large numbers of engineers all working on features of a single app concurrently without stepping on each other's work
This is optimizing for a wrong (minor) problem.
As the application grows, both accidental and essential complexity grows. More and more of engineers' time is spent analyzing existing code, and they typically spend less time writing new code. The limiting factor in large/old codebases (which is where the word "scalability" comes into play IMHO) is the inability of even experienced engineers to understand aspects of the codebase.
This is why I think e.g. Java has been so successful in the "big & complex" space. Java is annoying to write, but easy to read. I mean, some more conciseness would make it easier to read, but it's still quite OK. Very importantly, it's OK to read even if the dev writing it wasn't good / didn't observe best practices etc. I often have to deal with code written 20 years ago, and you can't change the past ...
Regarding the merge conflicts - they happen, most of the time causing only minor problems, sometimes it's a bigger deal. Some programming styles can help you avoid some minor merge conflicts, but I doubt they will be effective in avoiding the real troublesome merge conflicts (which go far beyond introducing a new field).
I have always used it to mean how large a codebase you can create without complexity growing out of control (of a fixed sized team).
But I do like your usage too. We do need new words here.
The other side of the coin for something like Clojure (lisps, really) allow tiny teams to manage large domains - so the team can't scale to a large number of programmers, but a 5-man army is viable because that is a small enough team to curate the individuals members for quality and they get a lot done.
But the author's rankings look extremely reasonable. All companies have management problems and some languages cope better with that than others. Another way of interpreting this is "What costs does the language impose on programmers doing something novel" - usually novel is a bad sign for long term success.
> allow tiny teams to manage large domains
Is that a virtue of the language (Clojure/lisp/any other) or a virtue of the 5 experienced programmers?
I ask because I’ve never delved deep into any language of the lisp family but I have seen that small experienced teams tend to produce good work regardless of the stack they choose. My guess is the recipe for good outcomes is: experience with the chosen stack + experience working together + small-ish team.
The technical influences of language are, theoretically, very small. As far as I can tell anything can be built in anything (evidenced by the remarkable number of great applications written in JavaScript of all languages).
But Clojure is a language that supports [0] that sort of team better than Java or Python (I won't say Rust or Go because it isn't really a first choice for systems programming). The Clojure community also has a cultural expectation that teams will be small and highly skilled, for obvious reasons.
[0] I don't see how this sort of statement could ever be anything other than opinion, so I won't try to justify the assertion.
Reading the article it's hard to deduce what the author is talking about developing, what type of software, is it a web application, is it a systems application, is it some parallel processing application that requires multi-threaded code?! but when looking it appears he's talking about web application development because of the mention of Ruby on rails.... also the use of the word scalability to describe readability of code doesn't make a whole lot of sense. also, the reference of programming languages misses the most commonly used ones (php), which are used for mainly web development, and are primarily the easiest to develop.. obviously if we're talking about systems level application development then yeah rust and go makes sense but if we're talking about web application development then I think that they take a lot of time to develop an application in and then although it can be done it's not that popular compared to other ways of doing it.
Same thoughts...but the authors definition of "scalability" seems to relate to making employees disposable and enabling cross team collaboration.
Bu that definition Java is probably the best just due to it's presence in teaching and while I prefer python, better threading support ...
but I do look forward to the rise of LOGO!!!
? FORWARD 20
? RIGHT 90
? FORWARD 20
? HIDETURTLE
?
(Really items like the available talent pool and what languages align best with business needs are better metrics, I'll put on my flame proof pants and say language choice is an architecturally insignificant implementation detail outside of concerns like the above)
An experienced engineer will tell you: it doesn’t matter what software. It doesn’t even matter what kind of application. Actually, it doesn’t matter what it is at all.
Keeping things simple keeps them scalable. Cleverness and magic are antithetical to that goal.
> Scalability is often at odds with peak effectiveness
This is the astute point of this article. It transcends the domain at hand: interchangeable software developers. It basically hits any professional talent pool. It even applies to various tools, machines, products we use in every day life.
The question one has to ask is what price one is willing to pay for scalability. And whether you’re paying for something you really need or not.
I have programmed in many languages over a 2 decade long career. C / C++/ C# / Java / Scala / Python / Rust / Go / lisp / JavaScript, and probably more that I have forgotten.
I agree with the framing that scaling a language is about scaling to more engineers easily.
Having said that, you can write scalably in any language, and write unscalably elsewhere. This is a bigger factor on scalability overall by far, than the language itself.
For example, I can make unmaintainable macro magic in rust. I can avoid dynamic pitfalls by using pydantic effectively in python.
Anyway, my 2c is speed to delivery of features matters more than scalability as defined here. Again, that depends more on technical components that have been invested in, rather than the selected language.
> If I wanted to select viable candidates from that list, I would probably pick between Rust, Go, or Java/Kotlin...I arrive there with a simple heuristic, as all of these languages are statically typed and considered mainstream, almost boring.
Rust is mainstream and boring?
Using only c api outside and whatever language you want inside modules is the most scalable way.
The scroll bar appears to be missing from the article. In case the author is reading the comments: was this deliberate? If so, I'd be really interested to hear the thought process behind the design decision.
I have a scroll bar on mine. I am using the latest version of Firefox (124.0.2). It is the exact same scroll bar that is on this website, Hacker News.
If you want to optimize for hiring the median engineer, this is great advice.
This is the secret to end all secrets: anything is scalable if you fucking keep it simple.
No magic. No configuration. No cleverness. No DRY. Will it be beautiful? Not at all. Look at LLMs. We can make things that talk like people using ugly fucking matrix math and recursion so long as we keep it simple. We can scale it to infinity if we keep it simple.
It doesn’t matter what it is. QBASIC, JavaScript, Rust, jet engines, pneumatic circuits, CPU cores, manufacturing facilities… the list goes on.
Keep it so that it can be reasoned through by a single person and you can scale it forever.
Writing "simple" code in itself isn't so simple.
The problem is that a lot of simple processes combined are no longer simple. So you need patterns which are general enough to write less processes, without being so general that developers write processes which are unintuitive and therefore complex by themselves.
My opinion is that tooling (perhaps with or without LLMs) can achieve the best of both worlds, by creating a simple query/projection (something in between) of complex code. Then the developer can get and mutate whatever properties they want from the code, without seeing unrelated information, and seeing seemingly-not-but-actually-related information presented in a concrete (simple) way.