All bullet points under "Why Rust?" actually suggest that it would be much better to gradually rewrite all critical parts of the project in ATS, thanks to it being a frontend to GCC. Firstly, it's a much more powerful frontend than the current Rust. Secondly, I'd expect it to be less disruptive than a complete compiler switch.
If a rewrite is warranted, and despite Rust zealotry being annoying at times I think it has fair points, I think going for something even more hipster than Rust would be a mistake.
I wouldn't even call Rust "hipster" anymore. It's already been embraced and rolled out to production by big companies working on hard techinical problems (Cloudflare, AWS, Discord), as well as the crypto community.
I think the reasons from steering away from hipster languages for these critical projects are roughly the same ones we use to justify not using hipster crypto for critical, real-world applications
I also would call Rust the new normal for fast secure code.
I still do not understand. Ada is not exactly new. Have you ever seriously considered Ada/SPARK? Might want to give https://www.adacore.com/uploads/books/pdf/AdaCore-Tech-Cyber... a read, or even https://blog.adacore.com/, or my previous comments on Ada. I called Ada the programming language for writing "fast secure code" for quite a long time now, a "better C" in that regard.
Ada just doesn't have the steam behind it.
It certainly lacks the hype, and judging by the comments on here, many people still have loads and loads of misconceptions of the language, negative ones, unlike say Rust's "unsafe code does not affect the rest of the program", for example, which I would call a positive misconception. And of course thus, it also lacks libraries. This is really my biggest problem with it. :/
Rust isn't nearly popular enough to be referred to as "the new normal" in any field or context. One would have to be in a very closed off bubble to think it is. If they haven't made some big strides with having it certified for use there are entire industries where you can't even use it and get certification, if I understand correctly.
I wouldn't really call Discord a hard problem. It's web chat.
You might have missed the "technical" qualifier.
Twitter isn't an especially hard problem either.
I didn't miss the qualifier. Discord is a chat client with very restricted servers. It's really not that hard as technical problems go.
Please, do tell us what kind of exotic problems you work on, that you think massive scale real-time communications is not a hard technical problem
We're eager to know!
No, that's stupid. I couldn't build Discord. My point is that as relative problems go, I don't see Discord as a big one. I might be wrong but as far as I can tell it's a solved problem and has been for over a decade. I'd wager you could build it with an incompetent team that had domain expertise.
AWS is orders of magnitude more complicated than Discord. Frankly, so is World of Warcraft.
What does Discord do that wasn't solved in 2005?
> AWS is orders of magnitude more complicated than Discord. Frankly, so is World of Warcraft.
AWS maybe, insomuch as "orders of magnitude" can mean as little as "100x". But still, unconvinced. There's something like 200 AWS services, and Discord on the whole is certainly more than twice as complex as the average AWS service.
World of Warcraft being >100x more complex than Discord doesn't track.
> What does Discord do that wasn't solved in 2005?
Perhaps you are unaware that Discord has (pretty high quality!) voicechat and livestreaming? Certainly much better than anything that existed in 2005.
> What does Discord do that wasn't solved in 2005?
Discord was pretty much unrivaled in voice-chat quality for a long time, and to this day it's only "real" competitors in technical prowess in this space that come to mind are Apple's Facetime and Google's Duo.
Both Facetime and Duo have bespoke algorithms for this kind of thing: Duo for example leverages SOTA ML to correct audio gaps in the call[0], while Facetime leverages ECN on their custom adaptive bitrate algorithm so that the call is more resilient to noisy networks (couldn't find a good reference for this second one).
Keep in mind that both Google and Apple have highly-qualified teams working exclusively on networking protocols, and yet still there was a time a while ago where Discord pretty much had the best call quality experience around (GMeet is still wonky to this day).
The only thing about this that was "solved" in 2005 is shoveling packets over UDP, or at most, if you wanted to be really fancy, just shoveling them over SCTP instead.
I think you're severely underestimating how much knowledge of networks it takes to build something to par that level of quality. The only thing you can point at back in 2005 is Skype, and they do not come even close. You also seem to be equating difficulty with complexity, which even though they are at times correlated, is still a false equivalency.
[0] https://ai.googleblog.com/2020/04/improving-audio-quality-in...
From a cursory search, it looks like ATS is implemented as a "compile to C code" language. This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.
Also, from what I can tell of its mailing list, ATS is a pretty dead project: https://sourceforge.net/p/ats-lang/mailman/ats-lang-users/ shows only announcements post-2015.
It's a google group! Not dead at all, though it is... ATS :)
It’s not more concerning than a language that is compiled to assembler. It all depends on the quality of the compiler/transcoder.
> This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.
if it were a new project I'd agree, but if there's an existing C codebase, it already contains all these UBs and the goal of the rewrite is to grow a formally-verified critical core functionality in ATS that would still inline the rest of the project intact. This way the formally verified part of the project can grow naturally through small iterative steps without major disruptive changes to the tooling and supportive infrastructure.
People who are familiar with C can acquire basic ATS patterns pretty quickly (within a month or so), and the same is applicable to those who are familiar with ML-family languages.
> This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.
I think it's worth pointing out that you can't always get away from those undefined behavior situations even if you skip C all together. Rust, for instance, has some internal/implementation details I've run across that are just C undefined behavior details leaking out of the LLVM IR instructions they use to implement the language, since those instructions have the same undefined behavior semantics as the corresponding C code they're used for.
Not always, but a lot of the time. There are also a whole bootload of things that are UB in C but have defined semantics in Rust.
Sure, I was mostly responding to the "compiles to C vs. directly to GCC/LLVM". I don't think the difference is actually all that relevant in terms of avoiding undefined behavior, a lot of the undefined behavior situations still need to either be worked around (or accepted) when generating the IR - it's not as simple as "skip C, UB is gone".
It's far from dead, a new version of the compiler is being developed on top of ATS2 (https://github.com/githwxi/ATS-Xanadu). ATS2 is being updated too (https://github.com/githwxi/ATS-Postiats), the latest stable release happened last December:
https://github.com/githwxi/ATS-Postiats/blob/master/CHANGES-...
https://sourceforge.net/projects/ats2-lang/files/ats2-lang/
Does Rust ultimately not compile to machine code? Is the underlying hardware proven? You are ultimately going to get your code punted through untyped data lines and unproven logic units.
Rust has the good sense to keep all its Undefined Behavior behind unsafe blocks.
A language that compiles to C does not inherit all of C's problems, and doesn't "punt" on it for undefined behavior. I mean, after all, Rust compiles to Assembly, which is even unsafer than C, but Rust is still considered safe.
A compiler has greater ability to enforce invariants that make the resulting code safe. The issues with C result from faulty humans; running through a program gives an opportunity for those issues never to occur to begin with.
How is assembly less safe than C? C is full of undefined behavior which might be interpreted differently by different compilers, whereas (maybe with the exception of some unusual circumstances) the behavior of assembly language for major instruction sets is 100% documented.
I see that you have not coded in assembly language.
I haven’t; I’ve only read it a bit. I could be wrong about a relative lack of UB compared to C - do you have examples?
When you code in assembly language, you discover how incompletely documented every chip is. Any detail of behavior not documented might change in a later chip, and is thus UB.
Makers of compatible chips are obliged to keep a collection of systems with all the extant chip designs to run experiments against, and to maintain their own documentation of discovered but unspecified commonalities that users' code may have come to depend on.
Each released chip, and each stepping of each release, has its own unique set of bugs-- deviations from documentation, from observed behavior of previous chips, and of chips yet to be released.
Writers of assembly code need a compendium of behaviors, documented or not, that cannot be relied on for the set of release platforms. Many such bugs depend on an ill-characterized history of processor states, making them, often, (more or less) reliably reproducible only with a particular program that was discovered to evoke each. Often the behavior noted is seen only once in a million runs.
An example on Intel is the popcount instruction that runs very slowly, sometimes, in chips released up to a few years ago. The instruction appeared along with ...SSE3?, and had the bug until it was noticed and fixed in a subsequent chip. The bug is the reason Gcc uses the same register for source and destination when it emits this instruction. Intel's ISA manual does not mention this behavior.
Thank you for the details. I guess I had an overly rose-tinted view of the quality of Intel documentation.
> A language that compiles to C does not inherit all of C's problems, and doesn't "punt" on it for undefined behavior.
ATS avoids some things; you shouldn't be able to do the silly memory-unsafe parts.
Stuff like dividing by zero is permissible by default but can be avoided.
> ATS
For anyone else who was unfamiliar: https://en.wikipedia.org/wiki/ATS_(programming_language)
Is ATS really suited to being an industrial language? How do its standard library, build tools, and library ecosystem compare to Rust's?
Admittedly, if you're switching from C, you start with a pretty poor story about all that. But at least that story is widely and thoroughly understood.
ATS has an optional garbage collector. It seems the standard library has two implementations of each container type, one which uses GC, and one which uses linear types, so does not need GC, but is more demanding to use. That is pretty cool! What is the situation with other libraries? Is there a convention of assuming GC can be used, or that it can't, or of supporting both?
> How do its standard library, build tools, and library ecosystem compare to Rust's?
as ATS is a frontend to GCC, it means that the standard library can be any battle-tested C standard library. On top of that there can be a formally-verified interface that calls underlying C API. Here's an example - https://bluishcoder.co.nz/2012/08/30/safer-handling-of-c-mem...
There's also a safe Prelude and common data structures, implemented in ATS:
http://www.ats-lang.org/Libraries.html
> Is there a convention of assuming GC can be used, or that it can't, or of supporting both?
afaik there's no officially defined convention, but you can have a look how this switch may look like in practice in this video - https://youtu.be/Xkg_EmERYRE?t=1742
Back when it was on the PL shootout, ATS was consistently in the top five for performance. However, it was also consistently the worst in program size, comparable to Ada (but even more loquacious). Between that and the lack of existing uptake it would probably be difficult for QEMU to adopt.
Article author here. Thanks for sharing ATS. I will take a look at it.
Hey, no worries!
Personally I think this playlist provides a nice set of real-world examples implemented in ATS:
https://www.youtube.com/watch?v=dsILVOqYCU0&list=PL6BIXG1a4e...
And this series of blogposts is a pure gold:
https://bluishcoder.co.nz/tags/ats/
And here's the official language introductory resources:
Language Tutorial - http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...
Language Features Overview - http://ats-lang.sourceforge.net/DOCUMENT/ATS2TUTORIAL/HTML/A...