I get that not everybody likes systemd, but a lot of the criticisms seem misguided and looking at sysvinit with role-colored glasses.
For example, people are complaining about logind being included, but forget that consolekit was unmaintained before that. Yes, there's parts to systemd that not everyone will use, like container support, but in that case you can pretty safely ignore that use case.
For me, systemd has provided a much more consistent way of managing services. There's a tightly defined service definition format that allows one to comfortably modify any 3rd party service as well, (unlike random, variable-quality shell scripts), relatively small amount of commands to learn that work consistently across services, mounts, timers etc. There's also much better handling of modern hardware, hotplugging & the like.
I can't say I miss sysvinit. Also, stuff like [1] from the anti-systemd camp doesn't inspire confidence.
Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
It is easier to configure and is convenient to have one holistic system in a lot of ways, but...
They pushed for Gnome to have a hard dependency on it so that everyone would be _forced_ to use it instead of just letting it be accepted by merits.
They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
I don't think the code is high-quality, but even more than that I have a problem with the people being controlling, dismissive, and deceptive.
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
Additionally, this is not likely to stop, since they took an init system that was (mostly) written in a memory-safe language (bash) and rewrote it in a memory-unsafe language (C++), thus enabling a whole class of systemic security issues that will keep popping up in the future over and over again as well. This is one of the design decisions behind systemd that irks me the most.
> they took an init system that was (mostly) written in a memory-safe language (bash) and rewrote it in a memory-unsafe language (C++)
While Bash is memory safe, it is also very error prone to write and the potential of shooting yourself in the foot with it is rather big, especially with bigger scripts and considering these are executed mostly with root privileges, bash is not an ideal choice either.
I'd prefer if systemd was a Rust project for sure, but they went with C, presumably to remain consistent with the kernel, GNOME and most lower-level Linux system software, (also, Rust was pre-1.0 at the time).
Memory safety is not the alpha and omega of programming languages. People do not write million lines of code applications in bash, for a reason, trade-offs are to be made.
That said I agree they should have picked Go instead /s.
Sure, there are other concerns and bash is not an ideal language, but memory safety is responsible for quite a large number of security flaws, since its lack turns bugs into code execution vulnerabilities on a regular basis.
I don't really have an opinion on what language should have been picked instead, though OCaml may be a good candidate.
I think I know what my next Erlang project's gonna be. Erlang-as-PID-1, here we come!
EDIT: or maybe as PID 2, per https://github.com/omisego/ewallet/issues/108 , though I'd be interested in the idea of writing an OTP application that can reap zombies and forward signals.
> People do not write million lines of code applications in bash, for a reason, trade-offs are to be made.
Isn't that basically what SysVinit and a good-sized chunk of the management tooling in Linux is?
To their credit, Bash is a _weird_ language, but it's not going to corrupt memory when it crashes, and its failure modes are pretty well understood by the distro maintainers writing those scripts.
Those are all not million lines of code applications, but that only speaks in favor of a more traditional Unix userspace design. With modularity on the process level you don't need a million lines of code to herd services. Why an init system should a million lines of code is a mystery to me.
Ocaml parallelism/concurrency story was not that good - but Haskell has just got its fancy new epoll based I/O manager (100000+ lightweight threads) around that time, with the 7.0.1 release (16 November 2010)
They could have even take inspiration, imagine: typed, functional, monadic init files - "systemd: avoid success at all costs" :)
The topic is memory safety, not concurrency support, FWIW.
Lua would have been a good candidate: memory safe, coroutine to structure the code in a concurrent-ish way if needed, and you have a configuration file parser for free (just use Lua as the configuration language, it was its very first purpose after all).
That's nice in theory, but where are all the people who know Ocaml to write it and contribute to it ? I might be clueless but I can't really name any significant projects written in it, which might imply something about the number of people using it and being able to work on and contribute to something as large and important as systemd.
> Ocaml parallelism/concurrency story was not that good
I wouldn't expect CPU-bound multithreading to be much of a concern in an init/rc system. And resource thriftiness and predictability would likely be a bigger concern.
Rust would have been more appropriate. Something like SystemD at such a low level in terms of hardware and service management should not be done in a language with a garbage collector imho.
I understand that... Go still isn't an appropriate language for SystemD, also, when did SystemD really take hold? Lots of projects will shift underlying options with time.
The only thing that I can think of for that 2019 Go that would be a problem for this use case is binary size. None of the rest of the usual complaints would be stoppers or even that big a deal, and most of them, I'd pay to have a memory-safe language being used at that level. Init systems are a type of code that we don't have a good word for, but that I tend to end up in a lot, the code that isn't CPU intensive, or disk intensive, or RAM intensive, etc., but is all just logic and correctness rather than any of that. So "but Go is GC'ed!" isn't particularly relevant, for instance, because an init system pretty much ought to settle into a steady state and not be executing at all, on average, so who cares about a handful of microsecond GC pauses that may or may not occur at startup time?
Rust might be better now; my primary concern would be community size and whether I was limiting my contributor base, which Go also has as a concern. In both cases though I'd take it over C/C++ being used on such a critical project at this juncture.
I understand there's a much more limited subset of developers for Rust or even Go vs. C/C++. Also, I recall hearing about Rust before systemd, but am not sure. Today, it would probably be my first choice. Not that go is a bad choice, I just don't think it's a good spot for it. I also agree that either are probably quite a bit better than C/C++ options.
I'm not against go, other than gc and some size considerations. Given that a lot of k8s infrastructure for the likes of CoreOS and similar are written in it, it's definitely not a bad option. I think that today, and more so in late summer as async/await syntax settles, that Rust should be a first consideration for any low-level system code.
It doesn't execute low level code in terms of hardware. It's a init system, all it (should do) is create a dependency graph and initialize the services in the correct order.
Many init systems have been created in bash, and I'd hardly call that a low level language.
If you run or write that command into script without checking what DIR variable contains, the problem is in your lack of experience with shell, not the language choice. Also nowadays, rm itself has --preserve-root as default, so this won't delete your root fs.
By the same logic, if an app written in C has a security issue due to a null pointer or buffer overflow, the problem is clearly the coder's lack of experience with C.
Actually yes, because it is possible to write correct C, and because they could have chosen other language less conducive to those pitfalls. But your analogy doesn't hold, because in case of shell, the situation is different: the pitfalls are easy to avoid, and for unix admin tasks, there is no compelling alternative to shell.
This is true if you are running the GNU userland. It's not necessarily true with other userlands that are often run on top of Linux. In a discussion touching on systemd, it strikes me as a little surprising that somebody would rely on a quirk of another replaceable component. ;)
I’m sure the irony is not lost on you regarding the fact that systemd can not possibly run on an alternative platform, thus the situation you just ascribed is not “better” with systemd.
Sure. That wasn't in favor of systemd, though? I comfortably write code against the GNU userland and I use systemd where it crops up. Just not a big deal to me; if somebody wanted to use my stuff elsewhere I'd just say "pull requests welcome!". I found the inconsistency striking, that's all.
Checking for content of a variable before passing it to rm is a basic instinct that should be done in all shell scripts. The comment about --preserve-root is just a reminder that the problem of deleting / is very rare.
Not just -u - always <set -euo pipefail> unless you have a good reason. Fail on non-zero exit code outside of some special cases, undefined variables, and non-zero exit codes in a pipeline. And always use shellcheck, shellcheck is too good not to use tbh.
He is saying that Bash can also cause catastrophic behaviors by an as simple mistake as unsetting $DIR. C++ at least reject to compile when a variable is not set.
Any language can cause a catastrophic scenario like that. It's not a feature of the language. Not even the goal of the language to protect from something like rm -rf. The variable could have been set to empty string and C++ would just comply with what the programmer wrote (not what they meant, but that's a completely different thing).
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
This seems like it needs some citation. There certainly have been bugs. I think it's legitimate to talk (with specifics!) about cultural issues with systemd's development and its impact on security. But I don't see that this statement is particularly well supported. In particular I genuinely don't know what you're referring to with respect to ignored/suppressed bugs and CVEs.
I mean, systemd is a lot of code and makes up a big chunk of the system plumbing these days. Obviously it's going to make up a big chunk of the bug budget. People don't get upset about flaws in someone's random cgroup launcher or encrypted block device manager or daemon wrapper not because those projects don't have bugs, but because basically no one uses them. Because basically everyone is using systemd!
I think that's valid enough. Not sure it rises to the level of "OMG they ignore bugs" implied by the original comment though. I mean... they took the patch and fixed the bug, right? Poettering was just being a jerk about whether it deserved a CVE.
(Also... gotta be honest that's sort of a vanity CVE. Note the complete lack of details in the report itself, it never saw an eyeball after it was filed. It's valid enough, but frankly I'm with Poettering on the specifics here: if we filed a CVE for every theoretically-exploitable-but-unexploited-until-fix-released crash bug the system would be beyond useless.)
I think that's valid enough. Not sure it rises to the level of "OMG they ignore bugs" implied by the original comment though. I mean... they took the patch and fixed the bug, right? Poettering was just being a jerk about whether it deserved a CVE.
Poettering is dismissive of nearly every bug. Remember when systemd would default to starting units as root when it couldn't parse the username? We're not talking about some minor piece of software like a sound multiplexing daemon, we're talking about the core userland piece of every Linux install. Bug reports on something that central should be taken seriously and with a minimum of snark.
I'm with Poettering on the specifics here: if we filed a CVE for every theoretically-exploitable-but-unexploited-until-fix-released crash bug the system would be beyond useless.
All the more reason to reduce the attack vectors and ensure that your code is well thought out, and coded defensively. That you have too many security issues to generate CVEs for each and every one is a problem in and of itself. Ten years ago how many people thought that all of these speculative execution side channel attacks were possible or practical?
This is always the case with any new software. Things may get a bit worse before they get better. Honestly Systemd addresses the complexities of a modern system in the simplest fashion but no simpler, and that is the key. It is in the same spirit as launchd on macOS. It works well overall and end up being more portable and consistent in the long run. Hatred for it seems to come mostly out of dogma, something to be avoided in this age.
> addresses the complexities of a modern system in the simplest fashion but no simpler
I have a serious contention with the statement that systemd is the simplest possible solution for this problem. Especially since it involves not only service management, but user logins, a data bus, logging management, file system mounting, initd, and a host of other things.
Are all of those pieces necessary? Arguably yes. But not all in one system(d).
Thing is, though, most of those things are different systems. systemd-logind systemd, systemd-udevd, and journald (and others) are different systems in the same project. They're a bit tied together (through dbus and I'm not sure what else), but that doesn't mean they're the same system.
> This is always the case with any new software. Things may get a bit worse before they get better.
So why should I currently opt for the worse solution? Is the overall design of it promising?
> Honestly Systemd addresses the complexities of a modern system in the simplest fashion but no simpler, and that is the key.
What complexities does it address? I lost track of everything an init system is apparently responsible for.
> It works well overall and end up being more portable and consistent in the long run.
Since systemd only runs on Linux APIs, I can't figure out what this even means. There are init systems that run perfectly fine in plain POSIX environments. systemd is not one of them.
I'm not going to discuss the merits of portability or what "typical developers use". I'm questioning the idea that systemd is somehow more portable than its alternatives. That belief can only be rooted in plain ignorance of the actual situation. It has absolutely nothing to do with what typical developers use, and to that end your comment is a pointless derailment.
What I mean is that there are init systems that only require POSIX compliant APIs and as such are portable across systems that implement POSIX, e.g. GNU/Linux, BSD, Solaris, AIX...
systemd requires (in addition to POSIX) Linux APIs and is therefore not particularly portable by any reasonable definition of the word. You can use it if you are running a Linux kernel, and that's it.
> What I mean is that there are init systems that only require POSIX compliant APIs
Those are irrelevant as they can't even do basic tasks like shutting down the computer or restarting it.
Feel free to find a POSIX API for Linux' reboot syscall.
Some systems also require mounting additional filesystems, again not possible in a world limited to POSIX interfaces.
launchd doesn't break nohup. It doesn't have any opinions about processes started from shells. systemd does, which is completely inappropriate scope creep.
I agree with the systemd developers on this one. It's weird that processes started by users when their logged in by don't end when they log out unless they voluntarily terminate when sent a signal.
And nohup was always a hack. Now that we actually have a functioning user-level service manager keeping processes alive by throwing them at PID 1 to parent needs to go away. I really hope that systemd eventually uses subreapers to keep all user processes under the session leader.
I feel like it's weird to talk about scope creep when logind is pretty much the first system to define what a session even means on a Linux system.
If a process is interactive, it needs a way to be notified that its user is gone. That became a distributed system problem a while ago. PID 1 doesn't know whether an X client is active on a remote display, or a batch job is doing useful work.
dbus is also a dependency though. If you are on the bus (heh) that says systemd is a hard dependency that should be avoided then dbus is also the same thing.
I agree with your logical consistency, but I'm curious: is D-Bus sufficiently specified that one could write an alternative to it? That is, is it a hard dependency or is it itself an interface? I'm not nearly sufficiently experienced with desktop Linux application development to know. ;)
Yes, DBUS is specified [1]. There are multiple implementations shipping (dbus reference implementation and dbus-broker), as well as several different client libraries.
As dbus user, or application, you just need some client library. As an operating system, you also need a daemon running, which is routing the messages among all the clients.
IIRC at least part of the reason for sd-bus is the need for using D-Bus between parts of systemd during early boot when "normal" D-Bus is not yet up and running.
FreeBSD developers are actually looking at something like systemd themselves. The fact that systemd itself doesn't run on it is just the consequence of POSIX being inadequate for its implementation, since it relies on a bunch of Linux specific features, like cgroups etc.
I am personally in the camp that doesn't mind that, Linux users should get the best possible software given their system capabilities, not the lowest common denominator one just for the sake of it. (Free)BSD developers are also not looking to make sure their stuff works with Linux, which I think it's a valid approach.
GNOME itself does exist on the BSDs, by the way knocking down the argument of systemd being a hard dependency of GNOME along the way.
> FreeBSD developers are actually looking at something like systemd themselves
If this claim were true it still is beside the point of my comment, that systemd itself has a "Linux only, and we don't care" approach, and dbus for the moment is not that.
> GNOME itself does exist on the BSDs
Yes the ports tree is a wonderful place for those things to sit for those who are interested and for a larger portion to not bother installing them. I don't know how many patches they applied to get it working or what the cost and effect of those patches may be over time.
> Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
The 'Imperialistic Culture' you speak of is one of those made-up things that systemd haters keep repeating, but has no basis in reality.
> They pushed for Gnome to have a hard dependency on it so that everyone would be _forced_ to use it instead of just letting it be accepted by merits.
Again, another fairy tail.
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
A) This is par for the course in Linux software. See also: Linux kernel. One side you have is a bunch of security bros running around trying to fluff up their resumes and trying to turn themselves into heros. And on the other side you have devs working hard on various features and bugs and being resentful that most of their work goes unnoticed while stuff they already fixed is thrown back in their face.
B) Many of the bugs that make it onto Reddit/Hacker news are going to be ones that are largely changes in configuration defaults or behaviors that break things and are meant to be configured by the operating system designers that use it, not end users.
C) This level of scrutiny never existed before for low-level Linux features. The multitude of redundent and terrible shell scripts that made up the majority of functionality that systemd project seeks to replace was always a swampy mess of broken functionality and bad code. Having each and every major linux distribution rewriting 90% of it from scratch in a completely un-portable and estoric way didn't help matters.
Linux advocates actually went so far as to praising the fact that even though half of the scripts in their OS were completely broken and the OS still worked fine was a testament to it's robustness.
The fact that now, finally, after decades of this crap people can now track bugs related to low-level Linux userland 'plumbing' is actually a good thing.
> It is easier to configure and is convenient to have one holistic system in a lot of ways, but...
On the sysvinit side you had thousands and thousands of lines bunch of procedural code of dubious quality that is endlessly rewritten by hundreds of different teams with vastly differing levels of competency and success using a general purpose language for configuration. It was so unportable that you couldn't even use anything but the most basic and trivial (also broken) init scripts from one distribution to another without herculean levels of effort.
On the other side you have a even driven OS configuration and management engine that allows people to describe the state they want the OS to be in via a domain specific configuration language. It has managed to standardize the low level plumbing for Linux operating systems and allows distributions to share improvements with one another in a way that was previously impossible.
If people want a alternative to gain in popularity they need learn what systemd does and understand why Linux distribution makers switched to it. Then produce something of their own that takes those positive features and improves on it further.
All the hand waiving about 'Imperialism via releasing open source software to the public' is infantile.
The upside of all of this is that it's going to be a hell of a lot easier to make systemd unit files portable then it is to make Linux init shell scripts. There isn't any reason why a more capable init system wouldn't be able to parse existing unit files and know then how to start and manage the services and applications that use them... So the effort to move away from systemd should be significantly less then the effort to move to systemd.
> On the other side you have a even driven OS configuration and management engine
You are repeating the there-is-only-sysvinit-and-systemd fallacy, pointed out years ago by the Uselessd Guy and decried by many others since but (alas!) still going strong today in this very discussion, and ironically you are doing it whilst erroneously ascribing things to systemd that it was intentionally designed not to be. systemd was explicitly designed not to be an event-driven system. The wholly event-driven system was Upstart, and in practice it turned out to be a problem. Lennart Poettering xyrself discussed the problem, as did some of the Debian Technical Committee members during the Debian Hoo-Hah.
You're really not refuting anything here. Whatever bad situation existed with previous startup across, it doesn't excuse bad handling of issues in systemd. Whatever other projects do with CVEs, doesn't excuse Lennart complaining that CVEs should not be raised.
The imperialism part is likely due to score creep into parts they just can't handle well enough. systemd-resolved for example had terrible security and functionality issues still after it was being deployed to many distros. These were simple "point a fuzzer at it and watch it crash" issues which should not be seen in a DNS resolver these days.
> Many of the bugs that make it onto Reddit/Hacker news are going to be ones that are largely changes in configuration defaults or behaviors that break things and are meant to be configured by the operating system designers that use it, not end users.
Maybe it's not for systemd to decide what I'm meant to do. If changes to behavior that they only meant for operating system maintainers to rely on results in angry end-users, they probably made the wrong assumption.
> On the sysvinit side you had thousands and thousands of lines bunch of procedural code of dubious quality that is endlessly rewritten by hundreds of different teams with vastly differing levels of competency and success using a general purpose language for configuration.
Meanwhile, I have ~400 LOC of init scripts (including service scripts) on my daily driver, an init system with <1% the LOC of systemd, and plain text readable system logs. I agree that hand waving about "imperialism" isn't very constructive, but on the other hand I don't think that misrepresenting the alternatives to systemd and exaggerating their faults is particularly useful either, nor is pretending that sysvinit is the only alternative. A lot of people have thought "sysvinit sucks" and done something about it.
At the end of the day it is the handling and haptics that count. So let me give you some anecdotical experience. A few weeks ago when the new Fedora came out i tried it as a live system from usb keychain, out of curiosity. It was the xfce live respin. Anyways, long story short, clicked around for a while, wasn't impressed much, shut it down. Flicker, flicker, the usual console messages scrolled by, and then it was stuck trying to unmount somesuch yadda yadda yarr yarr... for minutes. I wasn't in a hurry and waited, curious
WHAT would happen. Nothing, just finally powering off after 2,5 to 3 minutes. I sat there and wondered: could it really be THAT slow? Was my usb-stick slow? Nope. Tried another stick, another laptop, even some other respins. Always the same. Dumbstruck of the audacity of the so called leader in Linux i just thought to myself that they must be on crack or
some other bad stuff. Because when i tried a few other systemd-free live systems on the same sticks and hardware everything just whooshed by. Why is that? Are they living in
a parallel universe? Do i have outliers in terms of hardware? At least in one case i don't think so because it is a very well supported Thinkpad which accepts almost anything i can throw at it, and the other a similar HP Elite Book. So there is that. Now what?
> Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
The most recent gripe I have is that systemd people can't even do randomness properly and even use non-cryptographic randomness for DNS transaction IDs (ok probably benign). But just boggles the mind.
When there was a problem with a sysvinit procedure, one could simply fix it. As a result, mostly nobody ever had a problem with it.
When there is a bug with systemd, you have to fix systemd. And your version of systemd is completely different from the head, because head was rewritten last month (in a process that persists bugs) and yours is a year old. You can't just use head either, because every low level system on your computer is tied to systemd, and they work differently on head. You won't fork it either, because it's so large that a single person can't keep up.
Without a doubt, a declarative init is much better than an imperative one. But that relatively small number of commands is no gain if each command is more complex than the entire set it is replacing, and I'll completely disagree on the handling of modern hardware, hotplugging and the like. Besides, there is the entire logging saga against systemd.
> As a result, mostly nobody ever had a problem with it.
The number of broken init scripts I have encountered throughout the years, that went unfixed for months if not years, stand in stark contrast to this statement. And I’m talking packages like uWSGI on Ubuntu 14.04! This is exactly why I gravitated immediately to systemd. The idea that init scripts actually worked and when they didn’t were easily fixable is not true. Yeah you can fix a broken script locally, but this doesn’t fly at scale n > 1. Submitting patches to distros is not a simple process, and not one that makes sense in this context either. Declarative beats imperative 9/10 times, and systemd is one of those 9.
> The people who prefer initv are the people who enjoy digging in other people’s (untestable) Bash scripts
This is demonstrably untrue. I strongly prefer SysV (which is not to say I think SysV is wonderful or anything) over systemD for a number of architectural and functional reasons.
But I do not enjoy digging into other people's init scripts. Fortunately, I rarely have to do that. I think the last time I had a problem with someone's init script was over 7 years ago.
> When there was a problem with a sysvinit procedure, one could simply fix it. As a result, mostly nobody ever had a problem with it.
To the extent that this isn't flat-out wrong, it's only because SysV init had so few features and so the actual patch had to go to one of many different upstreams. Having had to fix the same classes of problem in hundreds of different places like that has given me an appreciation for how long it's been since I've needed to do anything with systemd other than use it.
One important thing to remember is that the ability to have a standard way to start a process, as a non-privileged user, keep it running, with logging, etc. is something which has been standardized in the Windows world since the early 90s. It's really nice not to keep having to spend time working around the lack of a check-box competition feature for a quarter of a century.
> it's only because SysV init had so few features and so the actual patch had to go to one of many different upstreams
Yes.
Yet systemd is unfixable. The fact that it does too much was the most popular complaint at the time it got pushed into every distro... while I disagree, my main concern is that it's unmaintainable.
Systemd is a bunch of loosely coupled tools, though; it's not a monolith at all. PulseAudio was written by Lennart but isn't part of systemd. You don't have to use systemd's resolver. systemd-nspawn is a completely separate process and isn't required for systemd as a whole.
It's worth watching Benno Rice (a BSD developer, actually) discuss systemd and some of the misconceptions, some of which you're expressing, therein: https://www.youtube.com/watch?v=o_AIw9bGogo
There are lots of variations on the UNIX philosophy [1], but two core points are to:
1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
systemd has binary format data all over the place, and it is quite difficult to separate out an individual component and have it work properly on its own. Not to mention trying to replace a subcomponent with something else.
systemd is, in general, a tightly coupled system with lots of interdependencies. It is, in many respects, the antithesis of the UNIX design philosophy.
Hell, even modern Software Engineering practices still talk about decreasing coupling and increasing cohesion.
Regarding the UNIX philosophy thing, it's worth noting that it is not automatically The Right Thing to stick by it at any cost. You could argue that UNIX itself does not use the philosophy when it's inconvenient, for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)
Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.
The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.
> for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)
Sure. The UNIX Philosophy applies more to userland, I think, than the kernel.
Edit: I'll add with regard to the kernel, the internal structure of the kernel still has fairly well-defined subsystems that interact in clear, mostly consistent ways. So while from an execution standpoint it is monolithic, the philosophy is still something that is taken to heart to some extent.
> Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.
I think dismissing it as a semantics game is taking it pretty far. It's clearly not a black and white rule, but more a guiding philosophy. When it works, it's beautiful and elegant.
The ability to pipe basic UNIX utilities and programs together is something that has always left me a little awestruck. The flexibility and simplicity there is almost profound. SysVinit was far from perfect, but you could at least trace the execution flow from basically start to finish and, if you understood basic *NIX utilities, have a good idea of exactly what was happening.
> The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.
Again, it's a philosophy, not a universal truth. A guiding light, if you will.
With systemd, that light is pretty dim, if it's on at all.
Yup, seems you're right - journald appears to be the only mandatory non-PID1 process for systemd. Good to know. It also will forward happily to rsyslogd if that's your bag and then you never have to look at it, so I'm gonna just give a big ol' emoji shrug on this one.
D-Bus, as discussed elsewhere in this thread (where I learned about this, too), is a specification rather than a concrete implementation. Are you asserting that systemd does not work with alternative D-Bus implmentations? That would be weird, seeing as how they wrote an alternative one...
Also, since I've been "reading more", and I can check after I get back from lunch--at least at some point Debian shipped systemd without D-Bus on minimal installs. Maybe that's no longer the case, but the stuff I'm reading seems to be post-2015.
You need to stop constructing straw men, too. I asserted what I actually wrote. And Lennart Poettering asserted that
> D-Bus has been a core component of Linux systems since more than 10 years.
and
> D-Bus refers to both a specification, and a reference implementation.
back in 2015. I suggest that instead of arguing fallaciously about something that you tell us you have only just learned about, that you go and do that reading.
Asking for clarification isn't a strawman, sorry. And it sure seemed like you were correcting me, which is why I asked. But I said that journald was the only mandatory component of systemd outside of PID1. You said this was wrong. I'm waiting for you to extrapolate? Does systemd only work with the "canonical" dbus-broker? Or does it want ("want" as I'm reading stuff that suggests it isn't absolutely required, but I'm not sure and I'm happy to caveat where I'm not sure) a D-Bus broker? Because if it's the latter, and as it's specified and they seem big on meeting specs I'm betting that it's the case, then what I said is correct and you're just being kind of a jerk about it.
Or you can keep trying to big-time me instead of engage on the merits of your position. That's cool people stuff, I guess.
This is out of the line. You were given civil arguments for why systemd requires Dbus but in your response, you accuse the other person of being a jerk and "big-timeing". Please use the strongest interpretation of other people's claims on the topic, before using them as a means to accuse them of wrongdoing.
I am inviting the strengthening of his position in order to better understand it. I asked him to elaborate because as near as I can tell there is no hard dependency on any D-Bus broker. There is possibly--I genuinely can't tell from the documentation I've been browsing through--a dependency on a D-Bus broker. Requiring a D-Bus broker does not make any one D-Bus broker mandatory...and so, if that is the case, then my assertion that journald is the only non-PID1 part of systemd that is required would hold.
I'm happy to be corrected if I'm wrong. I invited it, even; I'm wrong all the time and being wrong is how one learns. The big-timing I referred to is the refusal to materially engage. Instead he quotes Lennart about D-Bus--out of context? who knows! It isn't an indication of the state of the software in question, just his opinion at a point in time--and to tell me to go read (read what, exactly? what generosity in citation!). Oddly enough, though, I don't see you tagging in to lecture him about that sort of sneering. Why's that?
That person is big-timing. Their tone ("you should read more" and so on) is in violation of the HN guidelines, filled with needless snark and condescension. The offending comment has been flagged by multiple users who agree that the comment doesn't meet community standards.
Perhaps the tone of JdeBP wasn't great, but they had a point. On the other hand, eropple's comment tone wasn't ideal either and they consistently miss the original point of systemd requiring dbus. Unfortunately, I am unable to check this since JdeBP post's have been censored.
It's good that systemd apologists have moved on from arguing that it isn't a single binary. But it is not a bunch of loosely coupled tools any more than it is "just" an init replacement.
Hmm. "Apologists." Couldn't be that it's been a lot better for me where I regularly interact with it and has been no worse where I don't. It's gotta be apologism. We have to be out to getcha.
I think you owe me an apology for that one, dude.
More productively: what parts of systemd's ecosystem--aside, apparently, from journald, but the above no-harm-no-foul reply to the sibling still applies as far as I'm concerned--have you had trouble pulling out when you wanted to, out of curiosity?
Apologists are engaging in apologia, which means a defense of one's ideas. It's not suggesting a hidden agenda, to the contrary, it suggests the agenda is fully on display since it's plain what is being defended.
My gripe is bad UX. The whole system is needlessly obtuse and esoteric. Git is another popular thing with this problem, but in the case of git it provides so much value it's enough to get me to overcome its obtuseness. For systemd it's not "enough better" to get me to forgive it for being designed with no thought whatsoever to simplicity or ease of use.
Another thing that makes me forgive git more is that its underlying design is rather novel and powerful. There is something to "get" in git (hah!) that justifies the cognitive investment. In the case of systemd the problem of managing services and their dependencies is not novel or really all that interesting. The complexity of the systemd UX is significantly greater than the novelty or sophistication of what it's doing. It's a piece of software built to complexify the simple.
I strongly agree that git has a UX issue, and the widespread adoption of it over mercurial is a perfect example of inertia leading us to get stuck in a local minima.
I haven't tried hg but from what I've seen one major advantage of git that gets under-acknowledged is its speed. Git is fast and fast really does matter for a version control system you use constantly. Is hg as fast as git?
No. You still have to launch the Python process per command, which is "extremely" slow (relative to C programs). I air-quote extremely because the difference for a single command is small relative to human time, but it matters for automation that invokes many commands.
This post is from 2018, and maybe Hg's frontend is written in C or Rust now (I don't follow its development). But if it is still a Python-driven runtime, the same basic criticisms apply:
Personally, I like pushing with hg, but I prefer pulling with git. An HgHub service at the scale of GitHub would probably result in massive end-user UX issues but maybe happier developers and repo administrators.
This reminds how pleasant it was to use hg as a git front-end (via an extention[0]) - compared to using the git cli. I think I'll have to revisit and see if that's still the case, and if I can make it work with feature branches etc.
Also looks like there's finally python3 support in the recent mercurial 5.0 release.
Git isn't really all that fast for what it is. The "git is fast" thing came about as a comparison to non-distributed version control systems, where changing what branch you were working on could involve coordinating with some (often overburdened) server on the other side of the planet.
That's not to say git is slow of course. It's just.. basically what you'd expect given it's architecture.
I like git's UX. Anything outside the typical flow is like walking into a garage filled with old stuff you thought you might need again some day, sifting through boxes of gadgets looking for that one thing you actually do need this time. Productive? Maybe not, but the experience is a much broader subject.
They probably do though, openrc defaults to sysvinit as its PID1.
I guess it's part of the confusion too, most of the systemd work was replacing the "rc bits" (process startup and management) not replacing PID1 itself.
I don’t really have a strong opinion anymore. Scott makes the right move here for professionalism and modesty, and Linux is more united on this than it isn’t.
Still, the spats with the Linux kernel team and the rest of the community has earned systemd a reputation I can only say it deserves, even if the initial ire drawn towards it was over the top. Systemd is not a project I have strong confidence in to act in the user’s best interest, based on what I’ve seen with handling security issues.
I get that systemd works for you, and I'm not telling you (or others) that you shouldn't use it. The problem is the systemd monoculture. Wayland, Gnome etc are non-starters because they have hard dependencies on systemd, which is ridiculous.
> For example, people are complaining about logind being included, but forget that consolekit was unmaintained before that.
I use neither logind nor consolekit. These are solutions for problems I don't have. Moreover, they often get in the way: when I ssh into my media PC, I don't want logind/consolekit to tell me I'm not allowed to access the sound or video cards. The only function that machine performs is to do the thing that logind/consolekit are designed to prevent. And it's a required component of systemd.
I'm glad it's easier for you to manage services with systemd. But it's easier for me to manage services with OpenRC, and there's nothing wrong with that. As far as handling modern hardware- huh? I guess it's not clear to me what interactions an init manager is supposed to have with hardware.
Which really speaks to my primary issue with systemd: it's a freaking monster. It does nine million things, but I only need it to do like three things: start services when I tell it to start services, stop them when I tell it to stop them, and restart them if they crash. (if it's configured to do so) That's it. Why do I need or want it to interact with my hardware? Why does it need an integrated web server in order to start apache? Why does it a logging daemon in order to start syslog-ng? Why do I have to use its built in log reader instead of tail, less, grep etc? Where is the boundary between systemd and the rest of my system?
And sysvinit wasn't pretty much a monocultur?. I mean even OpenRC is basically sysvinit with a lipstick.
> These are solutions for problems I don't have.
Good for you. But event handling is something that many systems today do need and I think we should try to optimize for the common use case.
As for you not needing logind/consolekit, sure, if security is not a concern on a system, they're not needed, but again, that's not true for most systems.
GNOME etc. also don't really have a hard dependency on systemd, they depend on logind, which does not necessarily require systemd and indeed GNOME ships on Gentoo, Void etc.
It's also not like the GNOME devs are stupid and got tricked into depending on logind or something. Mo, it solves real problems for them, so they depend on it. Why is this hard to grasp?
To start off with: Maybe I'm not on the same page, but your tone feels quite aggressive to me right now. I'm not telling you that you're wrong for choosing to use the software that you use. This isn't a flamewar, I'm not trying to tell you to do different things. I'm just explaining why the OP's criteria for an init manager don't apply to me.
It's also 3am my time, and I've had quite a bit to drink. If anything in my post comes off as aggressive, please let me know and I'll edit it in the morning.
> And sysvinit wasn't pretty much a monocultur?. I mean even OpenRC is basically sysvinit with a lipstick.
No. OpenRC is very happy sitting on top of busybox init instead of sysvinit.[0] When I encountered problems with my gentoo system built on musl (instead of glibc, #1 source of problems) and busybox (instead of coreutils #2 source of problems, and sysvinit #3 source of problems) they got fixed. A major part of why OpenRC is what it is is to prevent reinventing sysvinit/busybox's wheel.
> event handling is something that many systems today do need and I think we should try to optimize for the common use case.
> As for you not needing logind/consolekit, sure, if security is not a concern on a system, they're not needed, but again, that's not true for most systems.
Security is a concern on all of my systems, however, I do not agree that logind/consolekit add any security (at all) to most systems, or that they add substantial security to a small minority (say 5%) of systems. I would wager that 95+% of all linux systems are one of the following: A) no one is permitted access to any hardware, (video/sound/input) (headless systems, ie server farms, multiuser systems in a university, etc) B) single user account systems, (ie, my laptop. If someone has write access to my .bashrc file, they might as well have root access, and consolekit/logind can't bandaid anything by limiting access to my video device hardware) or C) hpc compute farms. (where the assumption is that all processes have god given write access to the GPUs) In the first class of systems, consolekit/elogind do not add security because all accesses are blocked, (in fact, the existence of consolekit/elogind simply expands the attack surface, making it less secure) in the second class no security is added because the video card is a more difficult attack vector than editing all the common home scripts, in the third class no security is added because whitelist-asterisk-to-video-card is the only sane configuration policy.
logind/consolekit are super awesome for eg educational institutions which have computers with multiple seats. Have four schoolchildren sitting at four monitors and four keyboards attached to one desktop computer? Sweet. (actually most schools use chromebooks now, which last I checked uses OpenRC) It's great that projects like consolekit/elogind exist to support those groups. But pretending like that's a common usecase, or pretending that sysvinint can't support those usecases- stop.
> GNOME etc. also don't really have a hard dependency on systemd, they depend on logind, which does not necessarily require systemd and indeed GNOME ships on Gentoo, Void etc.
This statement is, at best, disingenuous. If you want to install Gnome on Gentoo, you need to do one of two things: install systemd (Gentoo supports systemd as a first.point.five class citizen (my terminology))[1] or install elogind. elogind is a Gentoo specific forked version of systemd with everything that is not logind-related removed.[2] This code is still all upstream systemd code, and there is not a meaningful path towards an independent fork, the way busybox init exists independently of sysvinit, or syslog-ng exists independent of syslogd, or xorg-x11 exists independently of XFree11.
The fact that non-systemd installations of Gnome can possibly exist on Gentoo is a result of Gentoo developers going above and beyond the call of duty, despite barriers placed by the systemd team. And non-linux posix systems, eg FreeBSD - forget it.
> It's also not like the GNOME devs are stupid and got tricked into depending on logind or something.
I have nothing but respect for the Gnome team. I never implied anything of the sort. I am disgusted by your insinuation. Please walk this statement back.
I think that many of the things that the systemd monolith has to offer are valuable. If systemd were a federated constellation of small daemons speaking a reasonably stable API that didn't all have hard dependencies on each other, I would probably be quite happy with it. The problem - again - with systemd is the unforgiving, colossal monolith. I don't have the privilege of dipping my toes into the good parts without having to swallow whole the bad parts. And the bad parts are disgusting. (this is the same problem I have with Microsoft Windows) Modularity is a good thing; systemd is the antithesis of that.
Agreed, I very much like systemd. The documentation could be a bit better (though I haven't looked at it much in the last year), but overall it's a good improvement.
I first started using systemd with Arch before it got pulled into Ubuntu. It definitely took a little bit to get used to, but once you understand how to write services instead of cron jobs, it's way more betterer.
That's the thing: there are lots of alternatives besides sysvinit specifically (OpenRC, Upstart, BSD-style RC, launchd, SMF, runit, etc.) on varying scales from "simpler than even sysvinit" to "more feature-rich than systemd". And yet, the best the systemd advocates could ever muster up is/was criticizing specific flaws with just sysvinit.
Of course sysvinit sucks, but you'll notice that (Devuan being the notable exception) most of the article's listed distros use something that's very much not sysvinit.
I do think that systemd is very useful but have concerns that they simply don't have the man power to maintain that broad feature set and scope they have set themselves.
What previously would be done in multiple different competing projects by different people with different ideas in different companies, is now done in a single code-base. I can imagine that innovation will be hindered that way, because systemd is not Linux. But maybe some day it will be, IDK. Currently it is just to big to scale that well to small systems though.
That's a valid concern, but it's worth noting that not every component systemd replaced, (ie consolekit), was actually being maintained, so arguably we're in a better situation now from that standpoint.
Isn't it now much more difficult to figure out if certain components are or are not maintained?
ConsoleKit was used in many commercial linux distributions and still wasn't maintained, now that its part of systemd and systemd is maintained, does that mean that every component of it is as well?
sysvinit is easier to understand for me. It's just a bunch of shell scripts after all. So from the point of simplicity sysvinit wins for me and I never had much problems with it (while I had some problems with systemd). On the other hand, I recently had to write unit file for tomcat and it was extremely easy, systemd had all necessary options. Writing shell script was not that hard, of course, but it was some work.
One reason even big ol sysvinit is superior to systemd is that shell scripts are transitive knowledge. You can do almost everything with shell scripts. Learning some obscure (albeit very simple) systemd unit file format won't help you anywhere else besides systemd.
> One reason even big ol sysvinit is superior to systemd is that shell scripts are transitive knowledge.
In theory. In practice systemd's actually much more transitive since many distros have adopted it and systemd services tend to work cross-distro, so once you learn systemd you can somewhat confidently manage Fedora, RHEL, Ubuntu, Arch, SuSE, Nix etc.
sysvinit shell scripts often differed in subtle ways between distros and the particular distro implementation was rarely transportable without quite a bit of work.
> You can do almost everything with shell scripts.
This cuts both ways. I'd argue the problem with sysvinit was precisely the fact that you can do almost everything with shell scripts, since now you're executing a script as root that can do almost anything, depends heavily on the bash abilities of the author to not introduce subtle bugs, or break easily, (which varied heavily from script to script and distro to distro), and to not be a security nightmare.
RHEL has Systemd since 2014, Ubuntu and Debian as well. Even the purists at Arch have Systemd as default since 2012.
Sure, it ain't perfect. But maybe if all the professional kernel developers and distro managers chose Systemd, and yet some angry people online keep basing their opinions on random amateur blogs catering to a certain narative, regardless of expert opinions, then maybe we can conclude that...
... anti-systemd people are the anti-vaxxers of the Linux world.
Actually I do.
systemd is too complicated for my needs, maybe it is usefull in some complicated system setups but for me (and probably others) it is too big, too complex and completely unneeded.
What was the problem with sysvinit that they tried to fix? I don't see any.
systemd was aimed at the problems with Upstart rather than the problems with van Smoorenburg rc. van Smooreburg rc had already been superseded for its creators. Fedora was using Upstart.
The problems with van Smoorenburg rc are widely acknowledged, however, and motivated the creation of Upstart, finit, init-ng, and many other softwares; indeed, the problems of System 3/5 init/rc, which van Smoorenburg rc began as a clone of, similarly motivated the creation of the SAF in AT&T Unix System 5 release 4, the Solaris SMF, and the AIX SRC. There is a history of superseding them that goes back for decades. It is clearly wrong to claim that they are without problem. They've been recognized as problematic and people have been replacing them since before Linux was invented.
I read everything I could find on systemd to form my own opinion. I used FreeBSD for a decade and have many years of Ubuntu/upstart experience after that.
Based on my experience and all the anti-systemd posts online, I expected to dislike it.
But on it's merits I really like the consistency it brings to the "system layer" of the OS.
I find it's shame when developers skip over systemd for consideration, adding extra layers of complexity when systemd alone would have been sufficient: For example, using Docker when the basic isolation features of systemd would do, or using "forever.js" to manage Node.js processes, when systemd could could be saving 15% memory overhead per process, using software already installed in the base system.
On the other hand, I do support diversity of software and am glad to see some other options still being promoted and used. The dangers and weaknesses of monoculture in the physical world apply to software systems as well.
One of my biggest issues with systemd is that I don't like the idea of not being able to replace pieces of it. If I don't like their logging, dns, or ntp daemons, I can't easily replace them without feeling like a second-class citizen and/or having the overhead you've mentioned.
From my understanding, alternatives _could_ be written, but haven't yet and that means we currently having to ignore more battle-hardened software in favour of whatever is in systemd by default.
The ntp and dns are easy to replace, they're not part of the actual systemd binary. Fedora from what I remember, doesn't even enable systemds ntp service on Workstation.
Logging is a bit different, but you can tell journald to use syslog.
No. One can tell syslogd to read from the journal. There is no mode of operation for journald that does not have the journal, on disc or in a memory filesystem. And journald itself is not optional. The only optional part is choosing to have an extra old-style syslog daemon that is using the journal as input.
> The ntp and dns are easy to replace, they're not part of the actual systemd binary.
They really aren't. It took us a full week of work to get dnsmasq to work in place of systemd-resolved. The problem is that they are tied in to the dhcp client and it only seems to know how to update systemd-resolved's configuration. So getting our custom search path out of dhcp was a huge pain.
I like almost every other part of systemd, but trying to excise systemd-resolved from Ubuntu 18.04 was massively frustrating to the point where I started thinking, "maybe the systemd haters are right…"
The success of systemd is in that it solves problems for distros that none other was solving (or even looking other way). If you want to successfully replace it, you have to solve the same problems, and do it better, so it would be worth replacing systemd.
And because none other was doing it, you would have to write solution that does that. Slapping some bash scripts together won't do it.
It's a very strange argument that I have seen previously that someone could write an alternative to systemd but haven't ergo systemd must be the best alternative.
This is wholly and totally imaginary. Every part of system has an alternative because it is all solving existing problems that people have been solving for decades with a variety of tools.
The only actual challenge is replacing just part of systemd while retailing compatibility with the rest of it.
> This is wholly and totally imaginary. Every part of system has an alternative because it is all solving existing problems that people have been solving for decades with a variety of tools.
Alternative doesn't mean 100% substitute both ways. The pre-systemd didn't go in their abstraction in the way, that systemd does - it is basically system-level, event handling machine. The events could be not only boot, but also hotplug, timer, socket connection or others, and their handling is configured by the system or admin in unified way, including unified way of adjusting system-supplied units. Other inits just ran scripts at some order the boot and that's it, ignoring, that something doesn't have to happen at boot, but at some other time.
That's one of the things, that systemd handles way better, than any suggested alternative. By ignoring where systemd is strong, you won't make the alternatives any better.
> The only actual challenge is replacing just part of systemd while retailing compatibility with the rest of it.
No. I don't see why we should have the Sisyphos task of maintaing sysv/cron/etc idiosyncrasies forever. Sometimes, it is better to get rid of the collected warts, and not limit the system to existing semantics.
On the contrary, that's exactly what systemd's predecessor, Upstart, did. systemd, in contrast, did not. Basing the system around the concept of events triggering things turned out to cause problems, as discussed by Lennart Poettering and by Debian Technical Committee members during the Debian Hoo-Hah as their reason for not choosing Upstart.
I didn't mean that everything was in a single binary, but that everything is tightly coupled. Just because a binary is replaceable, doesn't mean the replacements that have existed for decades will be "drop in" or integrate as well, which leads to feeling like a second class citizen or the overhead originally mentioned.
I meant, that systemd solves some category of problems, that other tools didn't solve, didn't intend to solve and didn't even care for what the problems are. For distribution maintainers, systemd was something to welcome, that did address their problems.
So yes, there is no replacement (the requirement for it being drop-in only places the bar higher), because there's nothing else that scratches the same itch.
SystemD did not invent cgroups. Not only is it possible to use cgroups without systemd it is often easier for small cases and binaries can call cgroups themselves for any child processes they create.
The timing of these systems releasing does not equate to causation here. Google themselves (creators of cgroups) do not even use systemd on their own hypervisors which leverage cgroups heavily.
Nobody said they did. You’re putting words in my mouth. The ease of use with 2-3 directives in a way that just works is what they brought to the table.
Why is it another open source developer's obligation to satisfy your design desires? Are you paying them? If not, put your time where you aren't putting your money, become the purveyor of diversity, and submit a patch. Fork, if necessary and it matters to you that much--it's your right to do so.
The problem is basically that Linux used to be about choice (choose your distro! choose your DE! choose your shell! choose your browser!) but now that it's become big business, large corporations like Red Hat (developer of systemd) have transformed the evolution of Linux from a bottom-up consensus system to a top-down "we develop all the stuff you use and now it all uses our stuff so now you do too" approach. It's _very_ hard for me to build a system without systemd; I can use a niche Linux distro, I can use a BSD, or I can hack another init system (etc.) in myself. None of those are satisfying, I think for obvious reasons.
systemd is not well written, and it is not well designed. Its style and architecture are inconsistent. Parsing is mixed with validation and business logic. The types of CVEs its generated show that there was no coherent thought put into input validation, fuzzing, taint analysis, and so on. The effects of those flaws (your whole system crashes, etc.) show that failure strategies were never considered. And if all that sounds like overkill to you, maybe it is! But I can tell you that these techniques and design strategies are employed by experts to build software that must be robust, like software that controls medical equipment or transportation control systems. We know how to build software that works and that mitigates failures. Looking at systemd's code, it's clear they used none of that knowledge. And that, combined with its sprawling control over my system, makes me not want it on my machines.
I am not a systemd skeptic, but I'm also not somebody who cares that much. (Which might seem at odds with this thread, but I'm in a lot of meetings today. ;) I use it and it gets out of my way. For me, it's fine.
I'm not qualified to judge the quality of the code in systemd directly; I've browsed it a little but I am a middling C++ programmer and while it hasn't stood out to me as smelly I would defer to more knowledgeable heads. I get the drive for very rugged underpinnings, and I'm on board with that; in things that aren't C++ I frequently am the one with the scalpels making criticisms similar to yours, so that totally resonates with me. I am not, however, convinced that any of the alternatives that currently exist are materially better, so I kinda shrug a bit? Further, it seems to me that the generally modular nature of systemd (regardless of how coupled one believes those separate modules to be) suggests that a determined programmer could probably build a drop-in replacement--Rust? Where's my Rust Evangelization Strike Force?--that plays nicely with it and grows to replace poor and soft design.
The social aspect you bring up is, to me, the most compelling argument against systemd (because monocultures aren't great in general, etc.--I think most people agree on that). Thing is, though, I think that that bottom-up consensus model was doomed from the start. I don't think it was ever going to be tenable as soon as there's any money in it. Red Hat wants consistency, their stuff is going to have consistency, you can use it or go off-roading.
I don't think there's a great solution aside from "build your own ecosystem and don't let it get capitalism.txt'd to death." Which is a deeply unsatisfying and impractical answer. And to be clear, people who feel strongly about it should do that. I'm not being snarky when I say that. I'm a pragmatist--if it's better, I'll use it, too, and submit patches if needed and when I'm capable. ;)
> I don't think there's a great solution aside from "build your own ecosystem and don't let it get capitalism.txt'd to death."
I do, it's "if you run a big mainstream distro like Fedora or Debian, let your users swap out init systems". I know there have been debates on this, and I know the general consensus is "it's not worth the trouble / it's not as simple as just swapping out something like systemd". My counter arguments are: there are distros that are doing it, so it doesn't seem like it's that bad after all, and when dealing with something as important as PID 1 (or your libc, or your filesystem) we should be conservative and let users swap things back when making a huge change.
To that I'd just go "sure!", but I'm just a guy. The first thing that jumps out to me, though, is the question of who pays for the doing of it and the testing of it? From what I know (and I've been learning a lot today, it's been productive!), it seems like you're likely to undertake a Devuan level of effort to do it with Debian, and your patchset is going to be so broad as to effectively fork the core anyway. Is there value, then, in tagging along with that major distro? What do you get, then, if you're the second-class citizen? I'm reminded of stuff like Kubuntu, which (and I say this as a long-time KDE fan) has historically limped along with only moderate love--and that's a DE, it's a lot easier to handle DE differences than core init differences, service management differences, etc. (We have the XDG standards and the like, after all. Though I guess, in a certain light, systemd's trying to be XDG-for-the-core, which attacks the question we're looking at directly, yeah?)
The other thing that jumps out at me is just that this hypothetical is mostly kicking the can down the road. If Fedora suddenly decided to support multiple init systems I think it's hard to envision a future where the happy path wouldn't remain systemd; it's what the core folks think is best, otherwise they wouldn't use it now, and even with additional support the lion's share of attention would go to that happy path. To that end I think that any sort of init choice in those distros just becomes "so we'll support it for now, but the set of third party stuff that supports it is going to continue to shrink, so I hope you have an off-ramp in the future."
Between these two issues it seems like separate distros that have a firm commitment to not-systemd end up being necessary anyway and an ecosystem of not-systemd stuff will need to compete for value on those merits. What am I missing?
> Why is it another open source developer's obligation to satisfy your design desires?
I never said it is.
What I said is that if someone values diversity of software, then they should object to applications including a hard dependency on systemD. That's just logical, as including hard dependencies on systemD reduces software diversity by requiring that the system it runs on uses systemD rather than allowing them the choice.
Docker is about deployment. The isolation model is built on the same tooling that systemd uses, but the usage model is all about how to come up with an easily isolated runtime environment that can be published and extended in a big development community.
Basically systemd has nothing like a dockerfile, that's not what it's for.
I think there's a good argument though that Docker should be retargetting its encapsulation work to a lower level OS primitive where possible (which would be systemd on linux) instead of rolling its own.
Cross platform stuff? Docker allows you to do many things on Windows and Linux.
I’m not arguing that this is worth the overhead, or even a useful feature in practice (most companies and products tend to pick one environment and stick to it), but one that comes to mind that systemd would likely never have.
The cross-platform stuff is interesting... however, in practice almost everything outside Linux containers is only weakly supported. You have to jump through some hoops to use alpha/nightly builds of Docker hand installed in Windows 2019 or later Win10 builds in order to support both Linux and Windows containers on a single system. I think the work with WSL2 may make things better better as a result.
The naming in Windows and Mac, along with the limitations of the Desktop implementation are also frustrating. Don't get me wrong, I really like Docker as an orchestration base, I just don't think the cross-platform support is a value add in practice and in fact detracts as a whole imho.
I'm not sure systemd had every container feature available when docker was released. If I'm not mistaken, systemd-nspawn (systemd alternative to docker) is much more recent.
"systemd is, to put it mildly, controversial. Depending on who you ask it's either a complete violation of the UNIX philosophy, a bloated pile of bugs, a complete violation of the elegant simplicity it replaced or, it most cases, some or all of the above. So why have so many Linux distributions taken to it? Is it as bad as people say?
I watched this a few weeks ago as well, and it is a very good, even-handed overview of systemd. He talks about the history and other tools that came before, like Apple's `launchd` and Ubuntu's `upstart`. He also talks about the actual problems with it, rather than the FUD hysteria (and also explains why some of them are FUD).
He's from the BSD world, so his POV is as an (mostly) outsider.
I love this video and have to share it every time I see this debate. He really show how this argument is kind of weak and this is coming from a BSD guy!
Systemd was criticizes right from the start for the same things that it is criticized now. No misunderstanding. No bad timing. No tragedy present.
> People don't like change
Wrong! People love change if it is an improvement. People don't like change if things get worse.
> Services, System configuration, daemons, etc are separate distinct things and need distinct C code to be managed
Nope. They are essentially the same thing and should be dealt with by small scripts.
> It does not really violate the Unix philosophy
Nobody really knows what "Unix philosophy" actually is. Therefore it does not matter if it gets "violated". However systemd definitely violates my and many others taste for good software. It is bad taste.
> It's not bloated and monolithic because it's not one binary
The stupidest argument ever. If you have a lot of tightly interdependent binaries that don't make any sense to exist outside of systemd you have a monolithic system were simple nuclear parts are not replaceable with alternatives. Besides from that: Its f*in bloated as hell.
> It's not as buggy as people claim
It is. Besides, the real problem is not the number of bugs, the prolem is getting rid of bugs. Recompiling C code just to get your init working is really a world of nightmares.
> People don't like systemd because they don't like Lennard Poettering
Most only care about technology and systemd is bad technology. Nobody cares about Poettering.
> UNIX is dead
Hearing that coming from a FreeBSD developer? Do his BSD buddy's agree? Why doesn't he switch to Windows? Sounds like conman to me. Remind me to never install FreeBSD.
A big part of what made Linux great are the things that were borrowed from Unix and Plan9.
To make it short: The dude loves change and he wants FreeBSD to go in the direction of what Windows is today. I love change and I want Linux to go into the direction of Plan9.
Define an alternative. Most systemd alternatives do not concern themselves with post-boot event handling much, for example, which is a rather big part of modern systems. Most alternatives do not use a declarative style for unit definitions, which is a safer model being adopted by many other parts of Linux, including whole distros such as Nix.
Things like Kubernetes have become popular for similar reasons.
Notepad.exe is also an alternative to Word, if the subset of functionality you're looking at is strictly text input.
"demonstrably better" ... there's a reason beyond entropy why a lot of distros decided to adopt SystemD. A talk linked to in another thread goes over a lot of this. TFA itself shows there are alternatives.
It's like arguing that KDE or Gnome shouldn't exist as options... nobody removed anyone's options. Don't like it, fork it.
That said, to me systemd has been a large improvement. Maybe it has its problems, but at least there is consistency in handling services and logs between Linux distributions, which is a big improvement over a bunch of homegrown sysvinit derivatives, Upstart, and homegrown non-sysvinit scripts.
I've used it just a bit. AFAICT Shepherd is a fairly minimal dependency graph.
It's not comprehensive - in Guix there are services that exist above and below Shepherd and you can't define ordering dependencies between these (ex: a shepherd service can't depend on a mount (above shepherd), or an inetd service (below shepherd) on a database). I'm not sure that's a huge problem in practice.
Shepherd process stdout/stderr are sent to syslog. Shepherd automatically restarts services if they fail but not forever, and the retry behavior isn't configurable ATM.
Guix is a weird little behemoth of a distribution, that allows you to install different versions of different packages at the same time which then builds your effective file system on the fly. That being said, as mad as this sounds, I've heard people are having a lot of fun with it too, so apparently it's also manageable.
Guix is for you if you want to run old software on a discontinued software stack simultaneously with a modern system.
That makes it immune to dependency hell; there are no two packages you can't have at the same time due to dependency version conflicts.
The big wins for me though are: having all my system configuration in a small set of versionable text files (rather than gigabytes of mutable state), and being able to attach dependencies to the scripts that require them rather than to the system they're expected to be run on.
Couldn't you have just looked into a little bit instead of saying you know nothing about it? It's a distro with functional package management.
https://www.gnu.org/software/guix/
I mainly use FreeBSD, but I have a number of Linux VMs that I use where appropriate.
The big thing, for me, that systemd brings to the table is consistency. Managing services on CentOS is the same as managing them on Debian which is the same as managing them on OpenSUSE. This makes it a lot easier to deal with the various different Linuxes.
Also, it seems to make writing a service a lot easier. Instead of the service needing to daemonize, write out a PID file, or whatever, it can just be a program that does its thing without forking, sends its output to stdout/stderr and systemd takes care of the details. I tried to go through this exercise on a FreeBSD system and ended up just installing 'pm2' to do it for me. I subsequently figured out what I did wrong, but pm2 is working and I see no need to change it.
Obviously systemd isn't suited for every use-case, but it doesn't seem nearly as bad as some people make it out to be.
> Managing services on CentOS is the same as managing them on Debian which is the same as managing them on OpenSUSE.
It's not really the same, some systemd-based distros diverge a lot and anything non-systemd much more so, including FreeBSD. Even something as basic as network configuration could be completely different on systemd distros, like Ubuntu with its netplan weirdness and Archlinux going into the opposite direction and forcing you to choose how to deal with network yourself from the start.
But still, if you want any sanity in your deployment across different distros - you simply have to avoid using systemd or other host's init and service management systems for anything and bring your own service management with you. It could be as simple as a script starting from root's cron at @reboot (this particular trick survived for decades for example).
> But still, if you want any sanity in your deployment across different distros
This might be true for your in-house app but if you want sanity in packaging your service for inclusion in various distros you include a .service file.
By managing services I really mean starting/stopping/checking status of/etc daemons. On a systemd distribution, you make a unit file that describes the daemon and use systemctl to manage it. It seems like sanity for deployment can be accomplished by delivering a template unit file and letting the distro maintainer or sysadmin customize it as required.
Networking is a little out of scope here - I'd guess that changing basic network configuration is something that's less frequent than the service management tasks.
As much as I'd love to get into the rancor and bikeshedding good time as someone who just wants to manage some services on a few hundreds of thousands of instances and not worry about it systemd does that exceedingly okay.
I actually really like systemd's service files, and the functionality it provides. I like the ability to express dependencies clearly, the fine-grained control over server restart behavior, and a ton of other systemd features. I'm also very happy to see it replace cron.
What I _do not_ like are:
- systemd-resolved
- systemd-networkd
- systemd taking over disk mounts
- systemctl
- unit files spread out all over the place
- mandatory journald
- mandatory dbus
- only works on Linux
- pushing the BSDs farther towards being obsolete
I'm very glad to see sysv-init on the way out, personally. I think systemd unit files are a huge improvement, and greatly improve standardization and init quality. But that doesn't intrinsically mean that the systemd project is doing the right thing.
It's comical how people get so "jazzed" about something like not liking systemd and make a whole movement about it (with their t-shirts and everything). I get people may not love everything about systemd (or maybe they hate it), but I'm not convinced it's so bad. I've been using it since RHEL 7 and I've really gotten used to all of the benefits of systemd. It also doesn't hurt that I went to the session on pid 1 by Lennart Poettering at Red Hat Summit 2014 and he convinced me of the at least not-horribleness of systemd.
The simple reason is that everybody is forced to deal with it. So you either like the way it works, or you dislike it, but there's no real option to just ignore it in Linux land anymore. For a lot of people, that means that their favorite distro suddenly works in a different way.
This happens every time you significantly change something that people use and have existing workflows for, regardless of whether your change is beneficial on the whole.
I didn't care much about the problem systemd was solving, I was content to let the init process go through without touching it.
Then one day several years ago I did an apt-get dist-upgrade and my system didn't boot after that. I have personally hit way more issues with unbootable systems or having to drop into single user mode to fix some BS with systemd. That was enough to make me dislike it, or convert into a hater, and switch most of my personal usage away from Linux.
I think it's gotten better since then. I still dread updating Linux boxes, because I know that some percentage of those updates will have a systemd bug introduced resulting in a non-bootable system or one where I have to manually fix it up.
Counter-anecdote, I switched to systemd all the way back in 2012 on Arch, even before it was mandatory and way before it was in a state where other distros felt comfortable switching and I remember to this day that after I did the conversion and hit reboot, an absolute panic hit me, I was 110% sure the system won't boot afterward, just because of how drastic the change was to perform on a live system, (because simpler upgres than that broke my system before). Since Arch is rolling is not like a new release came with systemd, no, you just switched your live system over.
I remember being absolutely floored when after that relatively complex procedure, not only did the system boot, but it booted faster. I think I became a fan that day.
...and when you care quite deeply about the problems being solved, think they're being solved in a pretty good way, and are unsettled by the anti-systemd frothers literally sending Lennart death threats? (Like...this is bad, yo. There are arguments--bad ones, IMO, but arguments--against systemd. But the people who like systemd aren't sending death threats to Devuan people or whatever.)
To be fair, the ones sending death threats are contributing negatively, while the fork effort is a positive contribution to the situation. We'd appreciate not being lumped in with the worst offenders, if you'd be so kind.
A lot of the western world has devolved into tribalism and identitarianism. People of similar views on anything are lumped in en-masse and presumed to hold all of the same values, opinions and support all actions of the collective whole.
It's a shame that a lot of core values are lost in favor of these viewpoints.
Yeah. It is really sad. We're getting hyper-polarized options and information everywhere we look, and it's affected people's ability to argue a specific point if they detect any misalignment of interests. I've been calling it "casually dismissing someone's opinion by lumping it in with an ideology", but looking for something more concise.
I get no end of bemusement from open source advocates that think it makes any kind of rational sense to campaign against an open source project. THIS IS ALL OPEN SOURCE. Freedom and choice is the whole point, isn't it?
If you prefer an alternative or forebear to systemd, that is great. Use it. Advocate for the system you prefer. Convince people that the choice you have made is better.
Meanwhile the vast majority of distributions have voluntarily chosen systemd. Perhaps if its critics reflect on the reality of that shift, they could look at ways to improve the alternative to address its perceived deficiencies—just as GCC started doing now that LLVM has taken a bite out of their dominance.
Notice that nobody complains about the existence of systemd, but about the enormous and shady pressures that distros had (particulary debian) to adopt it as standard. Nowadays, the complain is mostly about the systemd monoculture.
As someone who really wanted the BSDs to get more credit ~20 years back, and spent a fair amount of time on various commercial Unixes as well, I find that phrase rather amusing. Historically poor standardization of the userland aside, GNU/Linux is the poster child for monoculture in Unix, and has left a whole lot of dead or dying software in its wake.
Just to pile it on, I think it's becoming something of a viral force for monoculture, too. 20 years ago, I might occasionally reboot to Windows or fire up a Windows virtual machine in order to open up the occasional recalcitrant Word document in Word itself. Nowadays, if I'm not running Linux on the metal, then I need to keep some sort of a Linux virtual machine running constantly, because so much software has a hard dependency on either the GNU userland or a Linux kernel feature such as LXC.
This is worrisome to me, because, while GNU/Linux is FOSS, "free and open source" doesn't actually mean "unfettered". A small number of large corporations such as IBM and Google have an outsize amount of influence. Doubly so now that IBM owns Red Hat. And, as the whole GNOME/systemd thing demonstrates, even the non-corporate entities have a lot of opportunity and incentive to engage in rent-seeking behavior now that things have been whittled down to such a tight software monoculture. They'd presumably not have even attempted such a thing as taking a hard dependency on systemd if maintaining decent compatibility with FreeBSD (or any other OS) were still a relevant priority.
I realize I'm picking a nit, but I think that, in this situation, it reflects a distinction that's worth making:
Systemd and Docker do entrench the monoculture, but they can only have that effect because the monoculture is already so well-established. If this were EEE, they'd be part of the transition to the third E.
We lost ownership of our systems. No one wants a BOFH built and maintained IT system no matter how wonderful it is at actually meeting business needs, because BOFHs aint interchangeable and business built on individuals aint a thing anymore.
So open source is full of apparent forced goupthink and all or nothing half solutions, at least in the corporate world.
There's home and hobby and personal business systems that are really slick using the lesser known packages and things like "I forked this BSD kernel years ago and twiddled it since". We hear about them sometimes in news stories about "Airport HVAC system found to have been running on C64 for 19 years." Perhaps it inaccurate to say its 'dead or dying' software, until it's no longer appropriate for those use cases where we never even notice its still running.
I've decided to migrate my Linux boxen to BSD (this is a long-haul project that will take me a year or more, I estimate). In part because systemD itself really made me aware of the issue that you're raising.
Which BSD, may I ask? I’d personally recommend OpenBSD, for code quality and “it just works”, but there’s a lot of really interesting BSDs that are even more obscure (Dragonfly, Debian and Gentoo using BSD kernels, etc) aside from FreeBSD and NetBSD.
In fact many people did complain about the existence - I have read actual demands that distros should actively work against it back in the days. Also the personal attacks against Lennart make it difficult to see any rational point in the whole debate.
There's a definite [citation needed] on the assertion that integrating tasks into one project/process == monoculture.
And a [citation needed] that systemd is large enough to warrant being called a monoculture in the first place—and to flip the coin, that GNU init is small enough to not be a monoculture.
Let's not forget a huge [citation needed] that the addition of another alternative into the marketplace of ideas could ever be described as monoculture.
And while we're at it, how about a big juicy [citation needed] that a monoculture is a bad thing anyway, in this instance?
Open standards are more important than open source. Choice is indeed the whole point, but systemd is designed to take my choice away. It doesn't matter how good an init system I make, if it starts getting any traction then systemd will come up with a new incompatible API for doing something, Gnome will take a hard dependency on that API, and distributions won't be able to ship my init system.
It's worth realising that the GNU project could never have happened in the first place - Linux could never have been a thing - if historical Unix had been designed like systemd.
It's laughable to worry about Gnome taking a hard dependency on an API when hundreds of software programs take a hard dependency on Gnome. Hypocrisy, much?
Meanwhile if the maintainers of GNOME betray your trust, it sounds like you should hate this hypothetical GNOME for the same reasons you apparently hate systemd. Or rather than hating it, you could just fork it. If enough people agree with you, a disentangled fork will inevitably be maintained. Hell, people maintain forks of GNOME for BSD—and that's even more impressive since there is no open source monoculture more monolithic than the Linux kernel API dependency.
You could quite possibly be right that the open source movement couldn't have have started with the design principles of systemd[0] but it's naive to assume that what was good for open source in its infancy is still what's best for open source in its coming maturity.
[0] Given that the open source movement has monumentally benefited from monolithic design principles in the form of the Linux kernel, I think that hypothesis is demonstrably absurd and easily rejected.
> If the maintainers of GNOME betray your trust, it sounds like you should hate this hypothetical GNOME for the same reasons you apparently hate systemd. Or just fork it. If enough people agree with you, a disentangled fork will inevitably be maintained.
And then they move on to the next project and introduce a hard-dependency on systemd there. RedHat has more programmers than me, they're always going to be able to add systemd dependencies faster than I can fork things. The only way users will ever be able to have a choice of init system is if the community can agree that hard-depending on systemd is indeed a betrayal, and publicise this.
> it's naive to assume that what was good for open source in its infancy is still what's best for open source in its coming maturity.
Open source succeeded because it was easy to improve and replace things piecemeal. That's still as important as ever.
> [0] Given that the open source movement has monumentally benefited from monolithic design principles in the form of the Linux kernel, I think that hypothesis is demonstrably absurd and easily rejected.
Linux has suffered from its monolithic approach to kernel-space development, but the damage was mitigated by an extreme commitment to providing a solid API to userspace that never gets broken. So most progress took place in userspace and wasn't affected. There's no analogous boundary for systemd.
Your last paragraph is absurd and internally inconsistent. Linux (the larger operating system environment) exists only because of its monolithic approach to kernel-space development. Far from suffering from it, it made it all possible when the utopia of the GNU kernel flat-lined.
Open source has succeeded because it was easy to improve and replace things piecemeal. But deciding that the individual pieces must begin and end at the process level is blinkered. You can perform "piecemeal improvements" to the scheduling component in systemd just as easily as you can to, say, cron.
> Linux (the larger operating system environment) exists only because of its monolithic approach to kernel-space development. Far from suffering from it, it made it all possible when the utopia of the GNU kernel flat-lined.
Linux-the-kernel was largely just in the right place at the right time, but to the extent that it had development advantages they were an openness to contributions and the absence of an imposed top-down design.
> Open source has succeeded because it was easy to improve and replace things piecemeal. But deciding that the individual pieces must begin and end at the process level is blinkered. You can perform "piecemeal improvements" to the scheduling component in systemd just as easily as you can to, say, cron.
You can submit changes and improvements to a given cron implementation. But if there's a really hard tradeoff or a disagreement, you can also fork off your own cron implementation, and in fact there are three or four different cron implementations in common use on linux. This is really important and not just in the cases where a fork actually occurs: the possibility of a fork encourages maintainers to be reasonable and come to a consensus.
Systemd's tightly coupled design makes that impossible - it's not about process boundaries but about having any kind of boundary where there's a stable interface between different components. E.g. reasonable people can disagree over whether the advantages of a binary logging format outweigh the disadvantages. But the systemd maintainers have taken a particular position, and, crucially, people who disagree can't just fork the logging component and swap it out for their piece, because there's no stable interface between that and the rest of the system.
Suggesting that you can fork cron and you can't fork systemd is patently absurd.
You say that "people who disagree can't just fork the logging component and swap it out for their piece" when in reality you really can. Getting systemd to output text logs is trivial, just point it to syslogd instead of journalctl. It sounds like you weren't aware of that. So much for being a tightly coupled design.
You are presuming that everyone gives a damn about open source as an end unto itself, not merely a means to an end. Ironically, the people who do consider it an end unto itself are probably the people most likely to despise the term "open source" and prefer "Free Software".
Ask RMS what he thinks about SystemD and I bet you he'll shrug and say it's fine because it's Free, and that's all he really cares about. Needless to say, most people have a greater diversity of concerns.
Trust me, I don't give a damn about open source as an end in itself—to me RMS means root mean square first, Roads and Maritime Services second, and occasionally I might be reminded of an unwashed beard.
> "Trust me, I don't give a damn about open source as a means to an end"
See, I think that's where you differ from most people. To most people, I believe open source is a means to an end, not an end unto itself. What they want first and foremost is something that works, and it happens to be the case that having the source code freely available facilitates having something that works and gives them a sense of comfort, an assurance, that it will continue to work in the foreseeable future. So appealing to "it's opensource, so you should like it" totally misses the mark. The argument should be "it works, therefore you should like it."
Had SystemD never given anybody any trouble, I don't think it would be hated. I think SystemD is hated because (particularly during early adoption years) there were a lot of things about it that didn't work for a lot of people. This made them angry, because it violated what they actually cared about. Not permissive licensing, but rather having something that works. The principle of open source is not axiomatic to most people; it only has value when it is part of having something that works.
Somewhere else I see this mentality: Android. There are a few among the 'community' who would have me pop open the champagne because "Linux won" on account of Android being an incredible commercial success. But why should I celebrate that? Why is "number of CPUs running Linux" the metric I should be glued to? Because Linux is open source? So what? Honestly, who gives a damn?
It's not SystemD itself that's the problem; it's just technically terrible, but, as you say, it's open source. A lot of open source software is terrible but popular.
It's Debian violating their social contract that was the problem for me.
What the vast majority of distributions do is irrelevant, a lot of them are chasing popularity or have a proprietary interest in having a "unified platform", but I had come to expect better from Debian. There is really no longer a "reality of that shift" to reflect on, since the SystemD problem has been more or less fixed: there are now debian-based distributions to upgrade to. If other people want to run that dumpsterfire of cowboycode on their computers, that's fine by me.
Debian ranked SystemD adoption over their users, since it broke users' computers, even though there were viable alternatives available that would have not broken these systems.
I still like Debian, but I'm not letting them (directly) near my systems again.
Most arguments about systemd will devolve into a Motte and Bailey argument. The problems with systemd are all the stuff tacked on that are mandatory, but if you criticize them you'll get people switching to try to put words into your mouth about how the init replacement is good/bad.
The Motte is all the shovelware attached to systemd that is nearly impossible to detatch and the Bailey is the init replacement.
There are no official ties between Alpine Linux and ungleich.ch.
We at ungleich are simple people liking simple systems. And we think it is important to support systems that are not mainstream.
But...it doesn't include "everything else". My desktop doesn't use systemd-timesyncd, I'm not even sure systemd-nspawn is installed on my home server, that same server isn't using systemd-resolved. I haven't fiddled with journald anywhere 'cause I like it a lot more than rsyslogd, but you totally can forward journald stuff to rsyslogd if that's more your bag (and I think, though don't quote me on that, you can remove it entirely if you want).
Yes, there are additional, thoroughly optional, components to the systemd ecosystem that you can choose to opt into. And...so?
The thing I don't get is how bothered people are by it existence despite being able to just forward logs to syslog and then ignore it forever.
[Journald]
Storage=None
ForwardToSyslog=True
And then it's like it was never there. But technically it's better than if it was never there because you can still capture service stdout/stderr and it will show up in the syslog where on an older system it would just be dropped.
I have way too many cases of systemd startup items hanging with their timeout for no good reason.
Worse, my most complicated high-uptime machine usually does not shut down in a reasonable time. Systemd says "waiting for session user cracauer" (something like it) for whatever reason. It also hung on undoing swapspace, when that swapspace was a custom stack of block layers. I don't need swapspace to be "shut down". It went into 20 minutes something always stating another timeout when one expired. I mean, WTF?
The problem here is that I had to do an unclean shutdown on that machine (reset button) multiple times, and that I really can't have.
This also illustrates a point I have been making about Linux and the BSDs from day one: When the BSDs boot or shut down the keyboard is connected to the rc scripts. You can do Control-C for SIGINT and Control-\ for SIGQUIT, The latter makes everybody in the stack leave a coredump on disk.
Now, compare these two:
1)
in BSD when there is a startup or shutdown item taking too long or hanging I can abort that single item and I can later debug what was going on with the coredumps.
2)
on a Linux system with systemd I cannot "cut through" startup or shutdown items I don't want to wait for, and there is no way to debug any of that if the machine doesn't actually come up completely, or if thing happen at shutdown. The only interaction I can conduct is the reset button.
To add insult to injury, systemd also hides a lot of error messages that would traditionally be dumped on the console, and instead of giving me that error message systemd captures it and print on the screen instructions how to get that error message (which usually are longer than the error message was, WTF?). Instructions that I cannot follow because somebody detached my keyboard, and because I will never be in that machine in an up state while that context is available.
This also invalidates the point that systemd could be one platform that you learn once and then use on a wide basis. The debugging abilities are nowhere close to adequate, so you have to learn by try-n-error.
I've struggled with, and am still struggling with, units hanging on shutdown. It's my biggest complaint with systemD aside from journald eating its own logs.
I've been using magic SysRq to force hanging units to quit. SysRq + e sends SIGINT to all processes, and SysRq + i sends SIGKILL.
Sending SIGINT to all processes isn't what is needed here. It needs to go to the "processes" that are currently going on. With systemd's parallel execution that can still be narrowed down in a useful way, especially is some tail of some hanging command is running alone by now.
Obviously what the BSDs are doing, sending SIGQUIT to just the currently blocking-the-shutdown processes, is far superior. Now the coredumps are floating around the filesystem and they'll be there after reboot, and you can see who did what and was hanging in what backtrace.
systemd errs on the safe side and thus it tries really hard to shut things down safely vs just yanking the plug. The timeouts are easily configurable if that's a problem for you, but I'd argue that if services hang too often for you, there's an underlying problem with your system somewhere, (i.e. your mounts or such, I'd consult journald for dependency cycles).
There might be some way in which I had run a system that caused those endless slowdowns.
But the point is, and that's why I think systemd is garbage, that there is nothing there that helps me debug what is going on. If the shutdown is hanging there is nothing to do except the reset button or waiting for an eventual reboot. In neither case will there be any forensics about what exactly systemd had tried to do and how it thought that it failed.
That is the part that is not acceptable.
P.S. I shouldn't have to manually adjust a timeout on disabling swapspace on shutdown to prevent a need for a reboot by reset button. The kind of thinking that got big timeouts into those things is precisely why I say that systemd people just think differently than I do.
> that there is nothing there that helps me debug what is going on
That's not really true. Journald is your friend. I find many of the complaints about systemd stem from unfamiliarity with its tools.
The Arch Wiki gives a really nice overview of the most common usage scenarios[1] & [2], that's worth investing some time into. Also [3] is your friend. Just refusing to put any time into learning systemd and then complaining about it, is one approach, but not a particularly productive one.
In your case, I'd try something like `systemd-analyze verify default.target` and go from there.
Additionally, look into /etc/system.d/system.conf, specifically adjusting the Timeout values to a less conservative value.
> SystemD definitely spurred some controversies years ago but is it that bad in 2019?
The reasons for the controversies have not really changed (lennart, the idea that it's not "unixy", a takeover of many subsystems by various systemd components, …) so people who didn't like systemd years ago probably still don't.
As somebody who's not really involved, I found Benno Rice's The Tragedy of Systemd[0] interesting and enlightening.
I found that talk incredibly condescending. He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing and apples attempts to upstream the problem to the bsd developers has been met with open hostility.
It’s also to the point where Benno states that people don’t like change, but that’s really not true. Everyone jumped on virtualisation (except Theo De Raat), everyone jumped on containers, Kubernetes, GO, Rust, eBPF et al.
There are new technologies which don’t have to eat the universe to function.
Most people who have a problem with systemd don’t actually hate it, they’re upset because the environment that allowed systemd to grow no longer exists for its replacement. Coupled with the fact that we should really consider systemd a proof-of-concept for something better, means that we’ll be stuck with buggy, weird, non-deterministic C++ for decades.
That’s the root of the problem, not that we nerds dislike change, we tend to love change.
> He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing
This is your personal opinion being presented as fact. Speaking from some experience in the Mac admin world, I’d really want to see the survey you did to support that claim.
yes it's still that bad, my previously reliable systems are now non-deterministic and break randomly
every few days I discover some new bit of non-determinism:
- systems getting stuck for long periods at startup or reboot with with some useless text like "Waiting for user session to shutdown"
- reboots never completing at all (getting completely wedged shutting down)
- jobs starting in weird orders, like crond starting up before the network, causing various things to break in new novel ways (sometimes, not always)
I can fix most of these systemd-created issues one by one as I discover them, but it's hardly making my life easier
All of those issues happened with SysV init, as well — there are literally decades of history of people working around things like unmounting failed filesystems, watchdogs to kill -9 hung daemons, etc.
For example, you mentioned crond startup before the network but that's hardly new with systemd — checking the oldest system I have access to shows that it's configured with two dependencies: “# Required-Start: $local_fs $syslog”. To the extent that systemd changed anything it was exposing that an assumption was wrong — and the nice thing is that you can use a systemd service to replace that cron task with something which _does_ behave the way you thought with precise service dependencies and all of the other benefits to using systemd (e.g. resource management): https://www.freedesktop.org/software/systemd/man/systemd.tim...
as I said, I can fix the issues one by one as I notice them, but if they only occur 1 boot in 10, or maybe 1 in 100 it's miserable and a much less reliable system than before
but hey, the systems now boot 2 seconds faster (if they boot at all...)
Again, you already had a problem. It might not have been as obvious because it was enough slower to avoid the race condition under normal circumstances but I’ve hit those problems many times over the years.
Again, I've said that I know there are issues, that I end up discovering one at a time.
the problem is not the issues themselves, as once I know they're there I can trivially google for how to fix them, it's the fact they don't show up except sometimes
I do not care about boot speed at all, I care that I upgrade the system to a new distro release with systemd, test it, then it continues to work that way until I change something else
this is no longer the case, and in 20 odd years of maintaining systems I've never had as much "fun" before as I've had over the last couple of years with systemd
I just want my systems to boot, run my simple applications, then shutdown when I ask them to
> I just want my systems to boot, run my simple applications, then shutdown when I ask them to
Which is what you get out of the box with systemd. The only problems you mentioned either apply equally (e.g. shutdown delays due to blocked processes — having supported NFS clients, let me tell you that was not a new problem in the 90s…) or are due to misunderstandings about how to use other components.
For example, if you have a critical service started in cron with no error handling or recovery, you always had an unstable system and were simply lucky not to have seen it fail before now. Projects like Upstart and systemd were started in large part due to people getting tired of having to kludge around those limitations of the older approaches. Having fixed systems which were configured that way in the past, I much prefer using something which solves it by design than continuing to bet on being lucky.
you are not listening or being are deliberately obtuse
I have stated three times that I accept that these processes are incorrectly specified, and that my problem is not this -- the problem is that I cannot discover these missing dependencies by any reasonable process
whoever set them up 20 years ago is long gone, and what was reliable is now not
this is a regression
(and let's not get started that my brand new installations of Debian onto a single root partition get stuck booting or reboot 1 in 10 times... I get it, totally my fault as the end-user)
> you are not listening or being are deliberately obtuse
Any time you find yourself writing something like that, ask whether you’ve done a good job explaining your point first. In this case, you’ve been vague about everything except that you think systemd is the source of your non-deterministic behavior.
From what little information you’ve provided, it really sounds like the problem is a combination of blaming the init system for things which happen outside of it (i.e. processes hanging) and not learning how to use it. For example, launching a networked process under cron at boot with no retry mechanism has always been a problem because cron never guaranteed networking. Using the init system would have avoided that - even on SysV.
As far as how to discover it goes, fault tolerance has lots of edge cases — e.g. try mounting NFS and firewalling the server, `kill -STOP` a daemon with a clean shutdown process, reboot without a network or DNS (i.e. like you’d have during a facilities outage), etc. — but most of it comes down to learning to work with the way the distribution is intended to be used: use the init system (whatever it is) to launch processes, enable automatic restarts for everything, configure full dependencies on everything you need, etc.
Note that none of that depends on systemd - you can get better results on everything this way, but if you do it you’ll probably gain a better understanding the systemd developers made the choices they did. If you really can’t figure out how to be comfortable with it, switching to a distribution which doesn’t use it will be much better than fighting it and getting angry when the developers don’t support hammering a screw.
Suppose you run your system on a new, better, and faster computer architecture, newly supported by Linux. Now, let’s assume that this is a new architechure which in many ways is different in style to other processors, but let’s say GCC supports it and can generate 100% correct code from it. Your problem now is that much of all the old software was written and tested only on older architectures, and this new architecture exposes many bugs such as previously unknown reliance on undefined behavior, which worked on all old architectures, but not this new one.
Now. Do you blame the new architecture for this bad situation? Or do you blame the old software, which technically always had bugs but those bugs were previously not triggered?
The harder it is to avoid undefined behavior, the more I blame things other than the software. Especially when you can run the old version under emulation and it still works without bugs.
And a lot of the blame goes to the distro that shipped a bunch of broken binaries.
That is surely an aspect of the language, not the platform?
> And a lot of the blame goes to the distro that shipped a bunch of broken binaries.
Sure, I can see that. It is normally the case with new ports to new platforms to be in a state of unreadiness for quite a while. But this is not really relevant to the question of who to blame for the brokenness; the hardware platform or the software?
My analogy is that systemd merely exposed bugs in other software which had been there for ages (and only sporadically exposed). But then systemd comes along and expose a whole lot of them at once. Keeping with the analogy, some blame certainly goes to any disto which released this combination of systemd plus software which has bugs triggered by systemd, but still, is it reasonable to blame systemd in this case?
I think some of my pain came from attempts to unmount everything (particularly NFS) before letting long-running shutdowns (graphite/carbon) finished flushing, then never trying again. While diving into that and eventually coming up with a sufficient set of dependencies, I would fondly think back to the two (or more) phases of unmounting at shutdown that had that solved cold many years prior.
It seems the systemd hate mostly comes from a philosophical position these days than anything grounded in real user impact. You either agree or disagree with what systemd does, or do not care because you are not aware of it (which I would argue is a good thing).
Personally I think it is great. Sysvinit scripts were terrible to write and frequently had bugs because the scripts were usually afterthoughts "Oh yea... I guess we need one of those things". The nice integrations between systemd and journald is well worth it in my mind as well.
> Sysvinit scripts were terrible to write and frequently had bugs because the scripts were usually afterthoughts "Oh yea... I guess we need one of those things".
There were alternative solutions to that problem that didn't require replacing a long and ever-growing list of system components with tightly-coupled (and often buggy) systemd versions. In fact people who really cared about their init system were already using them.
Well systemd service files are great and represent a nice improvement. But I detest journalctl with its arcane query commands and binary storage; much prefer ability to use standard unix tools and my favorite editor for adhoc inspection at least. It's still is an annoyance on a pretty regular basis.
"How do I remove" - I dunno, because I don't care to remove it. Somebody's probably done it. Search on Google? It's open source, so look at the code, submit a patch, fork if you gotta?
"Buggy attack surface" - I am unaware of attacks that target journald; can you point me to a practicable PoC attack?
"Why should I have to" - because other people are building your distros, expending their effort on open source projects, and as part of doing that they made decisions on your behalf. Feel free to not use those distros for ones that make decisions you prefer; the choice is yours.
Then you are commenting on the wrong post. This post is about the systems that do not use systemd. Justifying your own use by demanding examples of failure is... odd here.
So when systemd comes up it's cool for people who Don't Even Own A systemd to go on endlessly about how it's terrible and An Conspiracy, but it's not cool for people to discuss that it might not be the Snidely Whiplash evildoers thing by evil people when the shoe's on the other foot?
Suddenly logins take 30s. Why? Some new systemd problem with reverse DNS.
Suddenly reboots don't happen. Why? Systemd is spinning on a task that didn't used to spin. Yes, probably fixable, but not discovered until trying to reboot.
Can't login at coffee shop WiFi suddenly. Why? Some random systemd DNS issue.
Is my system more secure? Maybe, but possibly because it breaks so often that low uptime keeps it safe!
For my servers, I seriously have to cross my fingers and have the colo phone number handy whenever I reboot, fearful that they won't restart because of some random systemd interaction that happened on some previous upgrade. It's gotten that bad.
sysvinit was at least a thousand times easier than bsd init scripts, and I occasionally see people that are opposed to both systemd and sysvinit and think rc.local was 'it'.
BSD rc scripts haven't worked in the way that you think that they have, for decades.
Mewburn rc, which reduced many rc scripts to a couple of shell function calls and a handful of variable declarations, has been used by NetBSD since 2001, and FreeBSD started using it about a year later. OpenBSD, furthermore, reinvented Mewburn rc around 8 years ago. Project Trident (formerly TrueOS formerly PC-BSD) has also since then switched from Mewburn rc to OpenRC.
> /etc/rc.local was first declared obsolete in the BSD world and replaced by a system of drop-in files in a directory back in 1995.
OpenBSD still documents it as an implicitly-recommended way to start programs (though if it's possible to start it via e.g. rc.(local.)conf, then that's obviously preferable).
> sysvinit was at least a thousand times easier than bsd init scripts
¿Qué?
I guess we'll have to beg to differ here, but I prefer OpenBSD's (for example) init system over sysvinit or systemd any day. rc.subr is a godsend for eradicating all the boilerplate that "justified" switching to systemd. Enabling/disabling a daemon is as easy as adding a line to /etc/rc.conf.local (or just running "rcctl enable mydaemond", which does that for me). No fiddling with symlinking scripts to runlevel-specific folders (or, in fact, dealing with runlevels at all; if I need single-user mode, that's what bsd.rd is for).
One of these days I'd love to start ripping out the initscripts on my Slackware machines and replace them with something closer to OpenBSD (or hell, I wonder if I can just port OpenBSD's init to Linux?).
The objections to systemd are still as valid as ever. We still see a pretty high rate of security vulnerabilities in it.
However, as projects adopt systemd-proprietary APIs, distros are forced to choose between maintaining ongoing forks of those projects, dropping those projects, or adopting systemd. E.g. I suspect Slackware only avoided being forced to adopt it because they'd already decided to not support gnome.
I have to call this out as a very poor comparision. I would love to know the linux kernel commits in 2019 vs CVE's in a percentage, vs systemd commits in 2019 vs CVE's in a percentage (or 2018 or 2017).
Now compare systemd CVE's with OpenRC or runit CVE's or any other init system and then you have an argument. But comparing it to the kernel with hundreds of people commiting changes is weak.
This comment is not defending nor attacking systemd.
A quick look at the list of CVEs in the grandparent comment indicates that there are at least 3 different Assigning CNAs for systemd CVEs: MITRE Corporation, Red Hat, Inc. and Canonical Ltd.
My conclusion is that the systemd project is not its own CNA and therefore not in a position to authoritatively decide which systemd bugs get CVE numbers assigned and which do not.
> We still see a pretty high rate of security vulnerabilities in it.
Prove it. As someone else pointed out, there were 5 CVEs for systemd in 2017 and 7 so far this year most of which are low severity. If you think the mess of shell scripts and programs that systemd replaces had less than that you're simply delusional.
Earlier versions of systemd were fairly buggy, but then it was still in its infancy and its in a much better state now.
systemd (not SystemD) changes a core part of the OS, we go from a userspace being bootstrapped by a bunch of different shell scripts calling a dozen individual binaries, to an actual system and service manager configured with unit files.
It's a large change, although one other operating systems went through a decade earlier, and some people don't like change.
Why on earth do systemd haters insist on capitalizing the D? It's really amazing how people think mistyping the name is some kind of useful or novel criticism of the software.
I don't follow. As you state yourself, "names have meaning". By conventions the developers actively disregard, the meaning of the name "systemd" implies several things that the program is not.
However juvenile, it is useful – as soon someone writes “SystemD”, I know that they’re either very inexperienced and don’t really know much, or they’re deliberately being spiteful and want to signal their hatred to other haters. Either way, I then know that I can safely ignore them completely, since they’re not part of a constructive conversation.
I think that giving unflattering nicknames to powerful entities is a long-standing tradition, and a valid act of protest and free expression. Of course it won't convince an opponent, and it may backfire as a rhetorical device, but I empathise with people wanting to use it.
Moreover, the term "System D" is somewhat synonymous with the term "black market", which Wikipedia defines as a: "series of transactions that has some aspect of illegality or is characterized by some form of noncompliant behavior with an institutional set of rules."
That is a surprisingly good analogy for how systemd acts on a machine, but there is another analogy to a similarly worded term: "Substance D", from the novel "A Scanner Darkly". In the novel, it is described as an addictive psychoactive drug, also known as "Slow Death". Credit for that comparison goes to this foundational criticism of systemd:
People's perceptions aren't unified. Some people will always dislike SystemD, and they are not be forced to use it. If every major distro maintainer loves it, some people still would hate it.
There's still a group of Unix users upset about the major init split that happened fourth years ago. Things will never be unified.
I think the main problem is not the init process, but the tooling around it that breaks systems. For instance systemd-resolved did break local resolving some years ago (maybe still today).
While it is optional, ubuntu installations come with it turned on by default.
They didn't really have an option: the same organisation controls GNOME and systemd, so they introduced systemd as a hard dependency of GNOME to force distros to adopt systemd. And you can't have a distro out there without GNOME, so...
Those distros that can run without systemd have to make their own patches to "fix" GNOME, which is very costly. Even Gentoo struggles with that, as it takes them a long time to release new versions of GNOME because they have to write the patches to strip out systemd first.
>And you can't have a distro out there without GNOME, so...
LxQt and any KDE-powered distro would beg to differ.
But I agree, in general, that KDE-default distros are few and far between. And the KDE team probably has better things to do with their time than try to maintain a systemd-less fork.
They're rare in the US. KDE-as-default is close to the norm in Europe.
KDE has long been a more crossplatform system than Gnome (much better windows support (and even e.g. solaris), BSD is treated as first-class) so I'd hope they'll continue to avoid being systemd-dependent.
> And you can't have a distro out there without GNOME
cough cough Slackware cough cough Kubuntu/Xubuntu/Lubuntu cough cough Tiny Core Linux cough cough the literal dozens if not hundreds of distros that ship with a desktop environment that's not GNOME or MATE or Cinnamon or some other derivation thereof cough cough cough
> as a hard dependency of GNOME to force distros to adopt systemd
Or because they like systemd and they think the abstractions it provides are useful and its better to not have that code in GNOME. But sure, lets go with the conspiracy angle.
Also, pure serve distros also adopted systemd.
> Those distros that can run without systemd have to make their own patches to "fix" GNOME, which is very costly.
OpenSource project not implementing the features some users want them to implement. Shocking stuff. Its almost as if that's the bases of Open Source. If you don't like the choices of the project you can take the source and do your own thing.
My experience with systemd has be extraordinarily positive. I like the unified syntax of unit files. I know that's like 50% of what systemd does, but it does it extraordinarily well.
Is systemd-nspawn secure enough now we can forget docker et all? I liked the idea of a "chroot" being supported by the service manager.
Notice that the text about slackware sounds a bit "pessimistic", as if it was a somewhat legacy distribution. Nothing further from the truth. We run a small farm of virtual machines with different up-to-date distributions, and slackware is typically the one that follows upstream packages more closely. For example it had gcc 9.1, bash 5, etc. a few days after they were released.
I don't like it when my process init and logging system also tries to manage my volume mounts. If I unmount a device, I don't want systemd going behind my back and remounting it for me.
It's non standard behavior changes like this that drive systemd hate.
I've never seen systemd users make t-shirts against the others, send death threats, hijack any conversation where systemd might come up to advocate their side… non-systemd advocates on the other hand…
I find it quite convenient, although I do admit the integration with /etc/fstab is a little weird. I've used it to auto-mount application data directories, for example, before an application comes online.
Alpine is great. I have nothing against systemd but Alpine stands on it's own for containers and VMs. I haven't given it a try on baremetal but I imagine it does just as well there. Not really desktop focused but if you want to run it with a GUI on a laptop it's technically possible.
Simplicity mostly. It just works. It feels so clean, nothing running there that is not supposed to be there. Installing something takes only sub-seconds, because packages are small.
I believe your questioner was specifically referring to your assertion that audio on Alpine is unusually good. It is a somewhat odd observation, since ALSA is entirely a kernel API and Alpine is using the same kernel as everyone else. The only possible source of audible difference would be not running Pulseaudio, which resamples audio (some say, badly). However, you can install Pulseaudio on Alpine, and you can not-install Pulseaudio on most anything else, so it wouldn't really be fair to call that a feature of Alpine.
Note this is only an issue for docker images using non default tools. Not good but not unique, for instance here is another high scored CVE for PAM issues in Ubuntu/Debian https://www.cvedetails.com/cve/CVE-2009-3232/
I challenge you to find any decade+ old OS software community you couldn't make the same comment towards.
The biggest complaint I regularly see about systemd from those that don't like it is that it violates the UNIX philosophy by combining too many functionalities into one project.
Based upon this I found the following quote from the article in regards to Alpine Linux, a distribution based upon BusyBox which implements the functionality of over 100 usually separate executables into one, quite amusing.
> It is small and simple, realising the UNIX spirit to the core: doing one thing and doing it well.
The difference is that with busy box, everything is optional (just replace the symlink in /bin with another implementation), and designed as a drop-in replacement for standalone utilities (instead of redesigning them poorly from scratch).
The number of binaries is irrelevant. Systemd has a lot of binaries but is still called a monolith. Unlike systemd with journald, Busybox allows you to use Vim instead of Busybox vi without any trouble. Busybox might also emphasize the “small and simple” aspect much more than GNU tools, which have a reputation for being needlessly bloated (see all of the flags to cat, for example).
I have been running Void Linux since Debian switched to SystemD both at work and at home (including playing Skyrim SSE on Steam using Proton, or StarCraft 2 and Diablo 3 using Lutris) .
It is a very nice distro although I would not recommend it to beginners since it is a little rough around the edges if you want to do full disk encryption during installation.
Initially I switched to FreeBSD and then OpenBSD but I missed Linux conveniences like native cloud sync clients[1], Steam, Docker, support for the hardware I already had, etc.
I use void now on everything, work, home even my Raspberry PIs.
Can confirm that installation is a PITA. If you want something more fancy than the standard installation like an encrypted boot disk you won't get anything running without touching chroot a few times.
But once it runs it runs! The most "stable" rolling release distro I have tried so far. Something that can not be said about Gentoo.
The most recent reason I found to hate systemd was it replaced my system DNS resolver with its own (thus requiring me to modify /etc/systemd/resolved.conf, and doing the daemon-reload-service-restart-hokey-pokey, to get a host to function on a network+VPN like it did before). The only reason systemd has its own resolver is they wanted extra DNS functionality. Did I ask for these features? Did I want a new resolver? No. But that's the whole idea behind systemd. A few devs shoving complex features down everyone's throats, just because they felt like it. Operating system fascism.
But I don't think we should be angry at systemd developers; they can develop whatever they want. I'm mostly angry at the distributions who opted in to their crap. Systemd has not made my life easier, but it has often made it harder, and the only reason that happened was because a distro allowed it.
Debian shipping this despite the obvious backlash was the whole reason I jumped ship too, that and the frequent boot failures and looooong timeouts (which I'm sure someone will be happy to tell me are my own fault).
I've actually done sysvinit style setup before and had to migrate to systemd when it came out later. It's not that bad. It's just another way of running things. It's actually quite simple and consistent. It's a step forward.
Systemd is the worst init system except for all the others that have been developed from time to time.
Some of the trade-offs it makes, and its expansionist attitudes, are really grating. But it introduced a higher level of consistency in the Linux world for userland developers, which is almost invariably a good thing from a commercial perspective.
Nailed it. The hobbyists are revolting. Anyone interested in making their system do something odd and clever is having problems, and the people running linux systems 9-5 are busy calling us morons for trying or even caring...
I maintain systems with and without. I thought I understood systemd, but I've given up trying to predict the behavior of systems running systemd.
The tragedy for me, is that I don't really like how the Init systems work, either. So there's really no one on my side. I seem to always fall into the trap of the Dreaded Centrist.
One of my biggest concerns about systemd is the ownership. Sure it is open sourced, however RedHat has a lot of pull on its maintainers (Soon to be IBM). What is to stop them from dual licensing it or close sourcing it. They have lots of products like that (jboss/wildfly...spacewalk/satellite). After everything runs on systemd the ball in in their court. Sure people can say they are going to fork it, but in reality how often does that work out (I concede are some examples where it does, X11, nextcloud, etc.).
In my paranoid mind I can see Redhat commercializing it and adding features to it for only paying customers after they win through attrition the monopoly on the OS (after systemd incorporates everything, resolvers, cron, ntp, etc...). They are first and foremost a business with investors and a bottom line.
Again not an attack on system itself, just my untrusting and paranoid nature (to many years of security scans and cve’s and users, can’t trust users)…
The moment it stops being free Debian drops/forks it.
Furthermore, if Red Hat can monetize a system manager daemon, well ... I mean, good for them.
Oracle tried with the JVM (they had some commercial features).
Furthermore, if systemd provides a lot of basic facilities (global/user services, device management, login sessions, cron, ntp, DNS resolving) someone ought to maintain and support it, and if a for-profit company does it, I don't mind if they try to cover those expenses. (As long as it's also GPL2.)
systemd is pervasive in the GNU/Linux world and I think it’s futile to stay on GNU/Linux and try to avoid systemd. Just look at the list of distros in the article: a bunch of obscure forks of prominent systemd distros like Debian and Arch, a distro (Gentoo) whose main claim to fame is compile it yourself, one where they say a leading Google search is whether the distro is dead, and one where they say “oh gee, we haven’t heard of it, let us know if it works.”
Seems to me that if you hate systemd, it’s time to try a BSD. GNU/Linux has been going down this path of complexity for years. These same arguments spawned with udev years ago, and Slackware held out then just as it is with systemd. OpenBSD on the other hand embraces simplicity and the old UNIX way. Of course it also lacks niceties like Bluetooth, but that’s the price you pay. Maybe systemd is needed to manage complex systems that support ugly things like Bluetooth.
To be clear, you choose at install time. There is no official installer (on purpose), so the user either decides to turn to page x for OpenRC or page y for systemd.
I wish there were some way to favorite discussions on HN, not merely the article that started them -- if that feature existed, then this would be one of my favorite discussions on HN ever... why? Well, you have a ton of bright minds, who, for whatever reason(s), can't agree, yet each one adds some valuable piece of information to the debate.
Basically, where there's controversy (especially with bright minds) then there's always something there.
I hope that future system designers and historians will read and re-read these systemd controversies we are having in the present day... maybe we'll have some fundamental key piece of understanding then that we don't now, and based on it, everyone could agree... of course, then again, perhaps that would never happen...
Still, a lively, highly informative, interesting and thought-provoking discussion! A+ as far as discussions on HN go!
systemd is good, but I think it came too soon. A lot of the comments here are about memory safe languages are spot on. systemd was written with two major assumptions in mind:
1. Linux system complexity has exploded and needs a top-to-bottom rewrite
2. It's safe to do such a rewrite in C
Since the systemd project, I'd say #1 has been largely affirmed. I like the result of systemd because it provides uniform tooling around complex Linux systems.
I'd also say the tide has shifted against #2 in a big way, there's much greater skepticism that anyone on this planet (short of djb or a few others) can write safe C, let alone safe C with concurrency and side effects.
This is really unfortunate, because it probably means the solution is to rewrite systemd in a memory safe language, and that's a huge undertaking.
What's the recommended way for someone to learn systemd properly? Is there a readable manual somewhere that documents everything?
That's the only thing I find scary about such big systems, the lack of comprehensive docs and a "starting point for learning them" from 0 to hero. Like, with Git there's tons of nice tutorials like "learn kit bottom up by reinventing it in way X" etc. plus a good "default book".
Also, check the related articles on the right sidebar on the top of the article.
It doesn't document everything, but it quickly gets you to learning the overview, the basics, and what's most important for general usage of systemd.
If you really want absolutelyeverything the references are the man-pages. Excluding those that document C functions, there seems to be 256 systemd man-pages in total. 36 are regular commands (section 1; like systemctl), 64 are file formats (section 5; like systemd.service), 139 are system administration commands (section 8; like daemons like systemd-networkd), and 17 are miscellaneous topics (section 7; like bootup).
I would consider the manpages of section 1 and 5 to be the important ones, and just section 1 is enough if you never intend to read or write a service or other unit file yourself.
SysV init had the ability to restart, allowing upgrades without rebooting. Can systemd do that?
The mechanism was kind of interesting:
SysV init would create socketpair, then fork. The parent (with PID 1, required for correct functionality) would then exec the new SysV init binary with an option specified to make it receive state over the socket. The child would supply that state, then exit.
> Reexecute the systemd manager. This will serialize the manager state, reexecute the process and deserialize the state again. This command is of little use except for debugging and package upgrades. Sometimes, it might be helpful as a heavy-weight daemon-reload. While the daemon is being reexecuted, all sockets systemd listening on behalf of user configuration will stay accessible.
This anti-systemd meme is getting tiring. I generally find it is perpetuated by people who use Linux at a hobby level. Rarely do I speak with guys in Ops who dislike systemd. I am not trying to discredit people here, there are hobby users who are more knowledgeable about Linux related systems than many guys who are professionals. My point is that Linux and systemd are both meant for use on servers, in production, where speed and consistency rule.
I manage 3000 Linux servers and 20,000 services running on those servers. When I log into a machine and am welcomed by a RHEL6 motd I groan, because sysvinit is not enjoyable to use, it takes longer to reboot those machines, and tracking down logs is a chore.
When I log into a RHEL7 machine I am not elated, but I am glad to know that pid 1 isn't trying to make my life harder.
Boot time: A possible 5-10 extra seconds from sysvinit is the problem here not the 5 minutes of BIOS/EFI POST time for those server boxen?
Logs: Where's the storage of the log data in journalctl and what tools can I use to read that data in a small PXE rescue environment? But hot damn I didn't have to ls /var/log so that's nice.
Sarcasm aside how does it make your life any easier? I got a stack of FIXMEs to work on to get our infrastructure working, lot alone stable, on systemd.
Yeah we measure boot up in seconds. If I restart a machine and it takes more than ~10 seconds to boot I send a ticket to our infra guys to figure out why the server is configured poorly/troubleshoot the slow start.
>Logs
I guess I prefer `journalctl -u service.service -f` or `journalctl -p "crit" --since "2 days ago"` to the ole `ls /var/log` and then `grep`ing
Honestly, I don't understand how it makes life harder. To me, systemd is a huge QoL impovement. Even in the most mundane tasks like `systemctl restart tomcat-*`
Ops person here. When you set up a node, you set up all its networking, its services, its packages, its configuration, etc. It doesn't matter what your technology is - you set it up to do whatever it needs, and then hopefully you never have to re-do that again. Systemd's hydra of services affected a lot of that, so we had to learn all the weird gotchas to get newer distros to work properly.
And it's not just a little different, it's more complex. This increases the number of bugs, the time to test and roll out changes, integration work, etc. So the process is more complex. And the operation is also more complex, meaning more training for anyone who's used to how things worked for 30 years. Doing minor ops work now requires looking at 5 different manuals, requiring extra software, adding configuration management where it wasn't before, changing support instructions, healthchecks/automation tasks, migrations to newer disk images while supporting legacy functionality, etc.
And here's the part that really kills me. Alternatives to all the parts of systemd already existed; if you really needed a better system, you could pick up any similar software and make it work! But then distributions start rolling this out en-masse for all new releases, and we don't have a choice to opt-in. It was a breaking change that did nothing for us that we didn't already do for ourselves, even requiring more work than before.
Do our systems boot faster now? My EC2 instances still seem to take forever to spin up, and physical machines still take ~5 minutes to POST, so I'm going to say no. Can we do new things that we couldn't before? No. Are our systems more stable now? No. Are they less complex? No. So while some people talk up the theoretical benefits, I as an ops person have not seen them. But I still have to look up man pages to remember the three different filesystem trees systemd keeps files in, or the wonky configuration files, or what 5 extra commands I need to get something done.
If you're going to make breaking changes, at least have it make my life easier after I get used to it. Systemd has not done that. And it seems to just keep getting heavier.
Our bottle neck is often pid 1 for boot time. We don't do mem check on boot so that saves us 1 second * Gb_of_RAM in boot, which is why most BIOS take forever on a server. We also don't run anything in PCI slots so we have the PCI checker disabled too (all our servers are Dells they have some software you can license to config the BIOS).
I'm new to the game, I only vaguely remember a time when Ubuntu wasn't on systemd. I don't have 30 years of experience with any init system, but I do have 10+ years with systemd. I didn't use another init until I got this job and had to learn sysv. So I'm coming from the exact opposite, it sounds like, place of you. To me, everything you said is 100% true, but about sysv and its line.
I also don't provision too many servers these days because with the exception of a few things that were deemed "to important" all our servers have automated build scripts to go from 0 to prod.
Being dismissed as a meme is tiring as well. I like hacking on my system at boot time, and dismissing my use case because of your own is kinda hostile.
If you are referring to my comment regarding Linux not being for desktops...
I use Linux on my desktop. The only non-Linux machines in my house are my work Windows machine and my sister in law's Windows machine. All my personal machines, my wife's laptop, and my kids' desktop are all Linux flavors. Linux is a great desktop imho. That said, Linux is not primarily a desktop machine. It gets some pro-Desktop patches but at the end of the day, Linux is designed for servers and paid for by people interested in its usage as a server.
I have the feeling systemd is meant exactly for what you call hobby or novice users.
I actually enjoy some of it on my laptop to be honest ;-)
But in the datacenter it has caused me hours and hours of 'fixing problems that did not exist before systemd came along'.
Most of it's 'improvements' target the desktop user.
Many of it's 'improvements' actually get in the way in the server room.
I mean, why would I want to optimize my boot time on a server with all this parallel startup crap?
The server spends 10 times more time in BIOS and POST then it takes to boot the OS.
It has always been like this. And it still is like this.
From a config management point of view I like systemd. I gives me a consistent API to enable/disable start/stop services. That's awesome. I also like that I can 'override/patch' upstream units without having to maintain a fork. Perfect.
But many other things that came from the systemd ecosystem are questionable.
E.g. systemd-resolved, timesyncd and that kind of stuff.
Maybe these are nice for desktop users who want a click and play experience on their laptop.
No problem with that. But why do they have to break my server's for this?
I mean seriously: Linux on the desktop is irrelevant. Why optimize the whole OS/init for a irrelevant use case?
Boot time is a big deal for us. Not sure if it matters, but I don't get paid enough to decide what matters. All I know is we paid extra for some Dell software that lets you fine tune the BIOS and we streamline for boot speed. We're currently in the process of moving anything that can be run on a VM to a VM so this will be even less of a concern in the future and pid 1 will be more of a bottle neck. That said, except for the db servers, we don't run mem checks on boot so our servers are very fast to boot (I believe Dell cites that each Gb of RAM is equal to 1 sec. of boot time when you are running with the mem checker on).
The only time I ever do a "deep dive" and manual setup a server is when I have to provision a server for a new department/service. Generally speaking ansible does the work for me and I just hit go on my console and then go get a coffee. So I don't see much of a headache in server provisioning. I also am on the automation side of Ops so I do a lot more work with a already setup server, and in that case, like you said, systemd API is a godsend.
This list is missing a linux-ey system which doesn't use Systemd that is growing in popularity!
WSL!
WSL somewhat annoyingly has a custom/proprietary init system. Partially due to it's "interesting" integration with windows, and partially because it's not even really a full linux system yet.
I'm curious if with the switch to WSL2 where they will begin shipping a full linux kernel if systemd will be included or will they stick with their current custom solution?
I think you mean GNU/Windows, or as I've taken to calling it, GNU Plus Windows. (Seriously though, the WSL name is deceptive because it doesn't actually contain any Linux code. There is no Linux in "Windows Subsystem for Linux")
Yeah but as far as I can tell that doesn't actually exist yet, and isn't scheduled to be available until the end of next month. (Existing inside Microsoft but not publicly is "does not exist" as far as I'm concerned, because I remember all kinds of legendary stuff Microsoft has apparently made but never released.)
I went on angry binge where I Ubuntu finally pushed me over the edge with it's latest... och. Between that and systemd, I've starting to dislike Linux, pretty much for the first time since I started using Linux back in the early 90's.
The second real, functional, genuinely working docker support shows up in *BSD and is working - I'll be running it on my desktop.
I do find it funny that people crap all over systemd, but don't seem to have a huge problem with launchd, despite the fact that systemd was almost a direct port of it.
Personally, while I do think that systemd's scope might have been a bit too ambitious, I do find it easier than something like Upstart
systemd was not a port of launchd, direct or otherwise. The problem with launchd being difficult to port off MacOS is one of the reasons why launchd never came about for FreeBSD.
systemd was not a port of anything. And its immediate inspiration was Upstart, the system that Fedora was using at the time, and whose drawbacks provoked Lennart Poettering et al. to start working on systemd.
(Interestingly, Upstart itself was inspired by the drawbacks of porting launchd.)
> systemd was not a port of anything. And its immediate inspiration was Upstart
Its motivation was perhaps upstart's shortcomings, but systemd was definitely inspired by the design of launchd and that was repeatedly and explicitly mentioned in Lennart's very first blog post introducing it:
> But first, let's clear a few things up: is this kind of logic new? No, it certainly is not. The most prominent system that works like this is Apple's launchd system ...
> For socket activation we support full compatibility with the traditional inetd modes, as well as a very simple mode that tries to mimic launchd socket activation and is recommended for new services.
> More importantly however, it is also our plan to experiment with systemd … to make it the ideal session manager… The problem set of a session manager and an init system are very similar… Using the same code for both uses hence suggests itself. Apple recognized that and does just that with launchd. And so should we…
> Writing Daemons:: … The list above is very similar to what Apple recommends for daemons compatible with launchd. It should be easy to extend daemons that already support launchd activation to support systemd activation as well.
No, that's not explicit. That's a few instances of "like launchd". Where M. Poettering actually is explicit about inspiration, xe writes the words:
> we took a lot of inspiration from Upstart's code-base
... which they actually did and continued to do, from observation. There were significant discussions amongst the systemd people involving the way that things worked with Upstart, including (for just three examples out of the many over the years) having to work like Upstart with respect to putting the X server on the first KVT, Upstart's handling of SIGPWR to process #1, and eliminating /dev/initctl because it was not really utilized in Upstart.
In comparison, there was almost zero similar consideration of launchd. The idea that launchd inspired systemd is a myth, not borne out by what M. Poettering explicitly wrote was the inspiration nor borne out by the evidence of what the systemd people did down the years.
I think we're going to have to agree to disagree about what "explicit" means, and possibly also "inspire." (Based on the excerpts quoted verbatim from Lennart's blog post in my previous comment and general understanding of the English language.)
Ah, you may be right, that's what I get for trying to remember a talk without double-checking. I thought I remembered that systemd was a port of launchd here [1]. My mistake!
If you use a Mac, you don’t have much choice about your init system and probably don’t get into idealogical arguments about them as often as Linux users do.
Plus, there’s always the BSDs, which don’t use systemd but still feel very clean and cohesive.
I remember liking how easy it was to set up startup scripts in FreeBSD, so I think I like it? I'll admit I haven't used it as much as systemd or Upstart, so I can't give a strong opinion.
> I do find it funny that people crap all over systemd, but don't seem to have a huge problem with launchd
Because I've never been forced to use launchd. Well, you could say that I've never been forced to use systemd either but that's kinda how it feels when distros swap it out for you and things start depending on it.
Guess what? I'd be furious if the distros I use forced me onto launchd all of a sudden.
AntiX Linux is my current favorite which happens to be systemd free and runs with lighter weight DEs. It takes up only about 500MB of RAM sitting still, so pretty great for older systems and laptops.
We absolutely love Alpine, but it's been hard to integrate with other systems because of it using musl. For instance, our dns resolvers do not work properly & our monitoring solution is incompatible with musl.
I would also like to know what the rationale is for this behavior. Showing immediate and useful feedback on the screen seems an obvious thing to do, especially when running a command at an interactive terminal.
I too was furious about the existence of systemd, but now I realize that was mostly because I don't want new things and consider many new things to be bad because they are new and there have been a few 'bad apples' in my personal experience
If you are still arguing about systemd, you don't know enough about it, so you should learn about it and you don't understand the big picture. Users crying about Linux desktop all the time, but systemd has a huge part making the situation way better.
> Users crying about Linux desktop all the time, but systemd has a huge part making the situation way better.
Definitely. Maybe it's not great that GNOME has a hard dependency on systemd ( I personally don't care), but most people don't appreciate how much things have improved on the desktop with systemd.
I've been using linux since kernel 2.2. Deployed hundreds if not thousands of workstations for home users and developers. Trust me, I've seen desktop ups and downs, and I disagree. Systemd works until it doesn't, and then it's an unfixable lost cause and you reload it like a naughty 9x box.
Yeah, same for me. Since about 2.2.
And I remember patching init.d scripts on Debian, adding sleep 5 In the network script, because you know our service had to have eth0 up and configured.
So it's not like the old init system was perfect either :)
Or when I was building VM appliance with dB inside, and I was reading the news about systemd and how it would allow cpu and ram limits per service and thinking how if I had it it would save me from weeks worth of work (yes IBM DB2 is total crap when you run it in VM).
and so on.
When I look back of all the stupid things I had to take care of and in systemd are just a config away, I can't help but wonder why people dislike it so much.
You were able to do those hacks because it was open to you at run time. You COULD fix it. Now if it breaks and there's no fix, you may as well have used boxed software because your options are only to wait for a fix or find something else. Honestly, I'd have very little problem with systemd if it was not written in a compiled language. Losing that flexibility sucks.
In another comment I noted that this is obviously a divide between the hobbyists and the 9-5 linux users. I am both, so this is particularly sucky.
I get that not everybody likes systemd, but a lot of the criticisms seem misguided and looking at sysvinit with role-colored glasses.
For example, people are complaining about logind being included, but forget that consolekit was unmaintained before that. Yes, there's parts to systemd that not everyone will use, like container support, but in that case you can pretty safely ignore that use case.
For me, systemd has provided a much more consistent way of managing services. There's a tightly defined service definition format that allows one to comfortably modify any 3rd party service as well, (unlike random, variable-quality shell scripts), relatively small amount of commands to learn that work consistently across services, mounts, timers etc. There's also much better handling of modern hardware, hotplugging & the like.
I can't say I miss sysvinit. Also, stuff like [1] from the anti-systemd camp doesn't inspire confidence.
1 - https://lwn.net/Articles/786593
Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
It is easier to configure and is convenient to have one holistic system in a lot of ways, but...
They pushed for Gnome to have a hard dependency on it so that everyone would be _forced_ to use it instead of just letting it be accepted by merits.
They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
I don't think the code is high-quality, but even more than that I have a problem with the people being controlling, dismissive, and deceptive.
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
Additionally, this is not likely to stop, since they took an init system that was (mostly) written in a memory-safe language (bash) and rewrote it in a memory-unsafe language (C++), thus enabling a whole class of systemic security issues that will keep popping up in the future over and over again as well. This is one of the design decisions behind systemd that irks me the most.
> they took an init system that was (mostly) written in a memory-safe language (bash) and rewrote it in a memory-unsafe language (C++)
While Bash is memory safe, it is also very error prone to write and the potential of shooting yourself in the foot with it is rather big, especially with bigger scripts and considering these are executed mostly with root privileges, bash is not an ideal choice either.
I'd prefer if systemd was a Rust project for sure, but they went with C, presumably to remain consistent with the kernel, GNOME and most lower-level Linux system software, (also, Rust was pre-1.0 at the time).
Memory safety is not the alpha and omega of programming languages. People do not write million lines of code applications in bash, for a reason, trade-offs are to be made.
That said I agree they should have picked Go instead /s.
Sure, there are other concerns and bash is not an ideal language, but memory safety is responsible for quite a large number of security flaws, since its lack turns bugs into code execution vulnerabilities on a regular basis.
I don't really have an opinion on what language should have been picked instead, though OCaml may be a good candidate.
To be fair, erlang would have been the better choice here. Memory safety, concurrency and solid error handling all in one.
I think I know what my next Erlang project's gonna be. Erlang-as-PID-1, here we come!
EDIT: or maybe as PID 2, per https://github.com/omisego/ewallet/issues/108 , though I'd be interested in the idea of writing an OTP application that can reap zombies and forward signals.
> People do not write million lines of code applications in bash, for a reason, trade-offs are to be made.
Isn't that basically what SysVinit and a good-sized chunk of the management tooling in Linux is?
To their credit, Bash is a _weird_ language, but it's not going to corrupt memory when it crashes, and its failure modes are pretty well understood by the distro maintainers writing those scripts.
Those are all not million lines of code applications, but that only speaks in favor of a more traditional Unix userspace design. With modularity on the process level you don't need a million lines of code to herd services. Why an init system should a million lines of code is a mystery to me.
I'm not such a lover of Go, but: non-sarcastically, Go may well have been a good choice.
> Lennart Poettering and Kay Sievers started the project to develop systemd in 2010.
https://en.wikipedia.org/wiki/Systemd#History
> Go was publicly announced in November 2009,[29] and version 1.0 was released in March 2012.
https://en.wikipedia.org/wiki/Go_(programming_language)#Hist...
The brand-new language that Google just announced last year was probably not considered a serious option here.
It's 2019 and Go still has a much weaker toolchain and infrastructure story than C (which has a 30 year head start).
Ha yeah, I didn't mean "they really should have used Go", I just meant, "Go as it exists today may be a good thing to use for this".
Given the timeframe, something like OCaml would likely have been a better option.
Ocaml parallelism/concurrency story was not that good - but Haskell has just got its fancy new epoll based I/O manager (100000+ lightweight threads) around that time, with the 7.0.1 release (16 November 2010)
They could have even take inspiration, imagine: typed, functional, monadic init files - "systemd: avoid success at all costs" :)
The topic is memory safety, not concurrency support, FWIW.
Lua would have been a good candidate: memory safe, coroutine to structure the code in a concurrent-ish way if needed, and you have a configuration file parser for free (just use Lua as the configuration language, it was its very first purpose after all).
That's nice in theory, but where are all the people who know Ocaml to write it and contribute to it ? I might be clueless but I can't really name any significant projects written in it, which might imply something about the number of people using it and being able to work on and contribute to something as large and important as systemd.
>I might be clueless but I can't really name any significant projects written in it
From system programming solely: libguestfs, Xen API, liquidsoap, Unison, MirageOS, google drive on fuse, 0install.
>Ocaml parallelism/concurrency story was not that good
OCaml concurrency is great, and Lwt [1] would be exactly what systemd need: run daemon and poll the answer concurrently.
Ada, C++ and D would have been another great choices.
[1] https://github.com/ocsigen/lwt
> Ocaml parallelism/concurrency story was not that good
I wouldn't expect CPU-bound multithreading to be much of a concern in an init/rc system. And resource thriftiness and predictability would likely be a bigger concern.
While that would be amusing for sure, systemd's pid 1 is actually single threaded.
Rust would have been more appropriate. Something like SystemD at such a low level in terms of hardware and service management should not be done in a language with a garbage collector imho.
> Lennart Poettering and Kay Sievers started the project to develop systemd in 2010.
https://en.wikipedia.org/wiki/Systemd#History
> Mozilla began sponsoring the project in 2009[16] and a nnounced it in 2010.
> The first numbered pre-alpha release of the Rust compiler occurred in January 2012.
https://en.wikipedia.org/wiki/Rust_(programming_language)#Hi...
Look, I like Rust and all, but are we really suggesting it for projects that predate public release of rustc?
I understand that... Go still isn't an appropriate language for SystemD, also, when did SystemD really take hold? Lots of projects will shift underlying options with time.
I understand "would have" in this thread as a hypothetical situation where the option had been available.
systemd predates Rust by a bit over a year.
The point stands, Go isn't a good option for this type of process.
The only thing that I can think of for that 2019 Go that would be a problem for this use case is binary size. None of the rest of the usual complaints would be stoppers or even that big a deal, and most of them, I'd pay to have a memory-safe language being used at that level. Init systems are a type of code that we don't have a good word for, but that I tend to end up in a lot, the code that isn't CPU intensive, or disk intensive, or RAM intensive, etc., but is all just logic and correctness rather than any of that. So "but Go is GC'ed!" isn't particularly relevant, for instance, because an init system pretty much ought to settle into a steady state and not be executing at all, on average, so who cares about a handful of microsecond GC pauses that may or may not occur at startup time?
Rust might be better now; my primary concern would be community size and whether I was limiting my contributor base, which Go also has as a concern. In both cases though I'd take it over C/C++ being used on such a critical project at this juncture.
In 2010, though... yeah, choices are a lot worse.
I understand there's a much more limited subset of developers for Rust or even Go vs. C/C++. Also, I recall hearing about Rust before systemd, but am not sure. Today, it would probably be my first choice. Not that go is a bad choice, I just don't think it's a good spot for it. I also agree that either are probably quite a bit better than C/C++ options.
I'm not against go, other than gc and some size considerations. Given that a lot of k8s infrastructure for the likes of CoreOS and similar are written in it, it's definitely not a bad option. I think that today, and more so in late summer as async/await syntax settles, that Rust should be a first consideration for any low-level system code.
It doesn't execute low level code in terms of hardware. It's a init system, all it (should do) is create a dependency graph and initialize the services in the correct order.
Many init systems have been created in bash, and I'd hardly call that a low level language.
> People do not write million lines of code applications in bash, for a reason
And that reason is that one rarely needs to write a million lines of Bash: http://catb.org/esr/writings/unix-koans/ten-thousand.html
If they'd picked Perl, they could have written an init system in one line... (hopefully, obviously /s)
> (hopefully, obviously /s)
You underestimate my masochism and/or overestimate my sense of restraint ;)
Bash is memory safe until you run 'rm -rf $DIR/' when DIR is unset.
Let's just say the language choice went from bad to equally bad.
If you run or write that command into script without checking what DIR variable contains, the problem is in your lack of experience with shell, not the language choice. Also nowadays, rm itself has --preserve-root as default, so this won't delete your root fs.
By the same logic, if an app written in C has a security issue due to a null pointer or buffer overflow, the problem is clearly the coder's lack of experience with C.
Actually yes, because it is possible to write correct C, and because they could have chosen other language less conducive to those pitfalls. But your analogy doesn't hold, because in case of shell, the situation is different: the pitfalls are easy to avoid, and for unix admin tasks, there is no compelling alternative to shell.
This is true if you are running the GNU userland. It's not necessarily true with other userlands that are often run on top of Linux. In a discussion touching on systemd, it strikes me as a little surprising that somebody would rely on a quirk of another replaceable component. ;)
I’m sure the irony is not lost on you regarding the fact that systemd can not possibly run on an alternative platform, thus the situation you just ascribed is not “better” with systemd.
Sure. That wasn't in favor of systemd, though? I comfortably write code against the GNU userland and I use systemd where it crops up. Just not a big deal to me; if somebody wanted to use my stuff elsewhere I'd just say "pull requests welcome!". I found the inconsistency striking, that's all.
Checking for content of a variable before passing it to rm is a basic instinct that should be done in all shell scripts. The comment about --preserve-root is just a reminder that the problem of deleting / is very rare.
That's not a memory-safe error.
do beginner bash guides not tell you to "set -u"? all the system scripts on my system have it
it causes the script to terminate with an error if it encounters use of an unset variable
Not just -u - always <set -euo pipefail> unless you have a good reason. Fail on non-zero exit code outside of some special cases, undefined variables, and non-zero exit codes in a pipeline. And always use shellcheck, shellcheck is too good not to use tbh.
Are you saying something in C++ is preventing you from issuing that 'rm -rf' command?
He is saying that Bash can also cause catastrophic behaviors by an as simple mistake as unsetting $DIR. C++ at least reject to compile when a variable is not set.
Any language can cause a catastrophic scenario like that. It's not a feature of the language. Not even the goal of the language to protect from something like rm -rf. The variable could have been set to empty string and C++ would just comply with what the programmer wrote (not what they meant, but that's a completely different thing).
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
This seems like it needs some citation. There certainly have been bugs. I think it's legitimate to talk (with specifics!) about cultural issues with systemd's development and its impact on security. But I don't see that this statement is particularly well supported. In particular I genuinely don't know what you're referring to with respect to ignored/suppressed bugs and CVEs.
I mean, systemd is a lot of code and makes up a big chunk of the system plumbing these days. Obviously it's going to make up a big chunk of the bug budget. People don't get upset about flaws in someone's random cgroup launcher or encrypted block device manager or daemon wrapper not because those projects don't have bugs, but because basically no one uses them. Because basically everyone is using systemd!
For the CVEs, it's the comments like this: https://github.com/systemd/systemd/pull/5998#issuecomment-30...
(Just in case, this is an security issue because it enables constant dosing, as well as clears the resolver cache making spoofing easier)
I think that's valid enough. Not sure it rises to the level of "OMG they ignore bugs" implied by the original comment though. I mean... they took the patch and fixed the bug, right? Poettering was just being a jerk about whether it deserved a CVE.
(Also... gotta be honest that's sort of a vanity CVE. Note the complete lack of details in the report itself, it never saw an eyeball after it was filed. It's valid enough, but frankly I'm with Poettering on the specifics here: if we filed a CVE for every theoretically-exploitable-but-unexploited-until-fix-released crash bug the system would be beyond useless.)
I think that's valid enough. Not sure it rises to the level of "OMG they ignore bugs" implied by the original comment though. I mean... they took the patch and fixed the bug, right? Poettering was just being a jerk about whether it deserved a CVE.
Poettering is dismissive of nearly every bug. Remember when systemd would default to starting units as root when it couldn't parse the username? We're not talking about some minor piece of software like a sound multiplexing daemon, we're talking about the core userland piece of every Linux install. Bug reports on something that central should be taken seriously and with a minimum of snark.
I'm with Poettering on the specifics here: if we filed a CVE for every theoretically-exploitable-but-unexploited-until-fix-released crash bug the system would be beyond useless.
All the more reason to reduce the attack vectors and ensure that your code is well thought out, and coded defensively. That you have too many security issues to generate CVEs for each and every one is a problem in and of itself. Ten years ago how many people thought that all of these speculative execution side channel attacks were possible or practical?
This is always the case with any new software. Things may get a bit worse before they get better. Honestly Systemd addresses the complexities of a modern system in the simplest fashion but no simpler, and that is the key. It is in the same spirit as launchd on macOS. It works well overall and end up being more portable and consistent in the long run. Hatred for it seems to come mostly out of dogma, something to be avoided in this age.
> addresses the complexities of a modern system in the simplest fashion but no simpler
I have a serious contention with the statement that systemd is the simplest possible solution for this problem. Especially since it involves not only service management, but user logins, a data bus, logging management, file system mounting, initd, and a host of other things.
Are all of those pieces necessary? Arguably yes. But not all in one system(d).
Thing is, though, most of those things are different systems. systemd-logind systemd, systemd-udevd, and journald (and others) are different systems in the same project. They're a bit tied together (through dbus and I'm not sure what else), but that doesn't mean they're the same system.
The problem is that they're not really swappable. So, effectively it is all one system(d).
> This is always the case with any new software. Things may get a bit worse before they get better.
So why should I currently opt for the worse solution? Is the overall design of it promising?
> Honestly Systemd addresses the complexities of a modern system in the simplest fashion but no simpler, and that is the key.
What complexities does it address? I lost track of everything an init system is apparently responsible for.
> It works well overall and end up being more portable and consistent in the long run.
Since systemd only runs on Linux APIs, I can't figure out what this even means. There are init systems that run perfectly fine in plain POSIX environments. systemd is not one of them.
I don't believe typical developers use "plain POSIX environments," whatever that's supposed to mean.
I'm not going to discuss the merits of portability or what "typical developers use". I'm questioning the idea that systemd is somehow more portable than its alternatives. That belief can only be rooted in plain ignorance of the actual situation. It has absolutely nothing to do with what typical developers use, and to that end your comment is a pointless derailment.
What I mean is that there are init systems that only require POSIX compliant APIs and as such are portable across systems that implement POSIX, e.g. GNU/Linux, BSD, Solaris, AIX...
systemd requires (in addition to POSIX) Linux APIs and is therefore not particularly portable by any reasonable definition of the word. You can use it if you are running a Linux kernel, and that's it.
> What I mean is that there are init systems that only require POSIX compliant APIs
Those are irrelevant as they can't even do basic tasks like shutting down the computer or restarting it. Feel free to find a POSIX API for Linux' reboot syscall.
Some systems also require mounting additional filesystems, again not possible in a world limited to POSIX interfaces.
Sandboxing, SELinux, ...? Not in POSIX.
And so on.
launchd doesn't break nohup. It doesn't have any opinions about processes started from shells. systemd does, which is completely inappropriate scope creep.
I agree with the systemd developers on this one. It's weird that processes started by users when their logged in by don't end when they log out unless they voluntarily terminate when sent a signal.
And nohup was always a hack. Now that we actually have a functioning user-level service manager keeping processes alive by throwing them at PID 1 to parent needs to go away. I really hope that systemd eventually uses subreapers to keep all user processes under the session leader.
I feel like it's weird to talk about scope creep when logind is pretty much the first system to define what a session even means on a Linux system.
If a process is interactive, it needs a way to be notified that its user is gone. That became a distributed system problem a while ago. PID 1 doesn't know whether an X client is active on a remote display, or a batch job is doing useful work.
you know in a system with a gui where multiple users are working it's really really stupid to keep a process running?!
But I think that the author of systemd himself wrote a patch for Gnome to keep it working without systemd. Am I correct?
My understanding is that it the patching of Gnome to work without systemd was done by others (Guix, Void developers), not Poettering: https://www.reddit.com/r/linux/comments/66ass2/update_void_l...
That would invalidate a lot of the criticism I saw.
> They pushed for Gnome to have a hard dependency on it so that everyone would be _forced_ to use it instead of just letting it be accepted by merits.
We should standardize on the interfaces, avoiding hard dependencies. There is dbus for that.
dbus is also a dependency though. If you are on the bus (heh) that says systemd is a hard dependency that should be avoided then dbus is also the same thing.
I agree with your logical consistency, but I'm curious: is D-Bus sufficiently specified that one could write an alternative to it? That is, is it a hard dependency or is it itself an interface? I'm not nearly sufficiently experienced with desktop Linux application development to know. ;)
Yes, DBUS is specified [1]. There are multiple implementations shipping (dbus reference implementation and dbus-broker), as well as several different client libraries.
As dbus user, or application, you just need some client library. As an operating system, you also need a daemon running, which is routing the messages among all the clients.
[1] - https://dbus.freedesktop.org/doc/dbus-specification.html
So it is--and, in reading, it looks like systemd actually ships their own sd-bus client. Huh. TIL. Thanks.
IIRC at least part of the reason for sd-bus is the need for using D-Bus between parts of systemd during early boot when "normal" D-Bus is not yet up and running.
I'm not a huge fan, but dbus at least runs on FreeBSD and others.
FreeBSD developers are actually looking at something like systemd themselves. The fact that systemd itself doesn't run on it is just the consequence of POSIX being inadequate for its implementation, since it relies on a bunch of Linux specific features, like cgroups etc.
I am personally in the camp that doesn't mind that, Linux users should get the best possible software given their system capabilities, not the lowest common denominator one just for the sake of it. (Free)BSD developers are also not looking to make sure their stuff works with Linux, which I think it's a valid approach.
GNOME itself does exist on the BSDs, by the way knocking down the argument of systemd being a hard dependency of GNOME along the way.
> FreeBSD developers are actually looking at something like systemd themselves
If this claim were true it still is beside the point of my comment, that systemd itself has a "Linux only, and we don't care" approach, and dbus for the moment is not that.
> GNOME itself does exist on the BSDs
Yes the ports tree is a wonderful place for those things to sit for those who are interested and for a larger portion to not bother installing them. I don't know how many patches they applied to get it working or what the cost and effect of those patches may be over time.
Yeah it is. It is a bit complex, but it has a spec that should allow for independent implementations (AFAIK KDE has its own, for example).
> Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
The 'Imperialistic Culture' you speak of is one of those made-up things that systemd haters keep repeating, but has no basis in reality.
> They pushed for Gnome to have a hard dependency on it so that everyone would be _forced_ to use it instead of just letting it be accepted by merits.
Again, another fairy tail.
> They have a track record of labeling serious bugs as non-issues, ignoring them, or just not filing them as CVEs. Serious issues keep coming up.
A) This is par for the course in Linux software. See also: Linux kernel. One side you have is a bunch of security bros running around trying to fluff up their resumes and trying to turn themselves into heros. And on the other side you have devs working hard on various features and bugs and being resentful that most of their work goes unnoticed while stuff they already fixed is thrown back in their face.
B) Many of the bugs that make it onto Reddit/Hacker news are going to be ones that are largely changes in configuration defaults or behaviors that break things and are meant to be configured by the operating system designers that use it, not end users.
C) This level of scrutiny never existed before for low-level Linux features. The multitude of redundent and terrible shell scripts that made up the majority of functionality that systemd project seeks to replace was always a swampy mess of broken functionality and bad code. Having each and every major linux distribution rewriting 90% of it from scratch in a completely un-portable and estoric way didn't help matters.
Linux advocates actually went so far as to praising the fact that even though half of the scripts in their OS were completely broken and the OS still worked fine was a testament to it's robustness.
The fact that now, finally, after decades of this crap people can now track bugs related to low-level Linux userland 'plumbing' is actually a good thing.
> It is easier to configure and is convenient to have one holistic system in a lot of ways, but...
On the sysvinit side you had thousands and thousands of lines bunch of procedural code of dubious quality that is endlessly rewritten by hundreds of different teams with vastly differing levels of competency and success using a general purpose language for configuration. It was so unportable that you couldn't even use anything but the most basic and trivial (also broken) init scripts from one distribution to another without herculean levels of effort.
On the other side you have a even driven OS configuration and management engine that allows people to describe the state they want the OS to be in via a domain specific configuration language. It has managed to standardize the low level plumbing for Linux operating systems and allows distributions to share improvements with one another in a way that was previously impossible.
If people want a alternative to gain in popularity they need learn what systemd does and understand why Linux distribution makers switched to it. Then produce something of their own that takes those positive features and improves on it further.
All the hand waiving about 'Imperialism via releasing open source software to the public' is infantile.
The upside of all of this is that it's going to be a hell of a lot easier to make systemd unit files portable then it is to make Linux init shell scripts. There isn't any reason why a more capable init system wouldn't be able to parse existing unit files and know then how to start and manage the services and applications that use them... So the effort to move away from systemd should be significantly less then the effort to move to systemd.
> On the other side you have a even driven OS configuration and management engine
You are repeating the there-is-only-sysvinit-and-systemd fallacy, pointed out years ago by the Uselessd Guy and decried by many others since but (alas!) still going strong today in this very discussion, and ironically you are doing it whilst erroneously ascribing things to systemd that it was intentionally designed not to be. systemd was explicitly designed not to be an event-driven system. The wholly event-driven system was Upstart, and in practice it turned out to be a problem. Lennart Poettering xyrself discussed the problem, as did some of the Debian Technical Committee members during the Debian Hoo-Hah.
* https://web.archive.org/web/20190306213420/https://uselessd....
You're really not refuting anything here. Whatever bad situation existed with previous startup across, it doesn't excuse bad handling of issues in systemd. Whatever other projects do with CVEs, doesn't excuse Lennart complaining that CVEs should not be raised.
The imperialism part is likely due to score creep into parts they just can't handle well enough. systemd-resolved for example had terrible security and functionality issues still after it was being deployed to many distros. These were simple "point a fuzzer at it and watch it crash" issues which should not be seen in a DNS resolver these days.
> Many of the bugs that make it onto Reddit/Hacker news are going to be ones that are largely changes in configuration defaults or behaviors that break things and are meant to be configured by the operating system designers that use it, not end users.
Maybe it's not for systemd to decide what I'm meant to do. If changes to behavior that they only meant for operating system maintainers to rely on results in angry end-users, they probably made the wrong assumption.
> On the sysvinit side you had thousands and thousands of lines bunch of procedural code of dubious quality that is endlessly rewritten by hundreds of different teams with vastly differing levels of competency and success using a general purpose language for configuration.
Meanwhile, I have ~400 LOC of init scripts (including service scripts) on my daily driver, an init system with <1% the LOC of systemd, and plain text readable system logs. I agree that hand waving about "imperialism" isn't very constructive, but on the other hand I don't think that misrepresenting the alternatives to systemd and exaggerating their faults is particularly useful either, nor is pretending that sysvinit is the only alternative. A lot of people have thought "sysvinit sucks" and done something about it.
Are your init scripts available for viewing anywhere?
At the end of the day it is the handling and haptics that count. So let me give you some anecdotical experience. A few weeks ago when the new Fedora came out i tried it as a live system from usb keychain, out of curiosity. It was the xfce live respin. Anyways, long story short, clicked around for a while, wasn't impressed much, shut it down. Flicker, flicker, the usual console messages scrolled by, and then it was stuck trying to unmount somesuch yadda yadda yarr yarr... for minutes. I wasn't in a hurry and waited, curious WHAT would happen. Nothing, just finally powering off after 2,5 to 3 minutes. I sat there and wondered: could it really be THAT slow? Was my usb-stick slow? Nope. Tried another stick, another laptop, even some other respins. Always the same. Dumbstruck of the audacity of the so called leader in Linux i just thought to myself that they must be on crack or some other bad stuff. Because when i tried a few other systemd-free live systems on the same sticks and hardware everything just whooshed by. Why is that? Are they living in a parallel universe? Do i have outliers in terms of hardware? At least in one case i don't think so because it is a very well supported Thinkpad which accepts almost anything i can throw at it, and the other a similar HP Elite Book. So there is that. Now what?
bzzzt! https://youtu.be/ulvgWoChvBU?t=9
Gotta love the amount of anecdotal evidence in here.
bzzzt! https://youtu.be/GHhD4PD75zY?t=157
> I have a problem with the people being controlling, dismissive, and deceptive.
They're probably just busy.
And calling the "imperialistic" is annoying them, so you've destroyed your political capital with them.
> Most of the problem I have it is with the imperialistic culture that came with it and with the serious security bugs that they keep sweeping under the rug.
The most recent gripe I have is that systemd people can't even do randomness properly and even use non-cryptographic randomness for DNS transaction IDs (ok probably benign). But just boggles the mind.
https://twitter.com/FiloSottile/status/1125840911299223554
If systemd were a gstreamer plugin it would be in gstreamer-plugins-bad
When there was a problem with a sysvinit procedure, one could simply fix it. As a result, mostly nobody ever had a problem with it.
When there is a bug with systemd, you have to fix systemd. And your version of systemd is completely different from the head, because head was rewritten last month (in a process that persists bugs) and yours is a year old. You can't just use head either, because every low level system on your computer is tied to systemd, and they work differently on head. You won't fork it either, because it's so large that a single person can't keep up.
Without a doubt, a declarative init is much better than an imperative one. But that relatively small number of commands is no gain if each command is more complex than the entire set it is replacing, and I'll completely disagree on the handling of modern hardware, hotplugging and the like. Besides, there is the entire logging saga against systemd.
> As a result, mostly nobody ever had a problem with it.
The number of broken init scripts I have encountered throughout the years, that went unfixed for months if not years, stand in stark contrast to this statement. And I’m talking packages like uWSGI on Ubuntu 14.04! This is exactly why I gravitated immediately to systemd. The idea that init scripts actually worked and when they didn’t were easily fixable is not true. Yeah you can fix a broken script locally, but this doesn’t fly at scale n > 1. Submitting patches to distros is not a simple process, and not one that makes sense in this context either. Declarative beats imperative 9/10 times, and systemd is one of those 9.
> The people who prefer initv are the people who enjoy digging in other people’s (untestable) Bash scripts
This is demonstrably untrue. I strongly prefer SysV (which is not to say I think SysV is wonderful or anything) over systemD for a number of architectural and functional reasons.
But I do not enjoy digging into other people's init scripts. Fortunately, I rarely have to do that. I think the last time I had a problem with someone's init script was over 7 years ago.
You're right. Bad generalization. Sentence removed, but archived in your comment.
Out of curiosity, what distros do you use that you've had such a good run with init scripts?
I currently use Debian on everything except my R-PIs, where I use Raspbian.
New installs of Debian use systemd by default. Systems which were upgraded from older releases may be grandfathered in to SysV init.
Systems which are freshly installed and want to change to sysvinit can do "sudo apt install sysvinit-core".
Or they can try out runit, openrc or nosh, if they feel adventurous. The last one is not yet in Debian proper.
Yes, I'm very, very aware of this.
> When there was a problem with a sysvinit procedure, one could simply fix it. As a result, mostly nobody ever had a problem with it.
To the extent that this isn't flat-out wrong, it's only because SysV init had so few features and so the actual patch had to go to one of many different upstreams. Having had to fix the same classes of problem in hundreds of different places like that has given me an appreciation for how long it's been since I've needed to do anything with systemd other than use it.
One important thing to remember is that the ability to have a standard way to start a process, as a non-privileged user, keep it running, with logging, etc. is something which has been standardized in the Windows world since the early 90s. It's really nice not to keep having to spend time working around the lack of a check-box competition feature for a quarter of a century.
> it's only because SysV init had so few features and so the actual patch had to go to one of many different upstreams
Yes.
Yet systemd is unfixable. The fact that it does too much was the most popular complaint at the time it got pushed into every distro... while I disagree, my main concern is that it's unmaintainable.
It's not the init replacement part that bugs me. It's the expansion into audio, kernel logging, resolvers, containers, and so forth that is the issue.
It's a monolith of sorts that breaks the loosely coupled toolbox concept of Unix.
Systemd is a bunch of loosely coupled tools, though; it's not a monolith at all. PulseAudio was written by Lennart but isn't part of systemd. You don't have to use systemd's resolver. systemd-nspawn is a completely separate process and isn't required for systemd as a whole.
It's worth watching Benno Rice (a BSD developer, actually) discuss systemd and some of the misconceptions, some of which you're expressing, therein: https://www.youtube.com/watch?v=o_AIw9bGogo
There are lots of variations on the UNIX philosophy [1], but two core points are to:
1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
systemd has binary format data all over the place, and it is quite difficult to separate out an individual component and have it work properly on its own. Not to mention trying to replace a subcomponent with something else.
systemd is, in general, a tightly coupled system with lots of interdependencies. It is, in many respects, the antithesis of the UNIX design philosophy.
Hell, even modern Software Engineering practices still talk about decreasing coupling and increasing cohesion.
[1] https://en.wikipedia.org/wiki/Unix_philosophy
Regarding the UNIX philosophy thing, it's worth noting that it is not automatically The Right Thing to stick by it at any cost. You could argue that UNIX itself does not use the philosophy when it's inconvenient, for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)
Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.
The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.
> for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)
Sure. The UNIX Philosophy applies more to userland, I think, than the kernel.
Edit: I'll add with regard to the kernel, the internal structure of the kernel still has fairly well-defined subsystems that interact in clear, mostly consistent ways. So while from an execution standpoint it is monolithic, the philosophy is still something that is taken to heart to some extent.
> Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.
I think dismissing it as a semantics game is taking it pretty far. It's clearly not a black and white rule, but more a guiding philosophy. When it works, it's beautiful and elegant.
The ability to pipe basic UNIX utilities and programs together is something that has always left me a little awestruck. The flexibility and simplicity there is almost profound. SysVinit was far from perfect, but you could at least trace the execution flow from basically start to finish and, if you understood basic *NIX utilities, have a good idea of exactly what was happening.
> The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.
Again, it's a philosophy, not a universal truth. A guiding light, if you will.
With systemd, that light is pretty dim, if it's on at all.
> Journald is also not part of systemd core--at least, I think so; this one I've never really looked at myself.
You really should. journald is not optional.
Yup, seems you're right - journald appears to be the only mandatory non-PID1 process for systemd. Good to know. It also will forward happily to rsyslogd if that's your bag and then you never have to look at it, so I'm gonna just give a big ol' emoji shrug on this one.
> journald appears to be the only mandatory non-PID1 process for systemd.
You need to read a lot more. The Desktop Bus is mandatory too. Lennart Poettering reminded everyone of this back in 2015.
D-Bus, as discussed elsewhere in this thread (where I learned about this, too), is a specification rather than a concrete implementation. Are you asserting that systemd does not work with alternative D-Bus implmentations? That would be weird, seeing as how they wrote an alternative one...
Also, since I've been "reading more", and I can check after I get back from lunch--at least at some point Debian shipped systemd without D-Bus on minimal installs. Maybe that's no longer the case, but the stuff I'm reading seems to be post-2015.
> Are you asserting
You need to stop constructing straw men, too. I asserted what I actually wrote. And Lennart Poettering asserted that
> D-Bus has been a core component of Linux systems since more than 10 years.
and
> D-Bus refers to both a specification, and a reference implementation.
back in 2015. I suggest that instead of arguing fallaciously about something that you tell us you have only just learned about, that you go and do that reading.
Please don't be personally abrasive in arguments here. No matter how right you or feel you are, it breaks the site guidelines: https://news.ycombinator.com/newsguidelines.html.
Asking for clarification isn't a strawman, sorry. And it sure seemed like you were correcting me, which is why I asked. But I said that journald was the only mandatory component of systemd outside of PID1. You said this was wrong. I'm waiting for you to extrapolate? Does systemd only work with the "canonical" dbus-broker? Or does it want ("want" as I'm reading stuff that suggests it isn't absolutely required, but I'm not sure and I'm happy to caveat where I'm not sure) a D-Bus broker? Because if it's the latter, and as it's specified and they seem big on meeting specs I'm betting that it's the case, then what I said is correct and you're just being kind of a jerk about it.
Or you can keep trying to big-time me instead of engage on the merits of your position. That's cool people stuff, I guess.
This is out of the line. You were given civil arguments for why systemd requires Dbus but in your response, you accuse the other person of being a jerk and "big-timeing". Please use the strongest interpretation of other people's claims on the topic, before using them as a means to accuse them of wrongdoing.
I am inviting the strengthening of his position in order to better understand it. I asked him to elaborate because as near as I can tell there is no hard dependency on any D-Bus broker. There is possibly--I genuinely can't tell from the documentation I've been browsing through--a dependency on a D-Bus broker. Requiring a D-Bus broker does not make any one D-Bus broker mandatory...and so, if that is the case, then my assertion that journald is the only non-PID1 part of systemd that is required would hold.
I'm happy to be corrected if I'm wrong. I invited it, even; I'm wrong all the time and being wrong is how one learns. The big-timing I referred to is the refusal to materially engage. Instead he quotes Lennart about D-Bus--out of context? who knows! It isn't an indication of the state of the software in question, just his opinion at a point in time--and to tell me to go read (read what, exactly? what generosity in citation!). Oddly enough, though, I don't see you tagging in to lecture him about that sort of sneering. Why's that?
That person is big-timing. Their tone ("you should read more" and so on) is in violation of the HN guidelines, filled with needless snark and condescension. The offending comment has been flagged by multiple users who agree that the comment doesn't meet community standards.
Perhaps the tone of JdeBP wasn't great, but they had a point. On the other hand, eropple's comment tone wasn't ideal either and they consistently miss the original point of systemd requiring dbus. Unfortunately, I am unable to check this since JdeBP post's have been censored.
"Systemd is a bunch of loosely coupled tools"
As just one example, the nspawn owning package requires a libnss package which pulls in systemd-resolve. I believe it also requires dbus.
It's good that systemd apologists have moved on from arguing that it isn't a single binary. But it is not a bunch of loosely coupled tools any more than it is "just" an init replacement.
Hmm. "Apologists." Couldn't be that it's been a lot better for me where I regularly interact with it and has been no worse where I don't. It's gotta be apologism. We have to be out to getcha.
I think you owe me an apology for that one, dude.
More productively: what parts of systemd's ecosystem--aside, apparently, from journald, but the above no-harm-no-foul reply to the sibling still applies as far as I'm concerned--have you had trouble pulling out when you wanted to, out of curiosity?
> It's gotta be apologism.
Apologists are engaging in apologia, which means a defense of one's ideas. It's not suggesting a hidden agenda, to the contrary, it suggests the agenda is fully on display since it's plain what is being defended.
My gripe is bad UX. The whole system is needlessly obtuse and esoteric. Git is another popular thing with this problem, but in the case of git it provides so much value it's enough to get me to overcome its obtuseness. For systemd it's not "enough better" to get me to forgive it for being designed with no thought whatsoever to simplicity or ease of use.
Another thing that makes me forgive git more is that its underlying design is rather novel and powerful. There is something to "get" in git (hah!) that justifies the cognitive investment. In the case of systemd the problem of managing services and their dependencies is not novel or really all that interesting. The complexity of the systemd UX is significantly greater than the novelty or sophistication of what it's doing. It's a piece of software built to complexify the simple.
I strongly agree that git has a UX issue, and the widespread adoption of it over mercurial is a perfect example of inertia leading us to get stuck in a local minima.
I haven't tried hg but from what I've seen one major advantage of git that gets under-acknowledged is its speed. Git is fast and fast really does matter for a version control system you use constantly. Is hg as fast as git?
> Is hg as fast as git?
No. You still have to launch the Python process per command, which is "extremely" slow (relative to C programs). I air-quote extremely because the difference for a single command is small relative to human time, but it matters for automation that invokes many commands.
This post is from 2018, and maybe Hg's frontend is written in C or Rust now (I don't follow its development). But if it is still a Python-driven runtime, the same basic criticisms apply:
https://mail.python.org/pipermail/python-dev/2018-May/153296...
Personally, I like pushing with hg, but I prefer pulling with git. An HgHub service at the scale of GitHub would probably result in massive end-user UX issues but maybe happier developers and repo administrators.
This reminds how pleasant it was to use hg as a git front-end (via an extention[0]) - compared to using the git cli. I think I'll have to revisit and see if that's still the case, and if I can make it work with feature branches etc.
Also looks like there's finally python3 support in the recent mercurial 5.0 release.
[0] https://hg-git.github.io/
Git isn't really all that fast for what it is. The "git is fast" thing came about as a comparison to non-distributed version control systems, where changing what branch you were working on could involve coordinating with some (often overburdened) server on the other side of the planet.
That's not to say git is slow of course. It's just.. basically what you'd expect given it's architecture.
I like git's UX. Anything outside the typical flow is like walking into a garage filled with old stuff you thought you might need again some day, sifting through boxes of gadgets looking for that one thing you actually do need this time. Productive? Maybe not, but the experience is a much broader subject.
You probably mean local maxima.
I was referring to an etherial notion of "loss", which is why I said minima.
> I can't say I miss sysvinit.
Distros that choose not to employ systemd do not necessarily retain sysvinit.
They probably do though, openrc defaults to sysvinit as its PID1.
I guess it's part of the confusion too, most of the systemd work was replacing the "rc bits" (process startup and management) not replacing PID1 itself.
I mean, before systemd there existed other initds. Upstart was Ubuntu’s for a time.
The creator of Upstart, Scott, explicitly said that he thought that systemd was better and that Debian and Ubuntu adopting it was the right decision: https://www.reddit.com/r/LinuxActionShow/comments/1xzfv0/ups...
I don’t really have a strong opinion anymore. Scott makes the right move here for professionalism and modesty, and Linux is more united on this than it isn’t.
Still, the spats with the Linux kernel team and the rest of the community has earned systemd a reputation I can only say it deserves, even if the initial ire drawn towards it was over the top. Systemd is not a project I have strong confidence in to act in the user’s best interest, based on what I’ve seen with handling security issues.
And much like systemd or openrc, replacing PID1 (init) itself was not the point of upstart.
It was redhat's also, albeit, for a very short time.
I get that systemd works for you, and I'm not telling you (or others) that you shouldn't use it. The problem is the systemd monoculture. Wayland, Gnome etc are non-starters because they have hard dependencies on systemd, which is ridiculous.
> For example, people are complaining about logind being included, but forget that consolekit was unmaintained before that.
I use neither logind nor consolekit. These are solutions for problems I don't have. Moreover, they often get in the way: when I ssh into my media PC, I don't want logind/consolekit to tell me I'm not allowed to access the sound or video cards. The only function that machine performs is to do the thing that logind/consolekit are designed to prevent. And it's a required component of systemd.
I'm glad it's easier for you to manage services with systemd. But it's easier for me to manage services with OpenRC, and there's nothing wrong with that. As far as handling modern hardware- huh? I guess it's not clear to me what interactions an init manager is supposed to have with hardware.
Which really speaks to my primary issue with systemd: it's a freaking monster. It does nine million things, but I only need it to do like three things: start services when I tell it to start services, stop them when I tell it to stop them, and restart them if they crash. (if it's configured to do so) That's it. Why do I need or want it to interact with my hardware? Why does it need an integrated web server in order to start apache? Why does it a logging daemon in order to start syslog-ng? Why do I have to use its built in log reader instead of tail, less, grep etc? Where is the boundary between systemd and the rest of my system?
> The problem is the systemd monoculture.
And sysvinit wasn't pretty much a monocultur?. I mean even OpenRC is basically sysvinit with a lipstick.
> These are solutions for problems I don't have.
Good for you. But event handling is something that many systems today do need and I think we should try to optimize for the common use case.
As for you not needing logind/consolekit, sure, if security is not a concern on a system, they're not needed, but again, that's not true for most systems.
GNOME etc. also don't really have a hard dependency on systemd, they depend on logind, which does not necessarily require systemd and indeed GNOME ships on Gentoo, Void etc.
It's also not like the GNOME devs are stupid and got tricked into depending on logind or something. Mo, it solves real problems for them, so they depend on it. Why is this hard to grasp?
systemd-logind does require systemd. That was one of the root causes of the Debian Hoo-Hah.
* http://jdebp.uk./FGA/debian-systemd-packaging-hoo-hah.html
To start off with: Maybe I'm not on the same page, but your tone feels quite aggressive to me right now. I'm not telling you that you're wrong for choosing to use the software that you use. This isn't a flamewar, I'm not trying to tell you to do different things. I'm just explaining why the OP's criteria for an init manager don't apply to me.
It's also 3am my time, and I've had quite a bit to drink. If anything in my post comes off as aggressive, please let me know and I'll edit it in the morning.
> And sysvinit wasn't pretty much a monocultur?. I mean even OpenRC is basically sysvinit with a lipstick.
No. OpenRC is very happy sitting on top of busybox init instead of sysvinit.[0] When I encountered problems with my gentoo system built on musl (instead of glibc, #1 source of problems) and busybox (instead of coreutils #2 source of problems, and sysvinit #3 source of problems) they got fixed. A major part of why OpenRC is what it is is to prevent reinventing sysvinit/busybox's wheel.
> event handling is something that many systems today do need and I think we should try to optimize for the common use case.
> As for you not needing logind/consolekit, sure, if security is not a concern on a system, they're not needed, but again, that's not true for most systems.
Security is a concern on all of my systems, however, I do not agree that logind/consolekit add any security (at all) to most systems, or that they add substantial security to a small minority (say 5%) of systems. I would wager that 95+% of all linux systems are one of the following: A) no one is permitted access to any hardware, (video/sound/input) (headless systems, ie server farms, multiuser systems in a university, etc) B) single user account systems, (ie, my laptop. If someone has write access to my .bashrc file, they might as well have root access, and consolekit/logind can't bandaid anything by limiting access to my video device hardware) or C) hpc compute farms. (where the assumption is that all processes have god given write access to the GPUs) In the first class of systems, consolekit/elogind do not add security because all accesses are blocked, (in fact, the existence of consolekit/elogind simply expands the attack surface, making it less secure) in the second class no security is added because the video card is a more difficult attack vector than editing all the common home scripts, in the third class no security is added because whitelist-asterisk-to-video-card is the only sane configuration policy.
logind/consolekit are super awesome for eg educational institutions which have computers with multiple seats. Have four schoolchildren sitting at four monitors and four keyboards attached to one desktop computer? Sweet. (actually most schools use chromebooks now, which last I checked uses OpenRC) It's great that projects like consolekit/elogind exist to support those groups. But pretending like that's a common usecase, or pretending that sysvinint can't support those usecases- stop.
> GNOME etc. also don't really have a hard dependency on systemd, they depend on logind, which does not necessarily require systemd and indeed GNOME ships on Gentoo, Void etc.
This statement is, at best, disingenuous. If you want to install Gnome on Gentoo, you need to do one of two things: install systemd (Gentoo supports systemd as a first.point.five class citizen (my terminology))[1] or install elogind. elogind is a Gentoo specific forked version of systemd with everything that is not logind-related removed.[2] This code is still all upstream systemd code, and there is not a meaningful path towards an independent fork, the way busybox init exists independently of sysvinit, or syslog-ng exists independent of syslogd, or xorg-x11 exists independently of XFree11.
The fact that non-systemd installations of Gnome can possibly exist on Gentoo is a result of Gentoo developers going above and beyond the call of duty, despite barriers placed by the systemd team. And non-linux posix systems, eg FreeBSD - forget it.
> It's also not like the GNOME devs are stupid and got tricked into depending on logind or something.
I have nothing but respect for the Gnome team. I never implied anything of the sort. I am disgusted by your insinuation. Please walk this statement back.
I think that many of the things that the systemd monolith has to offer are valuable. If systemd were a federated constellation of small daemons speaking a reasonably stable API that didn't all have hard dependencies on each other, I would probably be quite happy with it. The problem - again - with systemd is the unforgiving, colossal monolith. I don't have the privilege of dipping my toes into the good parts without having to swallow whole the bad parts. And the bad parts are disgusting. (this is the same problem I have with Microsoft Windows) Modularity is a good thing; systemd is the antithesis of that.
[0] https://wiki.gentoo.org/wiki/OpenRC#Replacing_init
[1] https://wiki.gentoo.org/wiki/Systemd
[2] https://wiki.gentoo.org/wiki/Elogind
Agreed, I very much like systemd. The documentation could be a bit better (though I haven't looked at it much in the last year), but overall it's a good improvement.
I first started using systemd with Arch before it got pulled into Ubuntu. It definitely took a little bit to get used to, but once you understand how to write services instead of cron jobs, it's way more betterer.
> looking at sysvinit
That's the thing: there are lots of alternatives besides sysvinit specifically (OpenRC, Upstart, BSD-style RC, launchd, SMF, runit, etc.) on varying scales from "simpler than even sysvinit" to "more feature-rich than systemd". And yet, the best the systemd advocates could ever muster up is/was criticizing specific flaws with just sysvinit.
Of course sysvinit sucks, but you'll notice that (Devuan being the notable exception) most of the article's listed distros use something that's very much not sysvinit.
I am not in any camp.
I do think that systemd is very useful but have concerns that they simply don't have the man power to maintain that broad feature set and scope they have set themselves.
What previously would be done in multiple different competing projects by different people with different ideas in different companies, is now done in a single code-base. I can imagine that innovation will be hindered that way, because systemd is not Linux. But maybe some day it will be, IDK. Currently it is just to big to scale that well to small systems though.
That's a valid concern, but it's worth noting that not every component systemd replaced, (ie consolekit), was actually being maintained, so arguably we're in a better situation now from that standpoint.
Isn't it now much more difficult to figure out if certain components are or are not maintained?
ConsoleKit was used in many commercial linux distributions and still wasn't maintained, now that its part of systemd and systemd is maintained, does that mean that every component of it is as well?
Not to mention systemd being used in all current enterprise Linux distros with people being paid for maintaining it in missing critical usecases.
sysvinit is easier to understand for me. It's just a bunch of shell scripts after all. So from the point of simplicity sysvinit wins for me and I never had much problems with it (while I had some problems with systemd). On the other hand, I recently had to write unit file for tomcat and it was extremely easy, systemd had all necessary options. Writing shell script was not that hard, of course, but it was some work.
One reason even big ol sysvinit is superior to systemd is that shell scripts are transitive knowledge. You can do almost everything with shell scripts. Learning some obscure (albeit very simple) systemd unit file format won't help you anywhere else besides systemd.
> One reason even big ol sysvinit is superior to systemd is that shell scripts are transitive knowledge.
In theory. In practice systemd's actually much more transitive since many distros have adopted it and systemd services tend to work cross-distro, so once you learn systemd you can somewhat confidently manage Fedora, RHEL, Ubuntu, Arch, SuSE, Nix etc.
sysvinit shell scripts often differed in subtle ways between distros and the particular distro implementation was rarely transportable without quite a bit of work.
> You can do almost everything with shell scripts.
This cuts both ways. I'd argue the problem with sysvinit was precisely the fact that you can do almost everything with shell scripts, since now you're executing a script as root that can do almost anything, depends heavily on the bash abilities of the author to not introduce subtle bugs, or break easily, (which varied heavily from script to script and distro to distro), and to not be a security nightmare.
RHEL has Systemd since 2014, Ubuntu and Debian as well. Even the purists at Arch have Systemd as default since 2012.
Sure, it ain't perfect. But maybe if all the professional kernel developers and distro managers chose Systemd, and yet some angry people online keep basing their opinions on random amateur blogs catering to a certain narative, regardless of expert opinions, then maybe we can conclude that...
... anti-systemd people are the anti-vaxxers of the Linux world.
I don’t think anyone thinks that sysvinit is better than systemd, but RC definitely is.
Actually I do. systemd is too complicated for my needs, maybe it is usefull in some complicated system setups but for me (and probably others) it is too big, too complex and completely unneeded.
What was the problem with sysvinit that they tried to fix? I don't see any.
systemd was aimed at the problems with Upstart rather than the problems with van Smoorenburg rc. van Smooreburg rc had already been superseded for its creators. Fedora was using Upstart.
The problems with van Smoorenburg rc are widely acknowledged, however, and motivated the creation of Upstart, finit, init-ng, and many other softwares; indeed, the problems of System 3/5 init/rc, which van Smoorenburg rc began as a clone of, similarly motivated the creation of the SAF in AT&T Unix System 5 release 4, the Solaris SMF, and the AIX SRC. There is a history of superseding them that goes back for decades. It is clearly wrong to claim that they are without problem. They've been recognized as problematic and people have been replacing them since before Linux was invented.
* http://jdebp.uk./FGA/system-5-rc-problems.html
* http://jdebp.uk./FGA/unix-service-access-facility.html
* https://blog.darknedgy.net/technology/2015/09/05/0/
You would be surprised.
It's a stupid position to take - watch luke smith's video on it for a neutral point of view.
https://www.youtube.com/watch?v=_ljfOCiP0XM
I read everything I could find on systemd to form my own opinion. I used FreeBSD for a decade and have many years of Ubuntu/upstart experience after that.
Based on my experience and all the anti-systemd posts online, I expected to dislike it.
But on it's merits I really like the consistency it brings to the "system layer" of the OS.
I find it's shame when developers skip over systemd for consideration, adding extra layers of complexity when systemd alone would have been sufficient: For example, using Docker when the basic isolation features of systemd would do, or using "forever.js" to manage Node.js processes, when systemd could could be saving 15% memory overhead per process, using software already installed in the base system.
On the other hand, I do support diversity of software and am glad to see some other options still being promoted and used. The dangers and weaknesses of monoculture in the physical world apply to software systems as well.
One of my biggest issues with systemd is that I don't like the idea of not being able to replace pieces of it. If I don't like their logging, dns, or ntp daemons, I can't easily replace them without feeling like a second-class citizen and/or having the overhead you've mentioned.
From my understanding, alternatives _could_ be written, but haven't yet and that means we currently having to ignore more battle-hardened software in favour of whatever is in systemd by default.
The ntp and dns are easy to replace, they're not part of the actual systemd binary. Fedora from what I remember, doesn't even enable systemds ntp service on Workstation.
Logging is a bit different, but you can tell journald to use syslog.
> you can tell journald to use syslog
No. One can tell syslogd to read from the journal. There is no mode of operation for journald that does not have the journal, on disc or in a memory filesystem. And journald itself is not optional. The only optional part is choosing to have an extra old-style syslog daemon that is using the journal as input.
* https://unix.stackexchange.com/a/294206/5132
* https://unix.stackexchange.com/a/332315/5132
> The ntp and dns are easy to replace, they're not part of the actual systemd binary.
They really aren't. It took us a full week of work to get dnsmasq to work in place of systemd-resolved. The problem is that they are tied in to the dhcp client and it only seems to know how to update systemd-resolved's configuration. So getting our custom search path out of dhcp was a huge pain.
I like almost every other part of systemd, but trying to excise systemd-resolved from Ubuntu 18.04 was massively frustrating to the point where I started thinking, "maybe the systemd haters are right…"
I don't mean in terms of the binary, but in terms of the whole system.
You would have to write quite a bit of code...
The success of systemd is in that it solves problems for distros that none other was solving (or even looking other way). If you want to successfully replace it, you have to solve the same problems, and do it better, so it would be worth replacing systemd.
And because none other was doing it, you would have to write solution that does that. Slapping some bash scripts together won't do it.
It's a very strange argument that I have seen previously that someone could write an alternative to systemd but haven't ergo systemd must be the best alternative.
This is wholly and totally imaginary. Every part of system has an alternative because it is all solving existing problems that people have been solving for decades with a variety of tools.
The only actual challenge is replacing just part of systemd while retailing compatibility with the rest of it.
> This is wholly and totally imaginary. Every part of system has an alternative because it is all solving existing problems that people have been solving for decades with a variety of tools.
Alternative doesn't mean 100% substitute both ways. The pre-systemd didn't go in their abstraction in the way, that systemd does - it is basically system-level, event handling machine. The events could be not only boot, but also hotplug, timer, socket connection or others, and their handling is configured by the system or admin in unified way, including unified way of adjusting system-supplied units. Other inits just ran scripts at some order the boot and that's it, ignoring, that something doesn't have to happen at boot, but at some other time.
That's one of the things, that systemd handles way better, than any suggested alternative. By ignoring where systemd is strong, you won't make the alternatives any better.
> The only actual challenge is replacing just part of systemd while retailing compatibility with the rest of it.
No. I don't see why we should have the Sisyphos task of maintaing sysv/cron/etc idiosyncrasies forever. Sometimes, it is better to get rid of the collected warts, and not limit the system to existing semantics.
> The pre-systemd didn't go in their abstraction
On the contrary, that's exactly what systemd's predecessor, Upstart, did. systemd, in contrast, did not. Basing the system around the concept of events triggering things turned out to cause problems, as discussed by Lennart Poettering and by Debian Technical Committee members during the Debian Hoo-Hah as their reason for not choosing Upstart.
The problem wasn't that Upstart was event-based, but HOW the event system was designed.
See http://0pointer.de/blog/projects/systemd.html, section "On Upstart". It names two big issues with HOW.
I didn't mean that everything was in a single binary, but that everything is tightly coupled. Just because a binary is replaceable, doesn't mean the replacements that have existed for decades will be "drop in" or integrate as well, which leads to feeling like a second class citizen or the overhead originally mentioned.
I didn't mean single binary either.
I meant, that systemd solves some category of problems, that other tools didn't solve, didn't intend to solve and didn't even care for what the problems are. For distribution maintainers, systemd was something to welcome, that did address their problems.
So yes, there is no replacement (the requirement for it being drop-in only places the bar higher), because there's nothing else that scratches the same itch.
> For example, using Docker when the basic isolation features of systemd would do
This is an interesting claim, mind throwing some pointers?
https://www.freedesktop.org/software/systemd/man/systemd.exe...
and some other directives documented on the same page.
I had completely forgotten about these directives! It's things like this that make me glad systemd is a choice, honestly.
SystemD did not invent cgroups. Not only is it possible to use cgroups without systemd it is often easier for small cases and binaries can call cgroups themselves for any child processes they create.
The timing of these systems releasing does not equate to causation here. Google themselves (creators of cgroups) do not even use systemd on their own hypervisors which leverage cgroups heavily.
Nobody said they did. You’re putting words in my mouth. The ease of use with 2-3 directives in a way that just works is what they brought to the table.
boramalper 8 hours ago | parent | flag | favorite | on: Linux distros without systemd
> For example, using Docker when the basic isolation features of systemd would do
> On the other hand, I do support diversity of software and am glad to see some other options still being promoted and used.
Then you should not be wanting to see applications including a hard dependency to systemD.
Why is it another open source developer's obligation to satisfy your design desires? Are you paying them? If not, put your time where you aren't putting your money, become the purveyor of diversity, and submit a patch. Fork, if necessary and it matters to you that much--it's your right to do so.
Full disclosure: I'm a systemd skeptic.
The problem is basically that Linux used to be about choice (choose your distro! choose your DE! choose your shell! choose your browser!) but now that it's become big business, large corporations like Red Hat (developer of systemd) have transformed the evolution of Linux from a bottom-up consensus system to a top-down "we develop all the stuff you use and now it all uses our stuff so now you do too" approach. It's _very_ hard for me to build a system without systemd; I can use a niche Linux distro, I can use a BSD, or I can hack another init system (etc.) in myself. None of those are satisfying, I think for obvious reasons.
systemd is not well written, and it is not well designed. Its style and architecture are inconsistent. Parsing is mixed with validation and business logic. The types of CVEs its generated show that there was no coherent thought put into input validation, fuzzing, taint analysis, and so on. The effects of those flaws (your whole system crashes, etc.) show that failure strategies were never considered. And if all that sounds like overkill to you, maybe it is! But I can tell you that these techniques and design strategies are employed by experts to build software that must be robust, like software that controls medical equipment or transportation control systems. We know how to build software that works and that mitigates failures. Looking at systemd's code, it's clear they used none of that knowledge. And that, combined with its sprawling control over my system, makes me not want it on my machines.
I am not a systemd skeptic, but I'm also not somebody who cares that much. (Which might seem at odds with this thread, but I'm in a lot of meetings today. ;) I use it and it gets out of my way. For me, it's fine.
I'm not qualified to judge the quality of the code in systemd directly; I've browsed it a little but I am a middling C++ programmer and while it hasn't stood out to me as smelly I would defer to more knowledgeable heads. I get the drive for very rugged underpinnings, and I'm on board with that; in things that aren't C++ I frequently am the one with the scalpels making criticisms similar to yours, so that totally resonates with me. I am not, however, convinced that any of the alternatives that currently exist are materially better, so I kinda shrug a bit? Further, it seems to me that the generally modular nature of systemd (regardless of how coupled one believes those separate modules to be) suggests that a determined programmer could probably build a drop-in replacement--Rust? Where's my Rust Evangelization Strike Force?--that plays nicely with it and grows to replace poor and soft design.
The social aspect you bring up is, to me, the most compelling argument against systemd (because monocultures aren't great in general, etc.--I think most people agree on that). Thing is, though, I think that that bottom-up consensus model was doomed from the start. I don't think it was ever going to be tenable as soon as there's any money in it. Red Hat wants consistency, their stuff is going to have consistency, you can use it or go off-roading.
I don't think there's a great solution aside from "build your own ecosystem and don't let it get capitalism.txt'd to death." Which is a deeply unsatisfying and impractical answer. And to be clear, people who feel strongly about it should do that. I'm not being snarky when I say that. I'm a pragmatist--if it's better, I'll use it, too, and submit patches if needed and when I'm capable. ;)
> I don't think there's a great solution aside from "build your own ecosystem and don't let it get capitalism.txt'd to death."
I do, it's "if you run a big mainstream distro like Fedora or Debian, let your users swap out init systems". I know there have been debates on this, and I know the general consensus is "it's not worth the trouble / it's not as simple as just swapping out something like systemd". My counter arguments are: there are distros that are doing it, so it doesn't seem like it's that bad after all, and when dealing with something as important as PID 1 (or your libc, or your filesystem) we should be conservative and let users swap things back when making a huge change.
To that I'd just go "sure!", but I'm just a guy. The first thing that jumps out to me, though, is the question of who pays for the doing of it and the testing of it? From what I know (and I've been learning a lot today, it's been productive!), it seems like you're likely to undertake a Devuan level of effort to do it with Debian, and your patchset is going to be so broad as to effectively fork the core anyway. Is there value, then, in tagging along with that major distro? What do you get, then, if you're the second-class citizen? I'm reminded of stuff like Kubuntu, which (and I say this as a long-time KDE fan) has historically limped along with only moderate love--and that's a DE, it's a lot easier to handle DE differences than core init differences, service management differences, etc. (We have the XDG standards and the like, after all. Though I guess, in a certain light, systemd's trying to be XDG-for-the-core, which attacks the question we're looking at directly, yeah?)
The other thing that jumps out at me is just that this hypothetical is mostly kicking the can down the road. If Fedora suddenly decided to support multiple init systems I think it's hard to envision a future where the happy path wouldn't remain systemd; it's what the core folks think is best, otherwise they wouldn't use it now, and even with additional support the lion's share of attention would go to that happy path. To that end I think that any sort of init choice in those distros just becomes "so we'll support it for now, but the set of third party stuff that supports it is going to continue to shrink, so I hope you have an off-ramp in the future."
Between these two issues it seems like separate distros that have a firm commitment to not-systemd end up being necessary anyway and an ecosystem of not-systemd stuff will need to compete for value on those merits. What am I missing?
> Why is it another open source developer's obligation to satisfy your design desires?
I never said it is.
What I said is that if someone values diversity of software, then they should object to applications including a hard dependency on systemD. That's just logical, as including hard dependencies on systemD reduces software diversity by requiring that the system it runs on uses systemD rather than allowing them the choice.
We did fork debian, and here are a TON of people who say they do not care, but still insist that we are doing it wrong for some reason.
Interesting. So what use cases is Docker good for? Why is it so popular if systemd suffices?
Docker is about deployment. The isolation model is built on the same tooling that systemd uses, but the usage model is all about how to come up with an easily isolated runtime environment that can be published and extended in a big development community.
Basically systemd has nothing like a dockerfile, that's not what it's for.
I think there's a good argument though that Docker should be retargetting its encapsulation work to a lower level OS primitive where possible (which would be systemd on linux) instead of rolling its own.
See podman[1]
[1] https://podman.io/
Docker provides some more approachable tooling on top and (IMO, at least) its networking is more straightforward for "developer-y" use cases.
I quite like systemd-nspawn too, though.
Cross platform stuff? Docker allows you to do many things on Windows and Linux.
I’m not arguing that this is worth the overhead, or even a useful feature in practice (most companies and products tend to pick one environment and stick to it), but one that comes to mind that systemd would likely never have.
The cross-platform stuff is interesting... however, in practice almost everything outside Linux containers is only weakly supported. You have to jump through some hoops to use alpha/nightly builds of Docker hand installed in Windows 2019 or later Win10 builds in order to support both Linux and Windows containers on a single system. I think the work with WSL2 may make things better better as a result.
The naming in Windows and Mac, along with the limitations of the Desktop implementation are also frustrating. Don't get me wrong, I really like Docker as an orchestration base, I just don't think the cross-platform support is a value add in practice and in fact detracts as a whole imho.
I'm not sure systemd had every container feature available when docker was released. If I'm not mistaken, systemd-nspawn (systemd alternative to docker) is much more recent.
One of the better videos on this subject:
https://www.youtube.com/watch?v=6AeWu1fZ7bY
BSDCan 2018—Benno Rice: The Tragedy of systemd
"systemd is, to put it mildly, controversial. Depending on who you ask it's either a complete violation of the UNIX philosophy, a bloated pile of bugs, a complete violation of the elegant simplicity it replaced or, it most cases, some or all of the above. So why have so many Linux distributions taken to it? Is it as bad as people say?
I watched this a few weeks ago as well, and it is a very good, even-handed overview of systemd. He talks about the history and other tools that came before, like Apple's `launchd` and Ubuntu's `upstart`. He also talks about the actual problems with it, rather than the FUD hysteria (and also explains why some of them are FUD).
He's from the BSD world, so his POV is as an (mostly) outsider.
You might be interested in these:
* https://blog.darknedgy.net/technology/2015/10/11/0/
* https://blog.darknedgy.net/technology/2015/09/05/0/
* http://uselessd.darknedgy.net/ProSystemdAntiSystemd/
I love this video and have to share it every time I see this debate. He really show how this argument is kind of weak and this is coming from a BSD guy!
This is a great talk, and helped change my stance on systemd from bad to neutral.
I watched the video a few moths ago. I disagree with almost everything this guy said.
I'm interested in which points you disagree, because I, in the opposite, agree with almost everything the guy said.
> there is a tragedy
Systemd was criticizes right from the start for the same things that it is criticized now. No misunderstanding. No bad timing. No tragedy present.
> People don't like change
Wrong! People love change if it is an improvement. People don't like change if things get worse.
> Services, System configuration, daemons, etc are separate distinct things and need distinct C code to be managed
Nope. They are essentially the same thing and should be dealt with by small scripts.
> It does not really violate the Unix philosophy
Nobody really knows what "Unix philosophy" actually is. Therefore it does not matter if it gets "violated". However systemd definitely violates my and many others taste for good software. It is bad taste.
> It's not bloated and monolithic because it's not one binary
The stupidest argument ever. If you have a lot of tightly interdependent binaries that don't make any sense to exist outside of systemd you have a monolithic system were simple nuclear parts are not replaceable with alternatives. Besides from that: Its f*in bloated as hell.
> It's not as buggy as people claim
It is. Besides, the real problem is not the number of bugs, the prolem is getting rid of bugs. Recompiling C code just to get your init working is really a world of nightmares.
> People don't like systemd because they don't like Lennard Poettering
Most only care about technology and systemd is bad technology. Nobody cares about Poettering.
> UNIX is dead
Hearing that coming from a FreeBSD developer? Do his BSD buddy's agree? Why doesn't he switch to Windows? Sounds like conman to me. Remind me to never install FreeBSD. A big part of what made Linux great are the things that were borrowed from Unix and Plan9.
To make it short: The dude loves change and he wants FreeBSD to go in the direction of what Windows is today. I love change and I want Linux to go into the direction of Plan9.
None of those are arguments, they're just "nuh-uhs".
Basically, people don't like systemd because they don't like change, and groups like shitting on other groups of people?
I'm not sure the presenter is condescending enough. Maybe he should have given an abbreviated history of Unix as well, oh wait, he did that.
Change is good. Let's change away from systemd.
Come up with something demonstrably better then. Nobody is stopping you.
Why are you acting as if alternatives don't already exist?
Define an alternative. Most systemd alternatives do not concern themselves with post-boot event handling much, for example, which is a rather big part of modern systems. Most alternatives do not use a declarative style for unit definitions, which is a safer model being adopted by many other parts of Linux, including whole distros such as Nix. Things like Kubernetes have become popular for similar reasons.
Notepad.exe is also an alternative to Word, if the subset of functionality you're looking at is strictly text input.
"demonstrably better" ... there's a reason beyond entropy why a lot of distros decided to adopt SystemD. A talk linked to in another thread goes over a lot of this. TFA itself shows there are alternatives.
It's like arguing that KDE or Gnome shouldn't exist as options... nobody removed anyone's options. Don't like it, fork it.
Guix is missing, which uses GNU Sheperd, which is in Scheme:
https://www.gnu.org/software/shepherd/
That said, to me systemd has been a large improvement. Maybe it has its problems, but at least there is consistency in handling services and logs between Linux distributions, which is a big improvement over a bunch of homegrown sysvinit derivatives, Upstart, and homegrown non-sysvinit scripts.
Just added as as-good-as-we-could to the website - do you have any experiences in using it?
I've used it just a bit. AFAICT Shepherd is a fairly minimal dependency graph.
It's not comprehensive - in Guix there are services that exist above and below Shepherd and you can't define ordering dependencies between these (ex: a shepherd service can't depend on a mount (above shepherd), or an inetd service (below shepherd) on a database). I'm not sure that's a huge problem in practice.
Shepherd process stdout/stderr are sent to syslog. Shepherd automatically restarts services if they fail but not forever, and the retry behavior isn't configurable ATM.
The last complaint seems feasible to address have you thought of adding an issue?
Guix is a weird little behemoth of a distribution, that allows you to install different versions of different packages at the same time which then builds your effective file system on the fly. That being said, as mad as this sounds, I've heard people are having a lot of fun with it too, so apparently it's also manageable.
Guix is for you if you want to run old software on a discontinued software stack simultaneously with a modern system.
That makes it immune to dependency hell; there are no two packages you can't have at the same time due to dependency version conflicts.
The big wins for me though are: having all my system configuration in a small set of versionable text files (rather than gigabytes of mutable state), and being able to attach dependencies to the scripts that require them rather than to the system they're expected to be run on.
(I use NixOS, but it's the same concept)
Couldn't you have just looked into a little bit instead of saying you know nothing about it? It's a distro with functional package management. https://www.gnu.org/software/guix/
I mainly use FreeBSD, but I have a number of Linux VMs that I use where appropriate.
The big thing, for me, that systemd brings to the table is consistency. Managing services on CentOS is the same as managing them on Debian which is the same as managing them on OpenSUSE. This makes it a lot easier to deal with the various different Linuxes.
Also, it seems to make writing a service a lot easier. Instead of the service needing to daemonize, write out a PID file, or whatever, it can just be a program that does its thing without forking, sends its output to stdout/stderr and systemd takes care of the details. I tried to go through this exercise on a FreeBSD system and ended up just installing 'pm2' to do it for me. I subsequently figured out what I did wrong, but pm2 is working and I see no need to change it.
Obviously systemd isn't suited for every use-case, but it doesn't seem nearly as bad as some people make it out to be.
> Managing services on CentOS is the same as managing them on Debian which is the same as managing them on OpenSUSE.
It's not really the same, some systemd-based distros diverge a lot and anything non-systemd much more so, including FreeBSD. Even something as basic as network configuration could be completely different on systemd distros, like Ubuntu with its netplan weirdness and Archlinux going into the opposite direction and forcing you to choose how to deal with network yourself from the start.
But still, if you want any sanity in your deployment across different distros - you simply have to avoid using systemd or other host's init and service management systems for anything and bring your own service management with you. It could be as simple as a script starting from root's cron at @reboot (this particular trick survived for decades for example).
> But still, if you want any sanity in your deployment across different distros
This might be true for your in-house app but if you want sanity in packaging your service for inclusion in various distros you include a .service file.
By managing services I really mean starting/stopping/checking status of/etc daemons. On a systemd distribution, you make a unit file that describes the daemon and use systemctl to manage it. It seems like sanity for deployment can be accomplished by delivering a template unit file and letting the distro maintainer or sysadmin customize it as required.
Networking is a little out of scope here - I'd guess that changing basic network configuration is something that's less frequent than the service management tasks.
As much as I'd love to get into the rancor and bikeshedding good time as someone who just wants to manage some services on a few hundreds of thousands of instances and not worry about it systemd does that exceedingly okay.
I agree with you on that one. It is certainly an advantage, if the interface to manage aspects of different Linux distributions is the same.
I actually really like systemd's service files, and the functionality it provides. I like the ability to express dependencies clearly, the fine-grained control over server restart behavior, and a ton of other systemd features. I'm also very happy to see it replace cron.
What I _do not_ like are:
I'm very glad to see sysv-init on the way out, personally. I think systemd unit files are a huge improvement, and greatly improve standardization and init quality. But that doesn't intrinsically mean that the systemd project is doing the right thing.Is there an alternative init implementation that is compatible with systemd unit files?
There are certainly automated tools for converting systemd unit files into the service definitions used by other systems.
* http://jdebp.uk./Softwares/nosh/worked-example.html
* http://jdebp.uk./Softwares/nosh/guide/converting-systemd-uni...
* http://jdebp.uk./Softwares/nosh/guide/commands/convert-syste...
It's comical how people get so "jazzed" about something like not liking systemd and make a whole movement about it (with their t-shirts and everything). I get people may not love everything about systemd (or maybe they hate it), but I'm not convinced it's so bad. I've been using it since RHEL 7 and I've really gotten used to all of the benefits of systemd. It also doesn't hurt that I went to the session on pid 1 by Lennart Poettering at Red Hat Summit 2014 and he convinced me of the at least not-horribleness of systemd.
The simple reason is that everybody is forced to deal with it. So you either like the way it works, or you dislike it, but there's no real option to just ignore it in Linux land anymore. For a lot of people, that means that their favorite distro suddenly works in a different way.
This happens every time you significantly change something that people use and have existing workflows for, regardless of whether your change is beneficial on the whole.
You either fall into a group that hates systemd or one that could care less about the problem systemd is solving.
I didn't care much about the problem systemd was solving, I was content to let the init process go through without touching it.
Then one day several years ago I did an apt-get dist-upgrade and my system didn't boot after that. I have personally hit way more issues with unbootable systems or having to drop into single user mode to fix some BS with systemd. That was enough to make me dislike it, or convert into a hater, and switch most of my personal usage away from Linux.
I think it's gotten better since then. I still dread updating Linux boxes, because I know that some percentage of those updates will have a systemd bug introduced resulting in a non-bootable system or one where I have to manually fix it up.
Counter-anecdote, I switched to systemd all the way back in 2012 on Arch, even before it was mandatory and way before it was in a state where other distros felt comfortable switching and I remember to this day that after I did the conversion and hit reboot, an absolute panic hit me, I was 110% sure the system won't boot afterward, just because of how drastic the change was to perform on a live system, (because simpler upgres than that broke my system before). Since Arch is rolling is not like a new release came with systemd, no, you just switched your live system over.
I remember being absolutely floored when after that relatively complex procedure, not only did the system boot, but it booted faster. I think I became a fan that day.
To be fair to your view, I did observe fewer issues in Arch's systemd packaging vs. say, Debian. Arch I think is pretty good overall.
...and when you care quite deeply about the problems being solved, think they're being solved in a pretty good way, and are unsettled by the anti-systemd frothers literally sending Lennart death threats? (Like...this is bad, yo. There are arguments--bad ones, IMO, but arguments--against systemd. But the people who like systemd aren't sending death threats to Devuan people or whatever.)
What group is that?
To be fair, the ones sending death threats are contributing negatively, while the fork effort is a positive contribution to the situation. We'd appreciate not being lumped in with the worst offenders, if you'd be so kind.
A lot of the western world has devolved into tribalism and identitarianism. People of similar views on anything are lumped in en-masse and presumed to hold all of the same values, opinions and support all actions of the collective whole.
It's a shame that a lot of core values are lost in favor of these viewpoints.
Yeah. It is really sad. We're getting hyper-polarized options and information everywhere we look, and it's affected people's ability to argue a specific point if they detect any misalignment of interests. I've been calling it "casually dismissing someone's opinion by lumping it in with an ideology", but looking for something more concise.
I get no end of bemusement from open source advocates that think it makes any kind of rational sense to campaign against an open source project. THIS IS ALL OPEN SOURCE. Freedom and choice is the whole point, isn't it?
If you prefer an alternative or forebear to systemd, that is great. Use it. Advocate for the system you prefer. Convince people that the choice you have made is better.
Meanwhile the vast majority of distributions have voluntarily chosen systemd. Perhaps if its critics reflect on the reality of that shift, they could look at ways to improve the alternative to address its perceived deficiencies—just as GCC started doing now that LLVM has taken a bite out of their dominance.
Notice that nobody complains about the existence of systemd, but about the enormous and shady pressures that distros had (particulary debian) to adopt it as standard. Nowadays, the complain is mostly about the systemd monoculture.
> systemd monoculture
As someone who really wanted the BSDs to get more credit ~20 years back, and spent a fair amount of time on various commercial Unixes as well, I find that phrase rather amusing. Historically poor standardization of the userland aside, GNU/Linux is the poster child for monoculture in Unix, and has left a whole lot of dead or dying software in its wake.
Just to pile it on, I think it's becoming something of a viral force for monoculture, too. 20 years ago, I might occasionally reboot to Windows or fire up a Windows virtual machine in order to open up the occasional recalcitrant Word document in Word itself. Nowadays, if I'm not running Linux on the metal, then I need to keep some sort of a Linux virtual machine running constantly, because so much software has a hard dependency on either the GNU userland or a Linux kernel feature such as LXC.
This is worrisome to me, because, while GNU/Linux is FOSS, "free and open source" doesn't actually mean "unfettered". A small number of large corporations such as IBM and Google have an outsize amount of influence. Doubly so now that IBM owns Red Hat. And, as the whole GNOME/systemd thing demonstrates, even the non-corporate entities have a lot of opportunity and incentive to engage in rent-seeking behavior now that things have been whittled down to such a tight software monoculture. They'd presumably not have even attempted such a thing as taking a hard dependency on systemd if maintaining decent compatibility with FreeBSD (or any other OS) were still a relevant priority.
Yes, this. And SystemD adds to this monoculture (as does Docker).
I realize I'm picking a nit, but I think that, in this situation, it reflects a distinction that's worth making:
Systemd and Docker do entrench the monoculture, but they can only have that effect because the monoculture is already so well-established. If this were EEE, they'd be part of the transition to the third E.
We lost ownership of our systems. No one wants a BOFH built and maintained IT system no matter how wonderful it is at actually meeting business needs, because BOFHs aint interchangeable and business built on individuals aint a thing anymore.
So open source is full of apparent forced goupthink and all or nothing half solutions, at least in the corporate world.
There's home and hobby and personal business systems that are really slick using the lesser known packages and things like "I forked this BSD kernel years ago and twiddled it since". We hear about them sometimes in news stories about "Airport HVAC system found to have been running on C64 for 19 years." Perhaps it inaccurate to say its 'dead or dying' software, until it's no longer appropriate for those use cases where we never even notice its still running.
I've decided to migrate my Linux boxen to BSD (this is a long-haul project that will take me a year or more, I estimate). In part because systemD itself really made me aware of the issue that you're raising.
Which BSD, may I ask? I’d personally recommend OpenBSD, for code quality and “it just works”, but there’s a lot of really interesting BSDs that are even more obscure (Dragonfly, Debian and Gentoo using BSD kernels, etc) aside from FreeBSD and NetBSD.
It's worth watching this[1] talk about systemd by a FreeBSD person, of all people.
1 - https://www.youtube.com/watch?v=o_AIw9bGogo
In fact many people did complain about the existence - I have read actual demands that distros should actively work against it back in the days. Also the personal attacks against Lennart make it difficult to see any rational point in the whole debate.
How is systemd any more of a monoculture than the Linux kernel itself? In principle they are more alike than you might care to admit.
That's a problem too, but existence of other problems doesn't excuse this one
There's a definite [citation needed] on the assertion that integrating tasks into one project/process == monoculture.
And a [citation needed] that systemd is large enough to warrant being called a monoculture in the first place—and to flip the coin, that GNU init is small enough to not be a monoculture.
Let's not forget a huge [citation needed] that the addition of another alternative into the marketplace of ideas could ever be described as monoculture.
And while we're at it, how about a big juicy [citation needed] that a monoculture is a bad thing anyway, in this instance?
Open standards are more important than open source. Choice is indeed the whole point, but systemd is designed to take my choice away. It doesn't matter how good an init system I make, if it starts getting any traction then systemd will come up with a new incompatible API for doing something, Gnome will take a hard dependency on that API, and distributions won't be able to ship my init system.
It's worth realising that the GNU project could never have happened in the first place - Linux could never have been a thing - if historical Unix had been designed like systemd.
None of that makes any sense to me.
It's all open source.
It's laughable to worry about Gnome taking a hard dependency on an API when hundreds of software programs take a hard dependency on Gnome. Hypocrisy, much?
Meanwhile if the maintainers of GNOME betray your trust, it sounds like you should hate this hypothetical GNOME for the same reasons you apparently hate systemd. Or rather than hating it, you could just fork it. If enough people agree with you, a disentangled fork will inevitably be maintained. Hell, people maintain forks of GNOME for BSD—and that's even more impressive since there is no open source monoculture more monolithic than the Linux kernel API dependency.
You could quite possibly be right that the open source movement couldn't have have started with the design principles of systemd[0] but it's naive to assume that what was good for open source in its infancy is still what's best for open source in its coming maturity.
[0] Given that the open source movement has monumentally benefited from monolithic design principles in the form of the Linux kernel, I think that hypothesis is demonstrably absurd and easily rejected.
> If the maintainers of GNOME betray your trust, it sounds like you should hate this hypothetical GNOME for the same reasons you apparently hate systemd. Or just fork it. If enough people agree with you, a disentangled fork will inevitably be maintained.
And then they move on to the next project and introduce a hard-dependency on systemd there. RedHat has more programmers than me, they're always going to be able to add systemd dependencies faster than I can fork things. The only way users will ever be able to have a choice of init system is if the community can agree that hard-depending on systemd is indeed a betrayal, and publicise this.
> it's naive to assume that what was good for open source in its infancy is still what's best for open source in its coming maturity.
Open source succeeded because it was easy to improve and replace things piecemeal. That's still as important as ever.
> [0] Given that the open source movement has monumentally benefited from monolithic design principles in the form of the Linux kernel, I think that hypothesis is demonstrably absurd and easily rejected.
Linux has suffered from its monolithic approach to kernel-space development, but the damage was mitigated by an extreme commitment to providing a solid API to userspace that never gets broken. So most progress took place in userspace and wasn't affected. There's no analogous boundary for systemd.
None of that makes any sense to me.
It's all open source.
Your last paragraph is absurd and internally inconsistent. Linux (the larger operating system environment) exists only because of its monolithic approach to kernel-space development. Far from suffering from it, it made it all possible when the utopia of the GNU kernel flat-lined.
Open source has succeeded because it was easy to improve and replace things piecemeal. But deciding that the individual pieces must begin and end at the process level is blinkered. You can perform "piecemeal improvements" to the scheduling component in systemd just as easily as you can to, say, cron.
> Linux (the larger operating system environment) exists only because of its monolithic approach to kernel-space development. Far from suffering from it, it made it all possible when the utopia of the GNU kernel flat-lined.
Linux-the-kernel was largely just in the right place at the right time, but to the extent that it had development advantages they were an openness to contributions and the absence of an imposed top-down design.
> Open source has succeeded because it was easy to improve and replace things piecemeal. But deciding that the individual pieces must begin and end at the process level is blinkered. You can perform "piecemeal improvements" to the scheduling component in systemd just as easily as you can to, say, cron.
You can submit changes and improvements to a given cron implementation. But if there's a really hard tradeoff or a disagreement, you can also fork off your own cron implementation, and in fact there are three or four different cron implementations in common use on linux. This is really important and not just in the cases where a fork actually occurs: the possibility of a fork encourages maintainers to be reasonable and come to a consensus.
Systemd's tightly coupled design makes that impossible - it's not about process boundaries but about having any kind of boundary where there's a stable interface between different components. E.g. reasonable people can disagree over whether the advantages of a binary logging format outweigh the disadvantages. But the systemd maintainers have taken a particular position, and, crucially, people who disagree can't just fork the logging component and swap it out for their piece, because there's no stable interface between that and the rest of the system.
None of that makes any sense to me.
It's all open source.
Suggesting that you can fork cron and you can't fork systemd is patently absurd.
You say that "people who disagree can't just fork the logging component and swap it out for their piece" when in reality you really can. Getting systemd to output text logs is trivial, just point it to syslogd instead of journalctl. It sounds like you weren't aware of that. So much for being a tightly coupled design.
You say the Linux kernel had an "absence of an imposed top-down design"?
Ha.
Haha.
Am I supposed to take that seriously? The Linux kernel wasn't a top-down design? Do you remember the Linux kernel? Do you know who Linus Torvalds is?
> RedHat
Soon to be IBM, darling child of precisely nobody on HN. I think a lot of people are still in denial over this acquisition.
You are presuming that everyone gives a damn about open source as an end unto itself, not merely a means to an end. Ironically, the people who do consider it an end unto itself are probably the people most likely to despise the term "open source" and prefer "Free Software".
Ask RMS what he thinks about SystemD and I bet you he'll shrug and say it's fine because it's Free, and that's all he really cares about. Needless to say, most people have a greater diversity of concerns.
Trust me, I don't give a damn about open source as an end in itself—to me RMS means root mean square first, Roads and Maritime Services second, and occasionally I might be reminded of an unwashed beard.
> "Trust me, I don't give a damn about open source as a means to an end"
See, I think that's where you differ from most people. To most people, I believe open source is a means to an end, not an end unto itself. What they want first and foremost is something that works, and it happens to be the case that having the source code freely available facilitates having something that works and gives them a sense of comfort, an assurance, that it will continue to work in the foreseeable future. So appealing to "it's opensource, so you should like it" totally misses the mark. The argument should be "it works, therefore you should like it."
Had SystemD never given anybody any trouble, I don't think it would be hated. I think SystemD is hated because (particularly during early adoption years) there were a lot of things about it that didn't work for a lot of people. This made them angry, because it violated what they actually cared about. Not permissive licensing, but rather having something that works. The principle of open source is not axiomatic to most people; it only has value when it is part of having something that works.
Somewhere else I see this mentality: Android. There are a few among the 'community' who would have me pop open the champagne because "Linux won" on account of Android being an incredible commercial success. But why should I celebrate that? Why is "number of CPUs running Linux" the metric I should be glued to? Because Linux is open source? So what? Honestly, who gives a damn?
Apologies, I erred in my previous post and meant to say the opposite. It is now edited. Sorry.
(Hopefully that makes my aggressive indifference to Stallman more comprehensible.)
It's not SystemD itself that's the problem; it's just technically terrible, but, as you say, it's open source. A lot of open source software is terrible but popular.
It's Debian violating their social contract that was the problem for me.
What the vast majority of distributions do is irrelevant, a lot of them are chasing popularity or have a proprietary interest in having a "unified platform", but I had come to expect better from Debian. There is really no longer a "reality of that shift" to reflect on, since the SystemD problem has been more or less fixed: there are now debian-based distributions to upgrade to. If other people want to run that dumpsterfire of cowboycode on their computers, that's fine by me.
> It's Debian violating their social contract that was the problem for me.
Which clause of the social contract did Debian violate?
https://www.debian.org/social_contract
"Our priorities are our users and free software"
Debian ranked SystemD adoption over their users, since it broke users' computers, even though there were viable alternatives available that would have not broken these systems.
I still like Debian, but I'm not letting them (directly) near my systems again.
"Breaking user's computers" sounds like a bug of severity "critical", which would block the release. Can you point me to your bug report?
Most arguments about systemd will devolve into a Motte and Bailey argument. The problems with systemd are all the stuff tacked on that are mandatory, but if you criticize them you'll get people switching to try to put words into your mouth about how the init replacement is good/bad.
The Motte is all the shovelware attached to systemd that is nearly impossible to detatch and the Bailey is the init replacement.
There's a lot of truth in what you're saying, but I think you have "Motte" and "Bailey" backwards.
You are right! Fortunately the sense can be easily obtained.
Welcome to 21st century arguments.
Humans used to be able to disagree and still get along, but that was boring for drama-preventing reasons.
More on topic, though, I wonder: What exaclty is the relationship between Alpine Linux and ungleich.ch?
There are no official ties between Alpine Linux and ungleich.ch. We at ungleich are simple people liking simple systems. And we think it is important to support systems that are not mainstream.
That's why we endorse Alpine Linux.
The “shovelware” is what makes it useful.
In my opinion, as an init script replacement, systemD is almost a great thing. Including everything else is what makes it a terrible thing.
But...it doesn't include "everything else". My desktop doesn't use systemd-timesyncd, I'm not even sure systemd-nspawn is installed on my home server, that same server isn't using systemd-resolved. I haven't fiddled with journald anywhere 'cause I like it a lot more than rsyslogd, but you totally can forward journald stuff to rsyslogd if that's more your bag (and I think, though don't quote me on that, you can remove it entirely if you want).
Yes, there are additional, thoroughly optional, components to the systemd ecosystem that you can choose to opt into. And...so?
You can't replace journald. See the 10 times this has been referenced throughout the thread.
The thing I don't get is how bothered people are by it existence despite being able to just forward logs to syslog and then ignore it forever.
And then it's like it was never there. But technically it's better than if it was never there because you can still capture service stdout/stderr and it will show up in the syslog where on an older system it would just be dropped.I have way too many cases of systemd startup items hanging with their timeout for no good reason.
Worse, my most complicated high-uptime machine usually does not shut down in a reasonable time. Systemd says "waiting for session user cracauer" (something like it) for whatever reason. It also hung on undoing swapspace, when that swapspace was a custom stack of block layers. I don't need swapspace to be "shut down". It went into 20 minutes something always stating another timeout when one expired. I mean, WTF?
The problem here is that I had to do an unclean shutdown on that machine (reset button) multiple times, and that I really can't have.
This also illustrates a point I have been making about Linux and the BSDs from day one: When the BSDs boot or shut down the keyboard is connected to the rc scripts. You can do Control-C for SIGINT and Control-\ for SIGQUIT, The latter makes everybody in the stack leave a coredump on disk.
Now, compare these two:
1) in BSD when there is a startup or shutdown item taking too long or hanging I can abort that single item and I can later debug what was going on with the coredumps.
2) on a Linux system with systemd I cannot "cut through" startup or shutdown items I don't want to wait for, and there is no way to debug any of that if the machine doesn't actually come up completely, or if thing happen at shutdown. The only interaction I can conduct is the reset button.
To add insult to injury, systemd also hides a lot of error messages that would traditionally be dumped on the console, and instead of giving me that error message systemd captures it and print on the screen instructions how to get that error message (which usually are longer than the error message was, WTF?). Instructions that I cannot follow because somebody detached my keyboard, and because I will never be in that machine in an up state while that context is available.
This also invalidates the point that systemd could be one platform that you learn once and then use on a wide basis. The debugging abilities are nowhere close to adequate, so you have to learn by try-n-error.
I've struggled with, and am still struggling with, units hanging on shutdown. It's my biggest complaint with systemD aside from journald eating its own logs.
I've been using magic SysRq to force hanging units to quit. SysRq + e sends SIGINT to all processes, and SysRq + i sends SIGKILL.
Sending SIGINT to all processes isn't what is needed here. It needs to go to the "processes" that are currently going on. With systemd's parallel execution that can still be narrowed down in a useful way, especially is some tail of some hanging command is running alone by now.
Obviously what the BSDs are doing, sending SIGQUIT to just the currently blocking-the-shutdown processes, is far superior. Now the coredumps are floating around the filesystem and they'll be there after reboot, and you can see who did what and was hanging in what backtrace.
systemd errs on the safe side and thus it tries really hard to shut things down safely vs just yanking the plug. The timeouts are easily configurable if that's a problem for you, but I'd argue that if services hang too often for you, there's an underlying problem with your system somewhere, (i.e. your mounts or such, I'd consult journald for dependency cycles).
There might be some way in which I had run a system that caused those endless slowdowns.
But the point is, and that's why I think systemd is garbage, that there is nothing there that helps me debug what is going on. If the shutdown is hanging there is nothing to do except the reset button or waiting for an eventual reboot. In neither case will there be any forensics about what exactly systemd had tried to do and how it thought that it failed.
That is the part that is not acceptable.
P.S. I shouldn't have to manually adjust a timeout on disabling swapspace on shutdown to prevent a need for a reboot by reset button. The kind of thinking that got big timeouts into those things is precisely why I say that systemd people just think differently than I do.
> that there is nothing there that helps me debug what is going on
That's not really true. Journald is your friend. I find many of the complaints about systemd stem from unfamiliarity with its tools.
The Arch Wiki gives a really nice overview of the most common usage scenarios[1] & [2], that's worth investing some time into. Also [3] is your friend. Just refusing to put any time into learning systemd and then complaining about it, is one approach, but not a particularly productive one.
In your case, I'd try something like `systemd-analyze verify default.target` and go from there.
Additionally, look into /etc/system.d/system.conf, specifically adjusting the Timeout values to a less conservative value.
1 - https://wiki.archlinux.org/index.php/Systemd
2 - https://wiki.archlinux.org/index.php/Systemd/Journal#Filteri...
3 - https://www.freedesktop.org/software/systemd/man/systemd-ana...
> "Journald is your friend."
With friends like these, who needs enemies?
Sure, if you live by 'ignorance is strength', I totally get your sentiment.
SystemD definitely spurred some controversies years ago but is it that bad in 2019?
All major distros adopted it and even some that are listed there as "no systemd" in reality just give you choice (e.g. Gentoo).
I'd gladly hear the opinion of distro maintainers why did they switch to SystemD if it's as bad as it looks like in people's perception.
> SystemD definitely spurred some controversies years ago but is it that bad in 2019?
The reasons for the controversies have not really changed (lennart, the idea that it's not "unixy", a takeover of many subsystems by various systemd components, …) so people who didn't like systemd years ago probably still don't.
As somebody who's not really involved, I found Benno Rice's The Tragedy of Systemd[0] interesting and enlightening.
[0] https://www.youtube.com/watch?v=o_AIw9bGogo
I found that talk incredibly condescending. He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing and apples attempts to upstream the problem to the bsd developers has been met with open hostility.
It’s also to the point where Benno states that people don’t like change, but that’s really not true. Everyone jumped on virtualisation (except Theo De Raat), everyone jumped on containers, Kubernetes, GO, Rust, eBPF et al.
There are new technologies which don’t have to eat the universe to function.
Most people who have a problem with systemd don’t actually hate it, they’re upset because the environment that allowed systemd to grow no longer exists for its replacement. Coupled with the fact that we should really consider systemd a proof-of-concept for something better, means that we’ll be stuck with buggy, weird, non-deterministic C++ for decades.
That’s the root of the problem, not that we nerds dislike change, we tend to love change.
> He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing
This is your personal opinion being presented as fact. Speaking from some experience in the Mac admin world, I’d really want to see the survey you did to support that claim.
systemd is not written in C++.
yes it's still that bad, my previously reliable systems are now non-deterministic and break randomly
every few days I discover some new bit of non-determinism:
I can fix most of these systemd-created issues one by one as I discover them, but it's hardly making my life easierAll of those issues happened with SysV init, as well — there are literally decades of history of people working around things like unmounting failed filesystems, watchdogs to kill -9 hung daemons, etc.
For example, you mentioned crond startup before the network but that's hardly new with systemd — checking the oldest system I have access to shows that it's configured with two dependencies: “# Required-Start: $local_fs $syslog”. To the extent that systemd changed anything it was exposing that an assumption was wrong — and the nice thing is that you can use a systemd service to replace that cron task with something which _does_ behave the way you thought with precise service dependencies and all of the other benefits to using systemd (e.g. resource management): https://www.freedesktop.org/software/systemd/man/systemd.tim...
it's the non-determinism that's the problem
as I said, I can fix the issues one by one as I notice them, but if they only occur 1 boot in 10, or maybe 1 in 100 it's miserable and a much less reliable system than before
but hey, the systems now boot 2 seconds faster (if they boot at all...)
Again, you already had a problem. It might not have been as obvious because it was enough slower to avoid the race condition under normal circumstances but I’ve hit those problems many times over the years.
Again, I've said that I know there are issues, that I end up discovering one at a time.
the problem is not the issues themselves, as once I know they're there I can trivially google for how to fix them, it's the fact they don't show up except sometimes
I do not care about boot speed at all, I care that I upgrade the system to a new distro release with systemd, test it, then it continues to work that way until I change something else
this is no longer the case, and in 20 odd years of maintaining systems I've never had as much "fun" before as I've had over the last couple of years with systemd
I just want my systems to boot, run my simple applications, then shutdown when I ask them to
> I just want my systems to boot, run my simple applications, then shutdown when I ask them to
Which is what you get out of the box with systemd. The only problems you mentioned either apply equally (e.g. shutdown delays due to blocked processes — having supported NFS clients, let me tell you that was not a new problem in the 90s…) or are due to misunderstandings about how to use other components.
For example, if you have a critical service started in cron with no error handling or recovery, you always had an unstable system and were simply lucky not to have seen it fail before now. Projects like Upstart and systemd were started in large part due to people getting tired of having to kludge around those limitations of the older approaches. Having fixed systems which were configured that way in the past, I much prefer using something which solves it by design than continuing to bet on being lucky.
you are not listening or being are deliberately obtuse
I have stated three times that I accept that these processes are incorrectly specified, and that my problem is not this -- the problem is that I cannot discover these missing dependencies by any reasonable process
whoever set them up 20 years ago is long gone, and what was reliable is now not
this is a regression
(and let's not get started that my brand new installations of Debian onto a single root partition get stuck booting or reboot 1 in 10 times... I get it, totally my fault as the end-user)
> you are not listening or being are deliberately obtuse
Any time you find yourself writing something like that, ask whether you’ve done a good job explaining your point first. In this case, you’ve been vague about everything except that you think systemd is the source of your non-deterministic behavior.
From what little information you’ve provided, it really sounds like the problem is a combination of blaming the init system for things which happen outside of it (i.e. processes hanging) and not learning how to use it. For example, launching a networked process under cron at boot with no retry mechanism has always been a problem because cron never guaranteed networking. Using the init system would have avoided that - even on SysV.
As far as how to discover it goes, fault tolerance has lots of edge cases — e.g. try mounting NFS and firewalling the server, `kill -STOP` a daemon with a clean shutdown process, reboot without a network or DNS (i.e. like you’d have during a facilities outage), etc. — but most of it comes down to learning to work with the way the distribution is intended to be used: use the init system (whatever it is) to launch processes, enable automatic restarts for everything, configure full dependencies on everything you need, etc.
Note that none of that depends on systemd - you can get better results on everything this way, but if you do it you’ll probably gain a better understanding the systemd developers made the choices they did. If you really can’t figure out how to be comfortable with it, switching to a distribution which doesn’t use it will be much better than fighting it and getting angry when the developers don’t support hammering a screw.
Suppose you run your system on a new, better, and faster computer architecture, newly supported by Linux. Now, let’s assume that this is a new architechure which in many ways is different in style to other processors, but let’s say GCC supports it and can generate 100% correct code from it. Your problem now is that much of all the old software was written and tested only on older architectures, and this new architecture exposes many bugs such as previously unknown reliance on undefined behavior, which worked on all old architectures, but not this new one.
Now. Do you blame the new architecture for this bad situation? Or do you blame the old software, which technically always had bugs but those bugs were previously not triggered?
The harder it is to avoid undefined behavior, the more I blame things other than the software. Especially when you can run the old version under emulation and it still works without bugs.
And a lot of the blame goes to the distro that shipped a bunch of broken binaries.
> The harder it is to avoid undefined behavior
That is surely an aspect of the language, not the platform?
> And a lot of the blame goes to the distro that shipped a bunch of broken binaries.
Sure, I can see that. It is normally the case with new ports to new platforms to be in a state of unreadiness for quite a while. But this is not really relevant to the question of who to blame for the brokenness; the hardware platform or the software?
My analogy is that systemd merely exposed bugs in other software which had been there for ages (and only sporadically exposed). But then systemd comes along and expose a whole lot of them at once. Keeping with the analogy, some blame certainly goes to any disto which released this combination of systemd plus software which has bugs triggered by systemd, but still, is it reasonable to blame systemd in this case?
I have the same getting stuck problems plus resolving issues as soon as systemd-resolved is in the game
I think some of my pain came from attempts to unmount everything (particularly NFS) before letting long-running shutdowns (graphite/carbon) finished flushing, then never trying again. While diving into that and eventually coming up with a sufficient set of dependencies, I would fondly think back to the two (or more) phases of unmounting at shutdown that had that solved cold many years prior.
It seems the systemd hate mostly comes from a philosophical position these days than anything grounded in real user impact. You either agree or disagree with what systemd does, or do not care because you are not aware of it (which I would argue is a good thing).
Personally I think it is great. Sysvinit scripts were terrible to write and frequently had bugs because the scripts were usually afterthoughts "Oh yea... I guess we need one of those things". The nice integrations between systemd and journald is well worth it in my mind as well.
> Sysvinit scripts were terrible to write and frequently had bugs because the scripts were usually afterthoughts "Oh yea... I guess we need one of those things".
There were alternative solutions to that problem that didn't require replacing a long and ever-growing list of system components with tightly-coupled (and often buggy) systemd versions. In fact people who really cared about their init system were already using them.
Well systemd service files are great and represent a nice improvement. But I detest journalctl with its arcane query commands and binary storage; much prefer ability to use standard unix tools and my favorite editor for adhoc inspection at least. It's still is an annoyance on a pretty regular basis.
You can tell journald to forward everything to rsyslogd and grep to your heart's content. https://wiki.archlinux.org/index.php/Rsyslog#journald's_sysl...
How do I remove this buggy attack surface from that workflow and why should I have to?
"How do I remove" - I dunno, because I don't care to remove it. Somebody's probably done it. Search on Google? It's open source, so look at the code, submit a patch, fork if you gotta?
"Buggy attack surface" - I am unaware of attacks that target journald; can you point me to a practicable PoC attack?
"Why should I have to" - because other people are building your distros, expending their effort on open source projects, and as part of doing that they made decisions on your behalf. Feel free to not use those distros for ones that make decisions you prefer; the choice is yours.
Then you are commenting on the wrong post. This post is about the systems that do not use systemd. Justifying your own use by demanding examples of failure is... odd here.
So when systemd comes up it's cool for people who Don't Even Own A systemd to go on endlessly about how it's terrible and An Conspiracy, but it's not cool for people to discuss that it might not be the Snidely Whiplash evildoers thing by evil people when the shoe's on the other foot?
Weird flex, but okay.
Well, just excuse my philosophical stance of not liking my computers breaking by a new random cause every couple of months.
This is my experience as well.
Suddenly logins take 30s. Why? Some new systemd problem with reverse DNS.
Suddenly reboots don't happen. Why? Systemd is spinning on a task that didn't used to spin. Yes, probably fixable, but not discovered until trying to reboot.
Can't login at coffee shop WiFi suddenly. Why? Some random systemd DNS issue.
Is my system more secure? Maybe, but possibly because it breaks so often that low uptime keeps it safe!
For my servers, I seriously have to cross my fingers and have the colo phone number handy whenever I reboot, fearful that they won't restart because of some random systemd interaction that happened on some previous upgrade. It's gotten that bad.
sysvinit was at least a thousand times easier than bsd init scripts, and I occasionally see people that are opposed to both systemd and sysvinit and think rc.local was 'it'.
shakes head slowly
BSD rc scripts haven't worked in the way that you think that they have, for decades.
Mewburn rc, which reduced many rc scripts to a couple of shell function calls and a handful of variable declarations, has been used by NetBSD since 2001, and FreeBSD started using it about a year later. OpenBSD, furthermore, reinvented Mewburn rc around 8 years ago. Project Trident (formerly TrueOS formerly PC-BSD) has also since then switched from Mewburn rc to OpenRC.
* https://news.ycombinator.com/item?id=10357589
* https://news.ycombinator.com/item?id=11550802
* https://news.ycombinator.com/item?id=14736175
/etc/rc.local was first declared obsolete in the BSD world and replaced by a system of drop-in files in a directory back in 1995.
* http://jdebp.info./FGA/rc.local-is-history.html
> /etc/rc.local was first declared obsolete in the BSD world and replaced by a system of drop-in files in a directory back in 1995.
OpenBSD still documents it as an implicitly-recommended way to start programs (though if it's possible to start it via e.g. rc.(local.)conf, then that's obviously preferable).
> sysvinit was at least a thousand times easier than bsd init scripts
¿Qué?
I guess we'll have to beg to differ here, but I prefer OpenBSD's (for example) init system over sysvinit or systemd any day. rc.subr is a godsend for eradicating all the boilerplate that "justified" switching to systemd. Enabling/disabling a daemon is as easy as adding a line to /etc/rc.conf.local (or just running "rcctl enable mydaemond", which does that for me). No fiddling with symlinking scripts to runlevel-specific folders (or, in fact, dealing with runlevels at all; if I need single-user mode, that's what bsd.rd is for).
One of these days I'd love to start ripping out the initscripts on my Slackware machines and replace them with something closer to OpenBSD (or hell, I wonder if I can just port OpenBSD's init to Linux?).
I don't hate it, but it has wasted more of my time as a sysadmin than anything else over the last few years.
The objections to systemd are still as valid as ever. We still see a pretty high rate of security vulnerabilities in it.
However, as projects adopt systemd-proprietary APIs, distros are forced to choose between maintaining ongoing forks of those projects, dropping those projects, or adopting systemd. E.g. I suspect Slackware only avoided being forced to adopt it because they'd already decided to not support gnome.
High rate?
Compared to the Linux kernel, systemd's CVE rate is very low.
I have to call this out as a very poor comparision. I would love to know the linux kernel commits in 2019 vs CVE's in a percentage, vs systemd commits in 2019 vs CVE's in a percentage (or 2018 or 2017).
Now compare systemd CVE's with OpenRC or runit CVE's or any other init system and then you have an argument. But comparing it to the kernel with hundreds of people commiting changes is weak.
This comment is not defending nor attacking systemd.
Systemd has a historic of refusing to fill CVEs, even for incredibly bad vulnerabilities.
Compared to sysvinit, upstart, or procd, say, to remain in the same domain space?
(Though the implication that systemd is an OS Kernel is ... interesting.)
Don't you know? systemd is the world's most popular operating system! A shame it has such a terrible init system...
Do you have a source for it?
In 2017, the Linux Kernel had over 450 CVEs, while systemd had 5.
https://www.cvedetails.com/product/47/Linux-Linux-Kernel.htm... https://www.cvedetails.com/vulnerability-list/vendor_id-7971...
So far in 2019, the gap has narrowed: "only" 45 CVEs so far for the Linux kernel, and 7 for systemd.
As marcosdumay said upthread, “Systemd has a historic of refusing to fill CVEs, even for incredibly bad vulnerabilities.”
This contributes to the low rate of CVEs in systemd relative to Linux (for which many less-important bugs get CVEs).
A quick look at the list of CVEs in the grandparent comment indicates that there are at least 3 different Assigning CNAs for systemd CVEs: MITRE Corporation, Red Hat, Inc. and Canonical Ltd.
My conclusion is that the systemd project is not its own CNA and therefore not in a position to authoritatively decide which systemd bugs get CVE numbers assigned and which do not.
Sure
https://www.cvedetails.com/product/47/Linux-Linux-Kernel.htm...
https://www.cvedetails.com/product/38088/Freedesktop-Systemd...
For a piece of software used by millions of machines, that's a very low CVE count.
> We still see a pretty high rate of security vulnerabilities in it.
Prove it. As someone else pointed out, there were 5 CVEs for systemd in 2017 and 7 so far this year most of which are low severity. If you think the mess of shell scripts and programs that systemd replaces had less than that you're simply delusional.
Earlier versions of systemd were fairly buggy, but then it was still in its infancy and its in a much better state now.
systemd (not SystemD) changes a core part of the OS, we go from a userspace being bootstrapped by a bunch of different shell scripts calling a dozen individual binaries, to an actual system and service manager configured with unit files.
It's a large change, although one other operating systems went through a decade earlier, and some people don't like change.
> and some people don't like change.
People really need to stop saying this rather than taking the time to actually understand what people are objecting to.
I like change! I switched to runit and are not looking back.
By that standard SystemD proponents' fetishization of ineptly implemented change for change's sake clearly makes their position invalid.
Why on earth do systemd haters insist on capitalizing the D? It's really amazing how people think mistyping the name is some kind of useful or novel criticism of the software.
It's a classic joke. http://catb.org/jargon/html/S/studlycaps.html
the -d suffix is traditionally for service daemons. SystemD is not a service daemon.
Lowercase is a UNIX convention. SystemD does not have a UNIX approach.
Since SystemD is not a Unix service daemon it shouldn't be spelled "systemd" to prevent confusion.
This is...bizarre.
Should I pejoratively call you "CRaIg" because I think you don't match up to my standards of what a Craig, or even a craig, is? Is that okay?
I don't think it's okay. Names have meaning. It is a really minor bit of respect to use them correctly.
I don't follow. As you state yourself, "names have meaning". By conventions the developers actively disregard, the meaning of the name "systemd" implies several things that the program is not.
I think you replied to the wrong comment.
However juvenile, it is useful – as soon someone writes “SystemD”, I know that they’re either very inexperienced and don’t really know much, or they’re deliberately being spiteful and want to signal their hatred to other haters. Either way, I then know that I can safely ignore them completely, since they’re not part of a constructive conversation.
I've been pointing out that people do this, for some time.
* https://lists.debian.org/debian-devel/2016/08/msg00547.html
* https://news.ycombinator.com/item?id=14171389
I think that giving unflattering nicknames to powerful entities is a long-standing tradition, and a valid act of protest and free expression. Of course it won't convince an opponent, and it may backfire as a rhetorical device, but I empathise with people wanting to use it.
Moreover, the term "System D" is somewhat synonymous with the term "black market", which Wikipedia defines as a: "series of transactions that has some aspect of illegality or is characterized by some form of noncompliant behavior with an institutional set of rules."
That is a surprisingly good analogy for how systemd acts on a machine, but there is another analogy to a similarly worded term: "Substance D", from the novel "A Scanner Darkly". In the novel, it is described as an addictive psychoactive drug, also known as "Slow Death". Credit for that comparison goes to this foundational criticism of systemd:
https://lkml.org/lkml/2014/8/12/459
People's perceptions aren't unified. Some people will always dislike SystemD, and they are not be forced to use it. If every major distro maintainer loves it, some people still would hate it.
There's still a group of Unix users upset about the major init split that happened fourth years ago. Things will never be unified.
I think the main problem is not the init process, but the tooling around it that breaks systems. For instance systemd-resolved did break local resolving some years ago (maybe still today).
While it is optional, ubuntu installations come with it turned on by default.
They didn't really have an option: the same organisation controls GNOME and systemd, so they introduced systemd as a hard dependency of GNOME to force distros to adopt systemd. And you can't have a distro out there without GNOME, so...
Those distros that can run without systemd have to make their own patches to "fix" GNOME, which is very costly. Even Gentoo struggles with that, as it takes them a long time to release new versions of GNOME because they have to write the patches to strip out systemd first.
>And you can't have a distro out there without GNOME, so...
LxQt and any KDE-powered distro would beg to differ.
But I agree, in general, that KDE-default distros are few and far between. And the KDE team probably has better things to do with their time than try to maintain a systemd-less fork.
They're rare in the US. KDE-as-default is close to the norm in Europe.
KDE has long been a more crossplatform system than Gnome (much better windows support (and even e.g. solaris), BSD is treated as first-class) so I'd hope they'll continue to avoid being systemd-dependent.
>They're rare in the US. KDE-as-default is close to the norm in Europe.
Based on what?
> And you can't have a distro out there without GNOME
cough cough Slackware cough cough Kubuntu/Xubuntu/Lubuntu cough cough Tiny Core Linux cough cough the literal dozens if not hundreds of distros that ship with a desktop environment that's not GNOME or MATE or Cinnamon or some other derivation thereof cough cough cough
> as a hard dependency of GNOME to force distros to adopt systemd
Or because they like systemd and they think the abstractions it provides are useful and its better to not have that code in GNOME. But sure, lets go with the conspiracy angle.
Also, pure serve distros also adopted systemd.
> Those distros that can run without systemd have to make their own patches to "fix" GNOME, which is very costly.
OpenSource project not implementing the features some users want them to implement. Shocking stuff. Its almost as if that's the bases of Open Source. If you don't like the choices of the project you can take the source and do your own thing.
My experience with systemd has be extraordinarily positive. I like the unified syntax of unit files. I know that's like 50% of what systemd does, but it does it extraordinarily well.
Is systemd-nspawn secure enough now we can forget docker et all? I liked the idea of a "chroot" being supported by the service manager.
Love this!
Notice that the text about slackware sounds a bit "pessimistic", as if it was a somewhat legacy distribution. Nothing further from the truth. We run a small farm of virtual machines with different up-to-date distributions, and slackware is typically the one that follows upstream packages more closely. For example it had gcc 9.1, bash 5, etc. a few days after they were released.
Under -current, though.
That is pretty cool!
I don't like it when my process init and logging system also tries to manage my volume mounts. If I unmount a device, I don't want systemd going behind my back and remounting it for me.
It's non standard behavior changes like this that drive systemd hate.
From what I observe, the hate from systemd USERS towards non-systemd advocates is higher than the hate of non-systemd advocates to systemd.
I've never seen systemd users make t-shirts against the others, send death threats, hijack any conversation where systemd might come up to advocate their side… non-systemd advocates on the other hand…
This whole comment section certainly supports that observation. Massive amounts of "you're doing it wrong" instead of compassion.
I find it quite convenient, although I do admit the integration with /etc/fstab is a little weird. I've used it to auto-mount application data directories, for example, before an application comes online.
If that's your goal, use automount or mount everything at boot. systemd is not the correct answer here.
Alpine is great. I have nothing against systemd but Alpine stands on it's own for containers and VMs. I haven't given it a try on baremetal but I imagine it does just as well there. Not really desktop focused but if you want to run it with a GUI on a laptop it's technically possible.
I've actually tried it on my new notebook a few days ago, it runs quite nice: https://www.nico.schottelius.org/blog/alpine-linux-on-the-hp...
Quite nice...?
>checkout why the firefox package is broken
You call that... nice?
In the benefits of Alpine, you have put that the sound is great. Out of curiosity, what makes it better than other Linux distros?
Simplicity mostly. It just works. It feels so clean, nothing running there that is not supposed to be there. Installing something takes only sub-seconds, because packages are small.
Alpine really is a beauty.
I believe your questioner was specifically referring to your assertion that audio on Alpine is unusually good. It is a somewhat odd observation, since ALSA is entirely a kernel API and Alpine is using the same kernel as everyone else. The only possible source of audible difference would be not running Pulseaudio, which resamples audio (some say, badly). However, you can install Pulseaudio on Alpine, and you can not-install Pulseaudio on most anything else, so it wouldn't really be fair to call that a feature of Alpine.
I currently use Alpine as my daily driver with GNOME, which is an odd mix I guess, but works nicely :)
I don't trust a "security-oriented" distribution with a small developer community:
https://threatpost.com/alpine-linux-docker-images-unlocked/1...
Note this is only an issue for docker images using non default tools. Not good but not unique, for instance here is another high scored CVE for PAM issues in Ubuntu/Debian https://www.cvedetails.com/cve/CVE-2009-3232/
I challenge you to find any decade+ old OS software community you couldn't make the same comment towards.
The biggest complaint I regularly see about systemd from those that don't like it is that it violates the UNIX philosophy by combining too many functionalities into one project.
Based upon this I found the following quote from the article in regards to Alpine Linux, a distribution based upon BusyBox which implements the functionality of over 100 usually separate executables into one, quite amusing.
> It is small and simple, realising the UNIX spirit to the core: doing one thing and doing it well.
The difference is that with busy box, everything is optional (just replace the symlink in /bin with another implementation), and designed as a drop-in replacement for standalone utilities (instead of redesigning them poorly from scratch).
The number of binaries is irrelevant. Systemd has a lot of binaries but is still called a monolith. Unlike systemd with journald, Busybox allows you to use Vim instead of Busybox vi without any trouble. Busybox might also emphasize the “small and simple” aspect much more than GNU tools, which have a reputation for being needlessly bloated (see all of the flags to cat, for example).
It does one thing, everything.
I have been running Void Linux since Debian switched to SystemD both at work and at home (including playing Skyrim SSE on Steam using Proton, or StarCraft 2 and Diablo 3 using Lutris) .
It is a very nice distro although I would not recommend it to beginners since it is a little rough around the edges if you want to do full disk encryption during installation.
Initially I switched to FreeBSD and then OpenBSD but I missed Linux conveniences like native cloud sync clients[1], Steam, Docker, support for the hardware I already had, etc.
[1] this was before I discovered Rclone
I use void now on everything, work, home even my Raspberry PIs. Can confirm that installation is a PITA. If you want something more fancy than the standard installation like an encrypted boot disk you won't get anything running without touching chroot a few times.
But once it runs it runs! The most "stable" rolling release distro I have tried so far. Something that can not be said about Gentoo.
The most recent reason I found to hate systemd was it replaced my system DNS resolver with its own (thus requiring me to modify /etc/systemd/resolved.conf, and doing the daemon-reload-service-restart-hokey-pokey, to get a host to function on a network+VPN like it did before). The only reason systemd has its own resolver is they wanted extra DNS functionality. Did I ask for these features? Did I want a new resolver? No. But that's the whole idea behind systemd. A few devs shoving complex features down everyone's throats, just because they felt like it. Operating system fascism.
But I don't think we should be angry at systemd developers; they can develop whatever they want. I'm mostly angry at the distributions who opted in to their crap. Systemd has not made my life easier, but it has often made it harder, and the only reason that happened was because a distro allowed it.
Debian shipping this despite the obvious backlash was the whole reason I jumped ship too, that and the frequent boot failures and looooong timeouts (which I'm sure someone will be happy to tell me are my own fault).
I don't think being angry at anyone will solve the problem. Using and advocating non-systemd distributions might help much more.
Being angry does not solve any problem, but it is a legitimate human experience that the OP considered worth sharing.
I've actually done sysvinit style setup before and had to migrate to systemd when it came out later. It's not that bad. It's just another way of running things. It's actually quite simple and consistent. It's a step forward.
Systemd is the worst init system except for all the others that have been developed from time to time.
Some of the trade-offs it makes, and its expansionist attitudes, are really grating. But it introduced a higher level of consistency in the Linux world for userland developers, which is almost invariably a good thing from a commercial perspective.
> from a commercial perspective
Nailed it. The hobbyists are revolting. Anyone interested in making their system do something odd and clever is having problems, and the people running linux systems 9-5 are busy calling us morons for trying or even caring...
I maintain systems with and without. I thought I understood systemd, but I've given up trying to predict the behavior of systems running systemd.
The tragedy for me, is that I don't really like how the Init systems work, either. So there's really no one on my side. I seem to always fall into the trap of the Dreaded Centrist.
One of my biggest concerns about systemd is the ownership. Sure it is open sourced, however RedHat has a lot of pull on its maintainers (Soon to be IBM). What is to stop them from dual licensing it or close sourcing it. They have lots of products like that (jboss/wildfly...spacewalk/satellite). After everything runs on systemd the ball in in their court. Sure people can say they are going to fork it, but in reality how often does that work out (I concede are some examples where it does, X11, nextcloud, etc.).
In my paranoid mind I can see Redhat commercializing it and adding features to it for only paying customers after they win through attrition the monopoly on the OS (after systemd incorporates everything, resolvers, cron, ntp, etc...). They are first and foremost a business with investors and a bottom line.
Again not an attack on system itself, just my untrusting and paranoid nature (to many years of security scans and cve’s and users, can’t trust users)…
The moment it stops being free Debian drops/forks it.
Furthermore, if Red Hat can monetize a system manager daemon, well ... I mean, good for them.
Oracle tried with the JVM (they had some commercial features).
Furthermore, if systemd provides a lot of basic facilities (global/user services, device management, login sessions, cron, ntp, DNS resolving) someone ought to maintain and support it, and if a for-profit company does it, I don't mind if they try to cover those expenses. (As long as it's also GPL2.)
Systemd is not GPLv2, according to wikipedia it is LGPLv2.1+.
Of course someone here at work told me i am splitting hairs and they are pretty much the same thing and compatible.
systemd is pervasive in the GNU/Linux world and I think it’s futile to stay on GNU/Linux and try to avoid systemd. Just look at the list of distros in the article: a bunch of obscure forks of prominent systemd distros like Debian and Arch, a distro (Gentoo) whose main claim to fame is compile it yourself, one where they say a leading Google search is whether the distro is dead, and one where they say “oh gee, we haven’t heard of it, let us know if it works.”
Seems to me that if you hate systemd, it’s time to try a BSD. GNU/Linux has been going down this path of complexity for years. These same arguments spawned with udev years ago, and Slackware held out then just as it is with systemd. OpenBSD on the other hand embraces simplicity and the old UNIX way. Of course it also lacks niceties like Bluetooth, but that’s the price you pay. Maybe systemd is needed to manage complex systems that support ugly things like Bluetooth.
To be precise: Gentoo allows one to run with or without systemd, and both ways are supported.
What is the default at the moment?
To be clear, you choose at install time. There is no official installer (on purpose), so the user either decides to turn to page x for OpenRC or page y for systemd.
OpenRC: https://wiki.gentoo.org/wiki/Comparison_of_init_systems
I wish there were some way to favorite discussions on HN, not merely the article that started them -- if that feature existed, then this would be one of my favorite discussions on HN ever... why? Well, you have a ton of bright minds, who, for whatever reason(s), can't agree, yet each one adds some valuable piece of information to the debate.
Basically, where there's controversy (especially with bright minds) then there's always something there.
I hope that future system designers and historians will read and re-read these systemd controversies we are having in the present day... maybe we'll have some fundamental key piece of understanding then that we don't now, and based on it, everyone could agree... of course, then again, perhaps that would never happen...
Still, a lively, highly informative, interesting and thought-provoking discussion! A+ as far as discussions on HN go!
systemd is good, but I think it came too soon. A lot of the comments here are about memory safe languages are spot on. systemd was written with two major assumptions in mind:
1. Linux system complexity has exploded and needs a top-to-bottom rewrite
2. It's safe to do such a rewrite in C
Since the systemd project, I'd say #1 has been largely affirmed. I like the result of systemd because it provides uniform tooling around complex Linux systems.
I'd also say the tide has shifted against #2 in a big way, there's much greater skepticism that anyone on this planet (short of djb or a few others) can write safe C, let alone safe C with concurrency and side effects.
This is really unfortunate, because it probably means the solution is to rewrite systemd in a memory safe language, and that's a huge undertaking.
> it probably means the solution is to rewrite systemd in a memory safe language, and that's a huge undertaking.
At least with Rust, the rewrite could be done piecemeal, since Rust can easily interoperate with C (as long as you don't abuse C macros).
What's the recommended way for someone to learn systemd properly? Is there a readable manual somewhere that documents everything?
That's the only thing I find scary about such big systems, the lack of comprehensive docs and a "starting point for learning them" from 0 to hero. Like, with Git there's tons of nice tutorials like "learn kit bottom up by reinventing it in way X" etc. plus a good "default book".
What's the equivalent of https://git-scm.com/book for systemd?
The Archlinux Wiki is my go-to for that kind of thing:
https://wiki.archlinux.org/index.php/Systemd
Also, check the related articles on the right sidebar on the top of the article.
It doesn't document everything, but it quickly gets you to learning the overview, the basics, and what's most important for general usage of systemd.
If you really want absolutely everything the references are the man-pages. Excluding those that document C functions, there seems to be 256 systemd man-pages in total. 36 are regular commands (section 1; like systemctl), 64 are file formats (section 5; like systemd.service), 139 are system administration commands (section 8; like daemons like systemd-networkd), and 17 are miscellaneous topics (section 7; like bootup).
I would consider the manpages of section 1 and 5 to be the important ones, and just section 1 is enough if you never intend to read or write a service or other unit file yourself.
Here's an online index of the manpages:
https://www.freedesktop.org/software/systemd/man/index.html
Good question! A great introduction that was written while the author was still building systemd can be found here:
http://0pointer.de/blog/projects/systemd.html
Also, the "systemd for administrators" series:
https://gist.github.com/bcremer/8cdf6900c35dda65f387
I found the man pages to be a pretty good read.
Use `man -t systemctl` to produce a pdf version.
There are a couple of files you want to read, e.g. systemd.service, systemd.unit, systemd, deamon etc.
But those are linked at the `SEE ALSO` section.
SysV init had the ability to restart, allowing upgrades without rebooting. Can systemd do that?
The mechanism was kind of interesting:
SysV init would create socketpair, then fork. The parent (with PID 1, required for correct functionality) would then exec the new SysV init binary with an option specified to make it receive state over the socket. The child would supply that state, then exit.
You’re looking for `daemon-reexec`:
https://www.freedesktop.org/software/systemd/man/systemctl.h...
> Reexecute the systemd manager. This will serialize the manager state, reexecute the process and deserialize the state again. This command is of little use except for debugging and package upgrades. Sometimes, it might be helpful as a heavy-weight daemon-reload. While the daemon is being reexecuted, all sockets systemd listening on behalf of user configuration will stay accessible.
This anti-systemd meme is getting tiring. I generally find it is perpetuated by people who use Linux at a hobby level. Rarely do I speak with guys in Ops who dislike systemd. I am not trying to discredit people here, there are hobby users who are more knowledgeable about Linux related systems than many guys who are professionals. My point is that Linux and systemd are both meant for use on servers, in production, where speed and consistency rule.
I manage 3000 Linux servers and 20,000 services running on those servers. When I log into a machine and am welcomed by a RHEL6 motd I groan, because sysvinit is not enjoyable to use, it takes longer to reboot those machines, and tracking down logs is a chore.
When I log into a RHEL7 machine I am not elated, but I am glad to know that pid 1 isn't trying to make my life harder.
Hi, ops guy; Fuck systemd.
Boot time: A possible 5-10 extra seconds from sysvinit is the problem here not the 5 minutes of BIOS/EFI POST time for those server boxen?
Logs: Where's the storage of the log data in journalctl and what tools can I use to read that data in a small PXE rescue environment? But hot damn I didn't have to ls /var/log so that's nice.
Sarcasm aside how does it make your life any easier? I got a stack of FIXMEs to work on to get our infrastructure working, lot alone stable, on systemd.
> A possible 5-10 extra seconds
> the 5 minutes of BIOS/EFI POST time
Yeah we measure boot up in seconds. If I restart a machine and it takes more than ~10 seconds to boot I send a ticket to our infra guys to figure out why the server is configured poorly/troubleshoot the slow start.
>Logs
I guess I prefer `journalctl -u service.service -f` or `journalctl -p "crit" --since "2 days ago"` to the ole `ls /var/log` and then `grep`ing
Honestly, I don't understand how it makes life harder. To me, systemd is a huge QoL impovement. Even in the most mundane tasks like `systemctl restart tomcat-*`
Ops person here. When you set up a node, you set up all its networking, its services, its packages, its configuration, etc. It doesn't matter what your technology is - you set it up to do whatever it needs, and then hopefully you never have to re-do that again. Systemd's hydra of services affected a lot of that, so we had to learn all the weird gotchas to get newer distros to work properly.
And it's not just a little different, it's more complex. This increases the number of bugs, the time to test and roll out changes, integration work, etc. So the process is more complex. And the operation is also more complex, meaning more training for anyone who's used to how things worked for 30 years. Doing minor ops work now requires looking at 5 different manuals, requiring extra software, adding configuration management where it wasn't before, changing support instructions, healthchecks/automation tasks, migrations to newer disk images while supporting legacy functionality, etc.
And here's the part that really kills me. Alternatives to all the parts of systemd already existed; if you really needed a better system, you could pick up any similar software and make it work! But then distributions start rolling this out en-masse for all new releases, and we don't have a choice to opt-in. It was a breaking change that did nothing for us that we didn't already do for ourselves, even requiring more work than before.
Do our systems boot faster now? My EC2 instances still seem to take forever to spin up, and physical machines still take ~5 minutes to POST, so I'm going to say no. Can we do new things that we couldn't before? No. Are our systems more stable now? No. Are they less complex? No. So while some people talk up the theoretical benefits, I as an ops person have not seen them. But I still have to look up man pages to remember the three different filesystem trees systemd keeps files in, or the wonky configuration files, or what 5 extra commands I need to get something done.
If you're going to make breaking changes, at least have it make my life easier after I get used to it. Systemd has not done that. And it seems to just keep getting heavier.
Our bottle neck is often pid 1 for boot time. We don't do mem check on boot so that saves us 1 second * Gb_of_RAM in boot, which is why most BIOS take forever on a server. We also don't run anything in PCI slots so we have the PCI checker disabled too (all our servers are Dells they have some software you can license to config the BIOS).
I'm new to the game, I only vaguely remember a time when Ubuntu wasn't on systemd. I don't have 30 years of experience with any init system, but I do have 10+ years with systemd. I didn't use another init until I got this job and had to learn sysv. So I'm coming from the exact opposite, it sounds like, place of you. To me, everything you said is 100% true, but about sysv and its line.
I also don't provision too many servers these days because with the exception of a few things that were deemed "to important" all our servers have automated build scripts to go from 0 to prod.
Being dismissed as a meme is tiring as well. I like hacking on my system at boot time, and dismissing my use case because of your own is kinda hostile.
If you are referring to my comment regarding Linux not being for desktops...
I use Linux on my desktop. The only non-Linux machines in my house are my work Windows machine and my sister in law's Windows machine. All my personal machines, my wife's laptop, and my kids' desktop are all Linux flavors. Linux is a great desktop imho. That said, Linux is not primarily a desktop machine. It gets some pro-Desktop patches but at the end of the day, Linux is designed for servers and paid for by people interested in its usage as a server.
The anti-systemd group contains vocal trolls, memers, and ranters, so anyone who’s on same side will have to live with being grouped with them.
My experience is like 100% different from yours.
I have the feeling systemd is meant exactly for what you call hobby or novice users. I actually enjoy some of it on my laptop to be honest ;-)
But in the datacenter it has caused me hours and hours of 'fixing problems that did not exist before systemd came along'.
Most of it's 'improvements' target the desktop user. Many of it's 'improvements' actually get in the way in the server room.
I mean, why would I want to optimize my boot time on a server with all this parallel startup crap? The server spends 10 times more time in BIOS and POST then it takes to boot the OS. It has always been like this. And it still is like this.
From a config management point of view I like systemd. I gives me a consistent API to enable/disable start/stop services. That's awesome. I also like that I can 'override/patch' upstream units without having to maintain a fork. Perfect.
But many other things that came from the systemd ecosystem are questionable. E.g. systemd-resolved, timesyncd and that kind of stuff. Maybe these are nice for desktop users who want a click and play experience on their laptop. No problem with that. But why do they have to break my server's for this?
I mean seriously: Linux on the desktop is irrelevant. Why optimize the whole OS/init for a irrelevant use case?
Boot time is a big deal for us. Not sure if it matters, but I don't get paid enough to decide what matters. All I know is we paid extra for some Dell software that lets you fine tune the BIOS and we streamline for boot speed. We're currently in the process of moving anything that can be run on a VM to a VM so this will be even less of a concern in the future and pid 1 will be more of a bottle neck. That said, except for the db servers, we don't run mem checks on boot so our servers are very fast to boot (I believe Dell cites that each Gb of RAM is equal to 1 sec. of boot time when you are running with the mem checker on).
The only time I ever do a "deep dive" and manual setup a server is when I have to provision a server for a new department/service. Generally speaking ansible does the work for me and I just hit go on my console and then go get a coffee. So I don't see much of a headache in server provisioning. I also am on the automation side of Ops so I do a lot more work with a already setup server, and in that case, like you said, systemd API is a godsend.
Don't Google's servers also run a custom systemd-less Debian derivative? https://www.usenix.org/node/177348
This list is missing a linux-ey system which doesn't use Systemd that is growing in popularity!
WSL!
WSL somewhat annoyingly has a custom/proprietary init system. Partially due to it's "interesting" integration with windows, and partially because it's not even really a full linux system yet.
I'm curious if with the switch to WSL2 where they will begin shipping a full linux kernel if systemd will be included or will they stick with their current custom solution?
I don't think WSL counts at all.
> WSL
I think you mean GNU/Windows, or as I've taken to calling it, GNU Plus Windows. (Seriously though, the WSL name is deceptive because it doesn't actually contain any Linux code. There is no Linux in "Windows Subsystem for Linux")
Well not today, but soon it will be GNU/Windows+Linux -- https://www.theverge.com/2019/5/6/18534687/microsoft-windows...
Soon there will be.
https://devblogs.microsoft.com/commandline/announcing-wsl-2/
WSL 2.0 contains a full Linux kernel.
Yeah but as far as I can tell that doesn't actually exist yet, and isn't scheduled to be available until the end of next month. (Existing inside Microsoft but not publicly is "does not exist" as far as I'm concerned, because I remember all kinds of legendary stuff Microsoft has apparently made but never released.)
I went on angry binge where I Ubuntu finally pushed me over the edge with it's latest... och. Between that and systemd, I've starting to dislike Linux, pretty much for the first time since I started using Linux back in the early 90's.
The second real, functional, genuinely working docker support shows up in *BSD and is working - I'll be running it on my desktop.
I do find it funny that people crap all over systemd, but don't seem to have a huge problem with launchd, despite the fact that systemd was almost a direct port of it.
Personally, while I do think that systemd's scope might have been a bit too ambitious, I do find it easier than something like Upstart
systemd was not a port of launchd, direct or otherwise. The problem with launchd being difficult to port off MacOS is one of the reasons why launchd never came about for FreeBSD.
* http://jdebp.uk./FGA/launchd-on-bsd.html
systemd was not a port of anything. And its immediate inspiration was Upstart, the system that Fedora was using at the time, and whose drawbacks provoked Lennart Poettering et al. to start working on systemd.
(Interestingly, Upstart itself was inspired by the drawbacks of porting launchd.)
> systemd was not a port of anything. And its immediate inspiration was Upstart
Its motivation was perhaps upstart's shortcomings, but systemd was definitely inspired by the design of launchd and that was repeatedly and explicitly mentioned in Lennart's very first blog post introducing it:
> But first, let's clear a few things up: is this kind of logic new? No, it certainly is not. The most prominent system that works like this is Apple's launchd system ...
> For socket activation we support full compatibility with the traditional inetd modes, as well as a very simple mode that tries to mimic launchd socket activation and is recommended for new services.
> More importantly however, it is also our plan to experiment with systemd … to make it the ideal session manager… The problem set of a session manager and an init system are very similar… Using the same code for both uses hence suggests itself. Apple recognized that and does just that with launchd. And so should we…
> Writing Daemons:: … The list above is very similar to what Apple recommends for daemons compatible with launchd. It should be easy to extend daemons that already support launchd activation to support systemd activation as well.
http://0pointer.de/blog/projects/systemd.html
No, that's not explicit. That's a few instances of "like launchd". Where M. Poettering actually is explicit about inspiration, xe writes the words:
> we took a lot of inspiration from Upstart's code-base
... which they actually did and continued to do, from observation. There were significant discussions amongst the systemd people involving the way that things worked with Upstart, including (for just three examples out of the many over the years) having to work like Upstart with respect to putting the X server on the first KVT, Upstart's handling of SIGPWR to process #1, and eliminating /dev/initctl because it was not really utilized in Upstart.
* https://news.ycombinator.com/item?id=12772915
* https://lists.freedesktop.org/archives/systemd-devel/2016-Ju...
* https://lists.freedesktop.org/archives/systemd-devel/2015-Se...
In comparison, there was almost zero similar consideration of launchd. The idea that launchd inspired systemd is a myth, not borne out by what M. Poettering explicitly wrote was the inspiration nor borne out by the evidence of what the systemd people did down the years.
I think we're going to have to agree to disagree about what "explicit" means, and possibly also "inspire." (Based on the excerpts quoted verbatim from Lennart's blog post in my previous comment and general understanding of the English language.)
Ah, you may be right, that's what I get for trying to remember a talk without double-checking. I thought I remembered that systemd was a port of launchd here [1]. My mistake!
[1] https://www.youtube.com/watch?v=o_AIw9bGogo&t=2178s
If you use a Mac, you don’t have much choice about your init system and probably don’t get into idealogical arguments about them as often as Linux users do.
Plus, there’s always the BSDs, which don’t use systemd but still feel very clean and cohesive.
I also think upstart was a bit nuts... but how do you think about openrc vs systemd?
FreeBSD uses OpenRC right?
I remember liking how easy it was to set up startup scripts in FreeBSD, so I think I like it? I'll admit I haven't used it as much as systemd or Upstart, so I can't give a strong opinion.
OpenRC isn’t the default but is portable to FreeBSD and NetBSD.
See https://www.freebsd.org/cgi/man.cgi?init
TrueOS (a FreeBSD-based project) uses OpenRC by default.
> I do find it funny that people crap all over systemd, but don't seem to have a huge problem with launchd
Because I've never been forced to use launchd. Well, you could say that I've never been forced to use systemd either but that's kinda how it feels when distros swap it out for you and things start depending on it.
Guess what? I'd be furious if the distros I use forced me onto launchd all of a sudden.
See also the search filter on Distrowatch for non-systemd distros: https://distrowatch.com/search.php?defaultinit=Not+systemd&s...
AntiX Linux is my current favorite which happens to be systemd free and runs with lighter weight DEs. It takes up only about 500MB of RAM sitting still, so pretty great for older systems and laptops.
I'm a big fan of systemd.
Comprehensive, well organized, reliable, powerful.
It makes a consistent service from what was an evolutionary hotchpotch of stuff.
Whatever it is that the haters hate, I don't get.
NixOS is one of the best distros there is and it uses systemd. Do I interact much with systemd myself? No, I don't even though I don't mind.
Is there a Linux distro which I can install on my Mac and where I don't have to touch the terminal ever?
Don’t forget one of my favorite init projects! LinuxBBQ!
I'm still waiting for it to eat dbus.
I think PCLinuxOS is still systemd-free.
Where is ChromeOS?!
We absolutely love Alpine, but it's been hard to integrate with other systems because of it using musl. For instance, our dns resolvers do not work properly & our monitoring solution is incompatible with musl.
So, how are any of you systemd apologists defending this little gem?
https://twitter.com/MartinCracauer/status/109470986305220199...
I would also like to know what the rationale is for this behavior. Showing immediate and useful feedback on the screen seems an obvious thing to do, especially when running a command at an interactive terminal.
I recommend watching this to give you some perspective on the SystemD hate: https://www.youtube.com/watch?v=o_AIw9bGogo
I too was furious about the existence of systemd, but now I realize that was mostly because I don't want new things and consider many new things to be bad because they are new and there have been a few 'bad apples' in my personal experience
If you are still arguing about systemd, you don't know enough about it, so you should learn about it and you don't understand the big picture. Users crying about Linux desktop all the time, but systemd has a huge part making the situation way better.
> Users crying about Linux desktop all the time, but systemd has a huge part making the situation way better.
Definitely. Maybe it's not great that GNOME has a hard dependency on systemd ( I personally don't care), but most people don't appreciate how much things have improved on the desktop with systemd.
I've been using linux since kernel 2.2. Deployed hundreds if not thousands of workstations for home users and developers. Trust me, I've seen desktop ups and downs, and I disagree. Systemd works until it doesn't, and then it's an unfixable lost cause and you reload it like a naughty 9x box.
Yeah, same for me. Since about 2.2. And I remember patching init.d scripts on Debian, adding sleep 5 In the network script, because you know our service had to have eth0 up and configured. So it's not like the old init system was perfect either :)
Or when I was building VM appliance with dB inside, and I was reading the news about systemd and how it would allow cpu and ram limits per service and thinking how if I had it it would save me from weeks worth of work (yes IBM DB2 is total crap when you run it in VM). and so on.
When I look back of all the stupid things I had to take care of and in systemd are just a config away, I can't help but wonder why people dislike it so much.
You were able to do those hacks because it was open to you at run time. You COULD fix it. Now if it breaks and there's no fix, you may as well have used boxed software because your options are only to wait for a fix or find something else. Honestly, I'd have very little problem with systemd if it was not written in a compiled language. Losing that flexibility sucks.
In another comment I noted that this is obviously a divide between the hobbyists and the 9-5 linux users. I am both, so this is particularly sucky.
If this is what it takes, it's not worth it.