Use the most radically copyleft and freedom preserving license you can. If the corporations want your software, you present a business solution: pay for special licensing conditions.
It's even blessed by Stallman. I emailed him to confirm. Unlike permissive licenses, only the original copyright holders get to benefit in this way. Others don't have this relicensing permission. The damage is contained.
I hope it works out for him. Watching beggar barons make billions off of free software that's being maintained for free is really hard to watch.
My friend sent me that article a few months ago. It _completely_ changed my approach to OSS contribution: from a 25yr MIT/BSD adherent to AGPL adherent in 30min.
> 1- Is this solution useful for subscription-based contract too?
If you mean SaaS, then maybe. I emailed Stallman about the ethics of the SaaS case and he said it's a net good.
You might want to think about whether the license actually gives you leverage in that case though. You might find that the corporations are perfectly willing to host a service using your AGPLv3 software. That's within their rights.
You only gain leverage if they want to create a proprietary version of your software.
> 2- Does it make a difference if the product is a app, library or hardware device?
Absolutely. The GPL has very specific wording with regards to linking and distribution which trigger license conditions. You should read the full license for a better understanding.
Hardware is a completely different matter, I won't even pretend to know anything about how licensing works in that case.
Remember, I'm not a lawyer. I'm just a hobbyist free software developer who's also trying his best to understand all this and make the best possible decision.
The main problem is that you need to have contributors sign a copyright assignment/CLA, otherwise their code is going to be AGPL only and you cannot license it commercially.
Or you don't have any contributors, which is the base case, I guess.
And, you'll trigger the same response in potential contributors: There's a generally anti-CLA attitude in open-source/free software circles because it means if you contribute your contributions can be used to enrich someone else.
The RedHat apparatchik at the top of the link is amazing. Attacking others from his secure position and celebrating more corporate influence:
"Good news is several Google and Apple engineers have volunteered to help with libxml2 and libxslt security issues, despite your effort to sabotage libxml2 users -- especially web browser users -- by disclosing all vulnerabilities immediately rather than allowing them the industry-standard 90 day disclosure deadline used by all other GNOME projects (#913 (closed)). They've posted a couple patches in the libxslt issue tracker already. I assume you're not satisfied with this, and are now trying to push them away. If that's your goal, you'll no doubt succeed pretty quickly."
RedHat often has a detrimental effect on open source, it is filled with bureaucrats and careerists.
Thanks Nick Wellnhofer for going AGPL. You are setting a great example!
This should be higher up and seems very relevant to understanding what's going on. Looks like the (former) maintainer does not actually want to abandon libxml2.
The bit about google employees willing to step up and work on the project is kind of disgusting: The company is too cheap to fund the project with money, but don't worry, they've got an endless supply of eager developers looking to 'pad their résumé' and deny you all the reward and satisfaction of what you built. It almost reads like a threat "work for free or we'll fork it and take it away from you". Wow
The maintainer's leverage is severely diminished due to the fact MIT licensed versions of the software exist. Only new code will be copylefted so it will be easier to cut him out of the picture.
This is why developers should AGPLv3 their personal projects from day one. Then others can't fork it under another license.
Even if they choose AGPLv3, the creator still maintains full freedom since they own the copyrights. They can make a commercial version if they want to. They can even relicense it under favorable terms to companies for a licensing fee. Everyone else must abide by the copyleft rules.
If they don't like it, let them pay hundreds of thousands of dollars a year for their own developers to make their own in house proprietary version.
> Even if they choose AGPLv3, the creator still maintains full freedom since they own the copyrights.
Only if they either refuse all contributions, require contributions to be made under an MIT license or similar (and then immediately relicense back to AGPLv3 before publishing), or require a CLA.
I'm all for personal projects to be licensed AGPLv3, but we must acknowledge that the moment you take others' AGPLv3 contributions, in practice you won't be able to do those other things.
Of course. The creator retains copyright. If there are multiple creators, things get complicated quickly. This is also why GNU projects ask that contributors assign copyright to the FSF. Gotta deal with this sort of bureaucracy before contributing is allowed.
Is there a free-for-corporate use under XX revenue or limited time license one can pack with an AGPLv3 project? Something that doesn't block people, even businesses, from using the project during the growth stage but also doesn't give away all your rights or ability to get paid later?
The copyright holder is free to grant use under any license they like to whoever they like.
It's rare but I've seen a number of projects over the years that have a hard copyleft license along with a line in the readme like "Want to use this with a different license? Send me an email and we'll sort it out"
There's gotta be a way to do this ethically, naturally and organically. I want people to engage with my projects too but I don't really want to sell my soul for it.
I hate advertising so I don't even post about my projects anywhere unless some very specific conditions are met. People found and shared my projects anyway. They've made it to the front page of HN. I even gained a GitHub sponsor because of that. Not enough to turn my hobby into full time work but still awesome.
Wow, indeed. To add insult to the injury, the whole message reads like this:
> Good news is several Google and Apple engineers have volunteered to help with libxml2 and libxslt security issues, despite your effort to sabotage libxml2 users...
I mean, c'mon. He's carrying the world on his shoulders and people are just pointing fingers?
Also, this shows how evil corporations are. I can understand Apple, it's their culture to avoid GPL code and and committing code to any public project needs permission from everyone plus the campus cat, but Google, the apparently bastion of open source software is doing the same thing without any shame...
I think mixing altruism with work is a mistake, which is the sentiment I read into the (sarcastic) comment. The maintainer has very little leverage for payment if they continue working on the project for free.
The maintainer has to pick a side and commit to it, and deal with the downsides. Alternatively, they may choose not to play.
I don't think he should be forced to pick sides. He made the thing, he knows the code base inside out. It would have been trivial for companies to hire him as a consultant or something since they're all depending on him. Why didn't they? It really makes no sense.
Still, it looks like he maintained the library for a long time. He no doubt has more knowledge about the code base than outsiders. That ought to be valuable to corporations relying on the library and contributing security patches.
Why buy the cow if the milk is free? The license let's them use it without payment, and in a just world, they'd pay all the maintainers of libraries they use, but ours isn't a just world, and we need to formulate our strategies with that in mind.
Because he continues to work for free? Companies are amoral actors. They aren’t going to donate out of charity and if someone wants to give them free work they won’t say no
Because according to the Gitlab comment you've taken that excerpt from, companies are so allergic to the GPL that they will not only avoid using the GPL version of libxml2, no. They will maintain a MIT fork of libxml2 out of spite, even though they've been actively running away from their responsibility to do so.
Let them maintain it. We can take their permissive code and add it to our AGPLv3 projects literally no questions asked, we just have to replicate the copyright notice and license file. Meanwhile they can't take our copyleft code without also copylefting any improvements they distribute. And with AGPLv3, network use is distribution.
If you think you need libxml2, think again. XML is a complex beast. Do you really need all those features? Maybe a much smaller, more easily maintained library would suit your needs while performing better at the same time!
For instance, consuming XML and creating it are two very different use cases. Zooming into consuming it, perhaps your input data has more guarantees than libxml2 assumes, such as the nonexistence of meta definition tags.
But different "yous" need different features, and so they get all glommed together into one big thing. So no one needs "all" of lbxml2/XML's features, each individual needs a different subset.
It's the same as the old joke about Microsoft Word: people only use 10% of Word's functionality, but the problem is each person uses a different 10%.
Of course this is an oversimplification, and there will no doubt be some sort of long tail, but it expresses the challenge well. I'd imagine the same is true for many other reasonably complex libraries, frameworks, or applications.
XML without DTDs is a very reasonable subset that eliminates significant complexity (no need for an HTTP client!) and security risks (no custom character entities that are infinitely recursive or read /etc/passwd!) and would probably still work for >80% of users.
Why throw out numbers when we all know you haven't actually measured that it's >80%?
In any case, the tooling around XML (DTDs, XPath, XSLT, etc.) is the reason to use it. I would go so far as to say the (supposed) >80% not using those features shouldn't have used XML in the first place.
Pretty sure beautifulsoup uses python’s builtin html.parser but can optionally use html5lib or lxml if installed, and it is lxml, not beautifulsoup, that actually depends on libxml2.
I kinda want something which just treats XML as a dumb tree definition language... give me elements with attributes as string key/value pairs, and children as an array of elements. And have a serialiser in there as well, it shouldn't hurt.
Basically something behaves like your typical JSON parser and serialiser but for XML.
To my knowledge, this is what TinyXML2 does, and I've used TinyXML2 for this before to great effect.
That's what you call a DOM Parser - the problem with them is, as they serialize all the elements into objects, bigger XML files tend to eat up all of your RAM. And this is where SAX2 parsers come into play where you define tree based callbacks to process the data.
Depending on the XML structure and the servers RAM - it can already happen while you approach 80-100 MB file sizes. And to be fair, in the Enterprise context, you are quite often not in a position to decide how big the export of another system is. But yes, back in 2010 we built preprocessing systems that checked XMLs and split them up in smaller chunks if they exceeded a certain size.
1. "This XML library is way bigger than what I need, I'll write something more minimal for my use case"
2. write a library for whatever minimal subset you need
3. crash report comes in, realise you missed off some feature x. Add support for some feature x.
4. Bob likes your library. So small, so elegant. He'd love to use it, if only you supported feature y, so you add support for feature y.
...
End result is x+1 big, complex XML libraries.
Obviously Im being a bit obtuse here because you might be able to guarantee some subset of it in whatever your specific circumstances are, but I think it's hard to do over a long period of time. If people think you're speaking XML then at some point they'll say "why don't we use this nice XML feature to add this new functionality".
XML is used in countless standards. You can't just not use it if you interact with the outside world. Every XML feature is still in the many XML libraries because someone has a need for it, even things like external entities.
Maybe you don't need libxml2 specifically (good luck finding an alternative to parse XML in C and other such languages though), but "I don't like the complex side of XML so let's pretend it doesn't exist" doesn't solve the problem most people pick libxml2 for. It's the de-facto standard because it supports everything you could possibly need.
Exactly. For example if you need to integrate SAML, you have to support a significant subset of several XML specs. It may be possible to write a SAML-only library that supports less, but it's not clear it would be any simpler.
> <blink>Expat is UNDERSTAFFED and WITHOUT FUNDING.</blink>
> The following topics need additional skilled C developers to progress
> in a timely manner or at all (loosely ordered by descending priority):
Its a shame that xslt seems to be struggling so much at the moment. If xslt 3 support was fully implemented in libxml2 (and therefore xsltproc and browsers) then it would be by far the most sensible option for designing anything to do with getting text onto the web.
* XSLT is still the only native templating option for HTML pages that runs natively in the browser (but just now you are limited to XSLT v1.0 which as a number of drawbacks and limitations)
* XSLT/XML is still best at text markup. In particular interpolation. There is no simple way to represent marked up text in, say, JSON.
* Content federation (atom, rss) is still very dependent on XML.
Surely somebody somewhere has money to pay for a greybeard to fix XSLT for us? It seems far to fundamental to be left to wither on the vine.
Rather than struggling/withering, it's actively being killed. Efforts are underway to completely remove XSLT support from browsers, due to the poor state of libxml2 and a lack of any new maintainer stepping up.
Right, but AKAIK its _still_ being maintained on a voluntary basis. Thats nuts, and its not clear why, say, Chrome or Firefox wouldn't want to take over XSLT/libsml2 development, particularly if they won market share from stuff like React, and created a developer acquisition pipeline for their respective ecosystems.
> its not clear why, say, Chrome or Firefox wouldn't want to take over XSLT/libxml2 development
Very few people actually like XSLT, presumably including the Chrome and Firefox devs. I know XSLT has its share of supporters and that's fine. I'm not here to argue to merits of XSLT – or lack thereof – but we need to be honest about this. They are the proverbial Black Metal fans; everyone else just thinks it's bloody noise.
In addition, many people have grown towards the idea that importing these large C libraries for little used features is just not a good idea in the first place. And that makes libxml and libxslt a dead end. The entire business with XSLT was kicked off by a bunch of security bugs.
Finally, I think a decent case can be made to slim down the "web platform" a bit. If you want XSLT you can still "bring it yourself", but does every browser need to implement it to be "standards compliant"? Seems like a bad trade-off to me. It's a win for newer browsers like Servo or Ladybird if they don't need to worry about XSLT.
So in short, it's not just a problem of "adding some more people to libxml", although obviously that is a problem.
> a decent case can be made to slim down the "web platform" a bit. If you want XSLT you can still "bring it yourself"
Completely agree, but there remains the massive unsolved issue of templating in HTML (at the moment XSLT is the only way to run a templated HTML website on s3 without a massive pile of javascript).
Just for the sake of example I'm one of the people who would be a candidate to take over development, but same as anyone else I have little desire to be stuck maintaining unnecessarily complex functionality. Instead I've chosen to work on reimplementing XML and its utilities from the ground up without making the same errors that made XML so miserable to users and implementers both...
They don’t want to because they don’t see any bright future for the technology even if it’s better maintained. XML/XSLT isn’t trendy anymore, nobody is building new apps on it. It is never going to win market share from react - it’s too baroque and dated.
> It is never going to win market share from react
Not sure what React has anything to do with this.
XSLT was pretty much never used as a rendering platform but for XML-data processing.
As JSON became the standard of API communication in early 2000s (less powerful, but also much less verbose and easier to manipulate in JS) XSLT became less relevant.
Parent comment mentioned react, I can sorta see the comparison if you squint. But I disagree with their assertion that a better maintained XSLT could compete.
In any case, I'm not sure I agree with you, while JavaScript and CSS are composable out of the box HTML really lacks a native, fully declarative, composable way to build documents.
I feel like it adds more weight to my feeling that we should have a software building code. When you have software that's critical infrastructure, with a nutso security policy like "no embargoes / 0day me bruh", we should have some regulations in place to require the software be maintained properly (that is to say, in a sane manner) or you can't use it commercially or for safety-critical things. Which would inevitably force commercial entities to pay for the maintenance so it could be done right.... which they should be doing already, the same way any company that builds safety-critical infrastructure has to pay to do it right.
If we want society to be safe, we have to make a law that enforces it. That's how that shit works.
(as an aside: holy shit, you're a prolific HN submitter, and all from different sources. where do you get it all?)
This made my brain go
"Oh no, not this again. Open source projects don't owe you..." etc etc.
> or you can't use it commercially or for safety-critical things
Oh. Yeah, okay, absolutely! For safety-critical, I would like to think the responsibility already lies with the integrator/seller, but making it explicitly so can't hurt.
> or you can't use it commercially or for safety-critical things
The license for libxml2 (like the license for almost any kind of open source software) already states "THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT." I don't see how you can put the responsibility even more on the integrator/seller than that. It literally states the devs don't even guarantee it works correctly.
Safety critical fields like aviation already have strict requirements. Usually there's very few software dependencies used in those projects.
Expanding that to more fields would be interesting, but difficult and expensive across the board. Particularly any sort of requirements like that generally incur significant regulatory and certification overhead.
However, if it was done similar to PCISS as an industry forum it might work better. Especially if certain fields like anything connecting with the electric grid we're required to use certified software.
Pretty much all construction uses materials which follow a specification. The least we could do is start requiring all commercial software do the following:
1. Declare an SBOM
2. Each software component must have a listed specification
We'd then need to make software specifications. Start with the most basic specification possible; "has performed linting", "has full integration test coverage", "has passed QA testing", "has an active maintainer", "lists its license", "does not have a hidden back door", "is free of known vulnerabilities", etc. Make more detailed specifications as-needed (for a particular industry, use case, requirements).
Once we have all that, you can glance at a company's SBOM and find out if they've done the bare minimum due-diligence. We could also make or modify regulations that require these same materials standards, like privacy regulations, financial regulations.
And yes, meeting minimum material standards is more expensive. We already accept that cost in the physical world, why not in the software world? If there's a TDS, SDS, MSDS, etc for physical products, we should have them for software too. I want to know your materials are safe before I use your products. I'm sick of being exposed by companies who are completely irresponsible.
Isn't this what the european Cyber Resilience Act (CRA) is about?
See https://orcwg.org/cra/ and the work of the Open Regulatory Compliance Working Group in general.
More or less, though the CRA is pretty minimal: it has a few basic requirements and hobby/unpaid open source software is not covered. A company integrating open source software is essentially responsible for covering those requirements themselves.
The company being responsible for the open source components they integrate should solve the biggest dependency problems, though. From a security perspective, it doesn't really matter if a company fixes the bugs themselves or if they pay someone to fix it for them.
> When you have software that's critical infrastructure, with a nutso security policy like "no embargoes / 0day me bruh",
...you then should stop and re-evaluate your life choices: specifically, choosing this particular piece of software, which is known to have always been insecure, to be a critical part of your infrastructure.
People building "safety critical" systems already pay for a "secure" ecosystem. It's called Microsoft. We don't need regulations to have Microsoft exist. Do you think some random med tech startup is going to pay to have libxml2 maintained? They'll see the regulation and go "oh ok, Windows licenses it is".
It's not the "safety critical" software that needs this fixed, it's all software in general. There's a million software systems that have important privacy sensitive data or safety relevant processes that fly under the "safety critical" radar.
Read your Microsoft licensing agreement. If you don't have one, read the EULA for OEM windows. The warranty, fitness for purpose and damages exclusion is not as extensive as what the grandparent cited, but it basically boils down to "as limited as legally possible, and the most damages you will get is your license fee back". You also won't get a binding requirements document anyways, so you don't even really know what the software microsoft sells you is fit for. At any point in time, there could be some knowledgebase article saying something like "oh, and btw, don't do this because it breaks", so per their warranty agreement you signed they are free from any responsibility simply by documenting the problem.
Really safety-critical stuff like ASIL-D, ISO26262, IEC61508 (and tons of other magic numbers) isn't something you can buy from microsoft. At best, you can sometimes get a reseller to sign something a little more binding, but with tons of restrictions that basically boil down to "use the microsoft stuff for the readout gauges, but the critical control part goes somewhere else".
It's not about warranties, it's about having a stable ecosystem with some guaranteed measure of maintenance. The point is not that there's even more stable and expensive options than Microsoft. The point is that there's very little space for OSS here. Go to any hospital and count the amount of Windows devices and compare that to the amount of other operating systems you see. The second something becomes even a little safety oriented, there's going to be proprietary software.
So when these regulations that OP would start to take hold, would we get companies to sponsor random open source dependencies like libxml2? Or would they gather around some stable proprietary ecosystem like Microsoft's and maybe some big innovative solutions built on top of Microsoft?
Even the "guaranteed measure of maintenance" is not guaranteed. You don't get an SLA on patches or bugfixes from microsoft. You don't get an uptime SLA. Its all "best effort" or worse "when we feel like it". And the few SLAs they give you, e.g. on cloud stuff, are useless because it basically is "get your money back for that month". And the SLA measurement is done by their own downtime announcements, so a complete joke. Software lifetimes exist and are published, but guess what? Within that lifetime, you get "updates", but nowhere do you get any kind of guarantee about what is updated, what is fixed, how fast, if ever.
And no kind of safety-oriented anything will run windows or any microsoft software. There is no windows edition of therac-25. The stuff you see in a hospital is normal workstation PCs for non-safety-relevant data entry and display. As soon as it becomes safety-relevant like controlling your heart-lung-machine, auto-dosing your medications, controlling the x-ray beam, you are far away from anything microsoft.
And actually, OSS is used more often in those safety-relevant settings. Why? Not because the OSS maintainers themselves would themselves provide any support, SLA or warranty. But because the nature of OSS provides third parties the possibility to certify, maintain and guarantee for their special 'safety-relevant-libxml2-fork'. Sometimes this is done by the device vendors themselves, sometimes they buy this from others. But it happens, and it is growing in frequency.
There is tons more. Basically any compiler for safety-relevant embedded stuff is either clang or gcc under the hood. Linux is frequently encountered when the real-time requirements aren't too strict. With Linux also comes the usual Linux ecosystem of OSS libs and services. It won't look like your normal desktop OS, but quite a lot in that area is OSS.
Nothing at all from microsoft (except a useless BS certification "you can use Azure Devops as a code repo to store you ASIL-D code...").
Nick Wellnhofer is stepping away from libxml2 after a decade of unpaid maintenance. He’s forking it under the AGPL, but that will probably scare off most corporate users.
Meanwhile libxml2 is still everywhere. Without someone with real backing, a core piece of infrastructure is about to go unmaintained.
Once again, the open-source funding problem is laid bare: the internet runs on the unpaid evenings of a few people until they burn out (add relevant reference from XKCD, obviously).
"Expected effort required to maintain implementation" should be an evaluation criteria when selecting technologies. Thousand page RFCs do not make sustainable standards in the long run. Most committee designed specs end up in this category. People are impressed by complexity but actively pursuing simplicity is what we should be doing.
With not enough time to develop an alternative and too many application ecosystems relying on this library, I think it's a matter of time before a large company forks the library to fix security issues with it now that they have no choice but to do the work themselves. At least until IBM and Google figure out a way to move away from this library.
Maybe my human interaction interfacing software has a glitch but I am having a hard time parsing this content. Do I detect a hint of sarcasm? Please add a '/s' at the end of your future posts to aid my very archaic and vintage brain matter.
Jia Tan was the alias of the hacker(s) who infiltrated xz to plant a backdoor. He/They were in the project for 2 years I believe, and so had "significant experience" "maintaining" open source software.
Related gitlab comment: https://gitlab.gnome.org/GNOME/libxml2/-/issues/976#note_253... .
> The basic idea is to offer a paid commercial license for people who don't want to use GPLed code.
This business model is known as selling exceptions to the GPL.
https://www.gnu.org/philosophy/selling-exceptions.html
Use the most radically copyleft and freedom preserving license you can. If the corporations want your software, you present a business solution: pay for special licensing conditions.
It's even blessed by Stallman. I emailed him to confirm. Unlike permissive licenses, only the original copyright holders get to benefit in this way. Others don't have this relicensing permission. The damage is contained.
I hope it works out for him. Watching beggar barons make billions off of free software that's being maintained for free is really hard to watch.
https://zedshaw.com/blog/2022-02-05-the-beggar-barons/
Personally what he's doing is very sensible, and how I personally advocate. Hope it works out for him.
I personally like the slow and steady tide of understanding the value of GPL family of licenses.
My friend sent me that article a few months ago. It _completely_ changed my approach to OSS contribution: from a 25yr MIT/BSD adherent to AGPL adherent in 30min.
It was very influential for me too. Here's another one, also from Zed Shaw:
https://web.archive.org/web/20120620103603/http://zedshaw.co...
> Why I (A/L)GPL
> Open source to open source, corporation to corporation.
> If you do open source, you’re my hero and I support you.
> If you’re a corporation, let’s talk business.
> I want people to appreciate the work I’ve done and the value of what I’ve made.
> Not pass on by waving “sucker” as they drive their fancy cars.
So if one wants to open-source his project and sell it :
- Licence as AGPL
- Mention that commercial use (without having to open source the derivative work) is available
Did I get it right?
1- Is this solution useful for subscription-based contract too?
2- Does it make a difference if the product is a app, library or hardware device?
> Did I get it right?
I think so.
> 1- Is this solution useful for subscription-based contract too?
If you mean SaaS, then maybe. I emailed Stallman about the ethics of the SaaS case and he said it's a net good.
You might want to think about whether the license actually gives you leverage in that case though. You might find that the corporations are perfectly willing to host a service using your AGPLv3 software. That's within their rights.
You only gain leverage if they want to create a proprietary version of your software.
> 2- Does it make a difference if the product is a app, library or hardware device?
Absolutely. The GPL has very specific wording with regards to linking and distribution which trigger license conditions. You should read the full license for a better understanding.
Hardware is a completely different matter, I won't even pretend to know anything about how licensing works in that case.
Remember, I'm not a lawyer. I'm just a hobbyist free software developer who's also trying his best to understand all this and make the best possible decision.
The main problem is that you need to have contributors sign a copyright assignment/CLA, otherwise their code is going to be AGPL only and you cannot license it commercially.
Or you don't have any contributors, which is the base case, I guess.
And, you'll trigger the same response in potential contributors: There's a generally anti-CLA attitude in open-source/free software circles because it means if you contribute your contributions can be used to enrich someone else.
The RedHat apparatchik at the top of the link is amazing. Attacking others from his secure position and celebrating more corporate influence:
"Good news is several Google and Apple engineers have volunteered to help with libxml2 and libxslt security issues, despite your effort to sabotage libxml2 users -- especially web browser users -- by disclosing all vulnerabilities immediately rather than allowing them the industry-standard 90 day disclosure deadline used by all other GNOME projects (#913 (closed)). They've posted a couple patches in the libxslt issue tracker already. I assume you're not satisfied with this, and are now trying to push them away. If that's your goal, you'll no doubt succeed pretty quickly."
RedHat often has a detrimental effect on open source, it is filled with bureaucrats and careerists.
Thanks Nick Wellnhofer for going AGPL. You are setting a great example!
This should be higher up and seems very relevant to understanding what's going on. Looks like the (former) maintainer does not actually want to abandon libxml2.
He just wants his sanity and dignity back while continuing his passion project. Respectable and commendable, if you ask me.
The bit about google employees willing to step up and work on the project is kind of disgusting: The company is too cheap to fund the project with money, but don't worry, they've got an endless supply of eager developers looking to 'pad their résumé' and deny you all the reward and satisfaction of what you built. It almost reads like a threat "work for free or we'll fork it and take it away from you". Wow
The maintainer's leverage is severely diminished due to the fact MIT licensed versions of the software exist. Only new code will be copylefted so it will be easier to cut him out of the picture.
This is why developers should AGPLv3 their personal projects from day one. Then others can't fork it under another license.
Even if they choose AGPLv3, the creator still maintains full freedom since they own the copyrights. They can make a commercial version if they want to. They can even relicense it under favorable terms to companies for a licensing fee. Everyone else must abide by the copyleft rules.
If they don't like it, let them pay hundreds of thousands of dollars a year for their own developers to make their own in house proprietary version.
> Even if they choose AGPLv3, the creator still maintains full freedom since they own the copyrights.
Only if they either refuse all contributions, require contributions to be made under an MIT license or similar (and then immediately relicense back to AGPLv3 before publishing), or require a CLA.
I'm all for personal projects to be licensed AGPLv3, but we must acknowledge that the moment you take others' AGPLv3 contributions, in practice you won't be able to do those other things.
Of course. The creator retains copyright. If there are multiple creators, things get complicated quickly. This is also why GNU projects ask that contributors assign copyright to the FSF. Gotta deal with this sort of bureaucracy before contributing is allowed.
> This is why developers should AGPLv3 their personal projects from day one.
That would be detrimental to "growth hacking" GitHub stars and gaining traction. One can't be paid without baiting users first.
Is there a free-for-corporate use under XX revenue or limited time license one can pack with an AGPLv3 project? Something that doesn't block people, even businesses, from using the project during the growth stage but also doesn't give away all your rights or ability to get paid later?
The copyright holder is free to grant use under any license they like to whoever they like.
It's rare but I've seen a number of projects over the years that have a hard copyleft license along with a line in the readme like "Want to use this with a different license? Send me an email and we'll sort it out"
There's gotta be a way to do this ethically, naturally and organically. I want people to engage with my projects too but I don't really want to sell my soul for it.
I hate advertising so I don't even post about my projects anywhere unless some very specific conditions are met. People found and shared my projects anyway. They've made it to the front page of HN. I even gained a GitHub sponsor because of that. Not enough to turn my hobby into full time work but still awesome.
Wow, indeed. To add insult to the injury, the whole message reads like this:
> Good news is several Google and Apple engineers have volunteered to help with libxml2 and libxslt security issues, despite your effort to sabotage libxml2 users...
I mean, c'mon. He's carrying the world on his shoulders and people are just pointing fingers?
Also, this shows how evil corporations are. I can understand Apple, it's their culture to avoid GPL code and and committing code to any public project needs permission from everyone plus the campus cat, but Google, the apparently bastion of open source software is doing the same thing without any shame...
They have morphed into the next Microsoft AFAICS.
Despicable.
mcatanzaro frequently makes these over the top comments, and when called out, refuses to backtrack on them.
What other over the top comments has Michael Catanzaro made?
The accusation of sabotage was pretty disgusting but there's something that's arguably even worse deeper in the thread:
> maybe if you stop fixing things for free, perhaps somebody will suddenly be willing to pay you to do so
We should all remember that line every time we think about being generous or altruistic. He essentially called the maintainer a fool.
I think mixing altruism with work is a mistake, which is the sentiment I read into the (sarcastic) comment. The maintainer has very little leverage for payment if they continue working on the project for free.
The maintainer has to pick a side and commit to it, and deal with the downsides. Alternatively, they may choose not to play.
I don't think he should be forced to pick sides. He made the thing, he knows the code base inside out. It would have been trivial for companies to hire him as a consultant or something since they're all depending on him. Why didn't they? It really makes no sense.
Nick Wellnhofer did not create libxml2 or libxslt, David Veillard did. Nick has been the primary contributor since about 2015, though.
My mistake, I apologize.
Still, it looks like he maintained the library for a long time. He no doubt has more knowledge about the code base than outsiders. That ought to be valuable to corporations relying on the library and contributing security patches.
> Why didn't they?
Why buy the cow if the milk is free? The license let's them use it without payment, and in a just world, they'd pay all the maintainers of libraries they use, but ours isn't a just world, and we need to formulate our strategies with that in mind.
> Why didn't they?
Because he continues to work for free? Companies are amoral actors. They aren’t going to donate out of charity and if someone wants to give them free work they won’t say no
Google does _not_ fix things even for money, so he may have a point. /s
I don't like the comments from Michael candaroza they feel entitled
I really hope the AGPL fork is called libxml3
> You are no doubt well aware that nobody will ever use a GPLv3 libxml2
Why wouldn't other FOSS projects like Gnome Web for example not use GPLv3 licensed software?
Because according to the Gitlab comment you've taken that excerpt from, companies are so allergic to the GPL that they will not only avoid using the GPL version of libxml2, no. They will maintain a MIT fork of libxml2 out of spite, even though they've been actively running away from their responsibility to do so.
Let them maintain it. We can take their permissive code and add it to our AGPLv3 projects literally no questions asked, we just have to replicate the copyright notice and license file. Meanwhile they can't take our copyleft code without also copylefting any improvements they distribute. And with AGPLv3, network use is distribution.
It's MIT-licensed, so they don't have to share their changes. That'll make it harder to take their changes.
Let them keep it. Refusing to share kills their open washing as well. Everyone will see them for what they are.
If you think you need libxml2, think again. XML is a complex beast. Do you really need all those features? Maybe a much smaller, more easily maintained library would suit your needs while performing better at the same time!
For instance, consuming XML and creating it are two very different use cases. Zooming into consuming it, perhaps your input data has more guarantees than libxml2 assumes, such as the nonexistence of meta definition tags.
> Do you really need all those features?
"You" probably do not.
But different "yous" need different features, and so they get all glommed together into one big thing. So no one needs "all" of lbxml2/XML's features, each individual needs a different subset.
It's the same as the old joke about Microsoft Word: people only use 10% of Word's functionality, but the problem is each person uses a different 10%.
Of course this is an oversimplification, and there will no doubt be some sort of long tail, but it expresses the challenge well. I'd imagine the same is true for many other reasonably complex libraries, frameworks, or applications.
XML without DTDs is a very reasonable subset that eliminates significant complexity (no need for an HTTP client!) and security risks (no custom character entities that are infinitely recursive or read /etc/passwd!) and would probably still work for >80% of users.
(I wrote such an XML parser a long time ago.)
Why throw out numbers when we all know you haven't actually measured that it's >80%?
In any case, the tooling around XML (DTDs, XPath, XSLT, etc.) is the reason to use it. I would go so far as to say the (supposed) >80% not using those features shouldn't have used XML in the first place.
I agree.. which is part of why I generally dislike using XML for most things.
Not to mention that libxml2 underlies things like nokogiri (the commonly used html parsing gem for ruby), beautifulsoup (python's equivalent), etc.
Pretty sure beautifulsoup uses python’s builtin html.parser but can optionally use html5lib or lxml if installed, and it is lxml, not beautifulsoup, that actually depends on libxml2.
You’re right about nokogiri, though.
Ah, you're right, in the codebase I'm familiar with lxml is used for performance, though it's not the default.
I kinda want something which just treats XML as a dumb tree definition language... give me elements with attributes as string key/value pairs, and children as an array of elements. And have a serialiser in there as well, it shouldn't hurt.
Basically something behaves like your typical JSON parser and serialiser but for XML.
To my knowledge, this is what TinyXML2 does, and I've used TinyXML2 for this before to great effect.
That's what you call a DOM Parser - the problem with them is, as they serialize all the elements into objects, bigger XML files tend to eat up all of your RAM. And this is where SAX2 parsers come into play where you define tree based callbacks to process the data.
The solution is simple: don't have XML files that are many gigabytes in size.
A lot of teleco stuff dumps multi-gb stuff of xml hourly. Per BTS. Processing few TB of XML files on one server daily
It's doable, just use the right tools and hacks :)
Processing schema-less or broken schema stuff is always hilarious.
Good times.
Lol I love the upbeat tone here. Helps me deal with my PTSD after working with XML files.
Depending on the XML structure and the servers RAM - it can already happen while you approach 80-100 MB file sizes. And to be fair, in the Enterprise context, you are quite often not in a position to decide how big the export of another system is. But yes, back in 2010 we built preprocessing systems that checked XMLs and split them up in smaller chunks if they exceeded a certain size.
Tell that to wikimedia, I've used libxml's SAX parser in the past to parse 80GB+ xml dumps.
Some formats are this and they are historical formats.
This process usually goes:
1. "This XML library is way bigger than what I need, I'll write something more minimal for my use case"
2. write a library for whatever minimal subset you need
3. crash report comes in, realise you missed off some feature x. Add support for some feature x.
4. Bob likes your library. So small, so elegant. He'd love to use it, if only you supported feature y, so you add support for feature y.
...
End result is x+1 big, complex XML libraries.
Obviously Im being a bit obtuse here because you might be able to guarantee some subset of it in whatever your specific circumstances are, but I think it's hard to do over a long period of time. If people think you're speaking XML then at some point they'll say "why don't we use this nice XML feature to add this new functionality".
If you want to read some XML quickly, there's always RapidXML and PugiXML, but if you need a big gun, there's libXML.
The former are blazingly fast. In real world they can parse instantly. So alternatives do exist for different use cases.
> Obviously Im being a bit obtuse here
No. This is the first good expkanation for the library hell in linux those days.
XML is used in countless standards. You can't just not use it if you interact with the outside world. Every XML feature is still in the many XML libraries because someone has a need for it, even things like external entities.
Maybe you don't need libxml2 specifically (good luck finding an alternative to parse XML in C and other such languages though), but "I don't like the complex side of XML so let's pretend it doesn't exist" doesn't solve the problem most people pick libxml2 for. It's the de-facto standard because it supports everything you could possibly need.
Exactly. For example if you need to integrate SAML, you have to support a significant subset of several XML specs. It may be possible to write a SAML-only library that supports less, but it's not clear it would be any simpler.
You shouldn't be down voted, its just the truth no matter how unfortunate.
There is always libexpat, which works very well, also for the streaming case.
Expat is suffering from similar problems: https://github.com/libexpat/libexpat/blob/7643f96bd5b9f5d3b2...
> <blink>Expat is UNDERSTAFFED and WITHOUT FUNDING.</blink> > The following topics need additional skilled C developers to progress > in a timely manner or at all (loosely ordered by descending priority):
Yep, another case of XKCD 2347, unfortunately.
Gratuitous use of XML does sometimes smell like a "now you have two problems" kind of affair.
Its a shame that xslt seems to be struggling so much at the moment. If xslt 3 support was fully implemented in libxml2 (and therefore xsltproc and browsers) then it would be by far the most sensible option for designing anything to do with getting text onto the web.
* XSLT is still the only native templating option for HTML pages that runs natively in the browser (but just now you are limited to XSLT v1.0 which as a number of drawbacks and limitations)
* XSLT/XML is still best at text markup. In particular interpolation. There is no simple way to represent marked up text in, say, JSON.
* Content federation (atom, rss) is still very dependent on XML.
Surely somebody somewhere has money to pay for a greybeard to fix XSLT for us? It seems far to fundamental to be left to wither on the vine.
Rather than struggling/withering, it's actively being killed. Efforts are underway to completely remove XSLT support from browsers, due to the poor state of libxml2 and a lack of any new maintainer stepping up.
Right, but AKAIK its _still_ being maintained on a voluntary basis. Thats nuts, and its not clear why, say, Chrome or Firefox wouldn't want to take over XSLT/libsml2 development, particularly if they won market share from stuff like React, and created a developer acquisition pipeline for their respective ecosystems.
> its not clear why, say, Chrome or Firefox wouldn't want to take over XSLT/libxml2 development
Very few people actually like XSLT, presumably including the Chrome and Firefox devs. I know XSLT has its share of supporters and that's fine. I'm not here to argue to merits of XSLT – or lack thereof – but we need to be honest about this. They are the proverbial Black Metal fans; everyone else just thinks it's bloody noise.
In addition, many people have grown towards the idea that importing these large C libraries for little used features is just not a good idea in the first place. And that makes libxml and libxslt a dead end. The entire business with XSLT was kicked off by a bunch of security bugs.
Finally, I think a decent case can be made to slim down the "web platform" a bit. If you want XSLT you can still "bring it yourself", but does every browser need to implement it to be "standards compliant"? Seems like a bad trade-off to me. It's a win for newer browsers like Servo or Ladybird if they don't need to worry about XSLT.
So in short, it's not just a problem of "adding some more people to libxml", although obviously that is a problem.
> a decent case can be made to slim down the "web platform" a bit. If you want XSLT you can still "bring it yourself"
Completely agree, but there remains the massive unsolved issue of templating in HTML (at the moment XSLT is the only way to run a templated HTML website on s3 without a massive pile of javascript).
Just for the sake of example I'm one of the people who would be a candidate to take over development, but same as anyone else I have little desire to be stuck maintaining unnecessarily complex functionality. Instead I've chosen to work on reimplementing XML and its utilities from the ground up without making the same errors that made XML so miserable to users and implementers both...
They don’t want to because they don’t see any bright future for the technology even if it’s better maintained. XML/XSLT isn’t trendy anymore, nobody is building new apps on it. It is never going to win market share from react - it’s too baroque and dated.
> It is never going to win market share from react
Not sure what React has anything to do with this.
XSLT was pretty much never used as a rendering platform but for XML-data processing.
As JSON became the standard of API communication in early 2000s (less powerful, but also much less verbose and easier to manipulate in JS) XSLT became less relevant.
Parent comment mentioned react, I can sorta see the comparison if you squint. But I disagree with their assertion that a better maintained XSLT could compete.
I missed it.
In any case, I'm not sure I agree with you, while JavaScript and CSS are composable out of the box HTML really lacks a native, fully declarative, composable way to build documents.
Related Libxml2's "no security embargoes" policy (298 points, 84 days ago, 270 comments) https://news.ycombinator.com/item?id=44381093
Thanks, this is really interesting.
I feel like it adds more weight to my feeling that we should have a software building code. When you have software that's critical infrastructure, with a nutso security policy like "no embargoes / 0day me bruh", we should have some regulations in place to require the software be maintained properly (that is to say, in a sane manner) or you can't use it commercially or for safety-critical things. Which would inevitably force commercial entities to pay for the maintenance so it could be done right.... which they should be doing already, the same way any company that builds safety-critical infrastructure has to pay to do it right.
If we want society to be safe, we have to make a law that enforces it. That's how that shit works.
(as an aside: holy shit, you're a prolific HN submitter, and all from different sources. where do you get it all?)
> we should have a software building code
This made my brain go "Oh no, not this again. Open source projects don't owe you..." etc etc.
> or you can't use it commercially or for safety-critical things
Oh. Yeah, okay, absolutely! For safety-critical, I would like to think the responsibility already lies with the integrator/seller, but making it explicitly so can't hurt.
> or you can't use it commercially or for safety-critical things
The license for libxml2 (like the license for almost any kind of open source software) already states "THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT." I don't see how you can put the responsibility even more on the integrator/seller than that. It literally states the devs don't even guarantee it works correctly.
Safety critical fields like aviation already have strict requirements. Usually there's very few software dependencies used in those projects.
Expanding that to more fields would be interesting, but difficult and expensive across the board. Particularly any sort of requirements like that generally incur significant regulatory and certification overhead.
However, if it was done similar to PCISS as an industry forum it might work better. Especially if certain fields like anything connecting with the electric grid we're required to use certified software.
Pretty much all construction uses materials which follow a specification. The least we could do is start requiring all commercial software do the following:
We'd then need to make software specifications. Start with the most basic specification possible; "has performed linting", "has full integration test coverage", "has passed QA testing", "has an active maintainer", "lists its license", "does not have a hidden back door", "is free of known vulnerabilities", etc. Make more detailed specifications as-needed (for a particular industry, use case, requirements).Once we have all that, you can glance at a company's SBOM and find out if they've done the bare minimum due-diligence. We could also make or modify regulations that require these same materials standards, like privacy regulations, financial regulations.
And yes, meeting minimum material standards is more expensive. We already accept that cost in the physical world, why not in the software world? If there's a TDS, SDS, MSDS, etc for physical products, we should have them for software too. I want to know your materials are safe before I use your products. I'm sick of being exposed by companies who are completely irresponsible.
Isn't this what the european Cyber Resilience Act (CRA) is about? See https://orcwg.org/cra/ and the work of the Open Regulatory Compliance Working Group in general.
More or less, though the CRA is pretty minimal: it has a few basic requirements and hobby/unpaid open source software is not covered. A company integrating open source software is essentially responsible for covering those requirements themselves.
The company being responsible for the open source components they integrate should solve the biggest dependency problems, though. From a security perspective, it doesn't really matter if a company fixes the bugs themselves or if they pay someone to fix it for them.
> When you have software that's critical infrastructure, with a nutso security policy like "no embargoes / 0day me bruh",
...you then should stop and re-evaluate your life choices: specifically, choosing this particular piece of software, which is known to have always been insecure, to be a critical part of your infrastructure.
People building "safety critical" systems already pay for a "secure" ecosystem. It's called Microsoft. We don't need regulations to have Microsoft exist. Do you think some random med tech startup is going to pay to have libxml2 maintained? They'll see the regulation and go "oh ok, Windows licenses it is".
It's not the "safety critical" software that needs this fixed, it's all software in general. There's a million software systems that have important privacy sensitive data or safety relevant processes that fly under the "safety critical" radar.
Read your Microsoft licensing agreement. If you don't have one, read the EULA for OEM windows. The warranty, fitness for purpose and damages exclusion is not as extensive as what the grandparent cited, but it basically boils down to "as limited as legally possible, and the most damages you will get is your license fee back". You also won't get a binding requirements document anyways, so you don't even really know what the software microsoft sells you is fit for. At any point in time, there could be some knowledgebase article saying something like "oh, and btw, don't do this because it breaks", so per their warranty agreement you signed they are free from any responsibility simply by documenting the problem.
Really safety-critical stuff like ASIL-D, ISO26262, IEC61508 (and tons of other magic numbers) isn't something you can buy from microsoft. At best, you can sometimes get a reseller to sign something a little more binding, but with tons of restrictions that basically boil down to "use the microsoft stuff for the readout gauges, but the critical control part goes somewhere else".
It's not about warranties, it's about having a stable ecosystem with some guaranteed measure of maintenance. The point is not that there's even more stable and expensive options than Microsoft. The point is that there's very little space for OSS here. Go to any hospital and count the amount of Windows devices and compare that to the amount of other operating systems you see. The second something becomes even a little safety oriented, there's going to be proprietary software.
So when these regulations that OP would start to take hold, would we get companies to sponsor random open source dependencies like libxml2? Or would they gather around some stable proprietary ecosystem like Microsoft's and maybe some big innovative solutions built on top of Microsoft?
Even the "guaranteed measure of maintenance" is not guaranteed. You don't get an SLA on patches or bugfixes from microsoft. You don't get an uptime SLA. Its all "best effort" or worse "when we feel like it". And the few SLAs they give you, e.g. on cloud stuff, are useless because it basically is "get your money back for that month". And the SLA measurement is done by their own downtime announcements, so a complete joke. Software lifetimes exist and are published, but guess what? Within that lifetime, you get "updates", but nowhere do you get any kind of guarantee about what is updated, what is fixed, how fast, if ever.
And no kind of safety-oriented anything will run windows or any microsoft software. There is no windows edition of therac-25. The stuff you see in a hospital is normal workstation PCs for non-safety-relevant data entry and display. As soon as it becomes safety-relevant like controlling your heart-lung-machine, auto-dosing your medications, controlling the x-ray beam, you are far away from anything microsoft.
And actually, OSS is used more often in those safety-relevant settings. Why? Not because the OSS maintainers themselves would themselves provide any support, SLA or warranty. But because the nature of OSS provides third parties the possibility to certify, maintain and guarantee for their special 'safety-relevant-libxml2-fork'. Sometimes this is done by the device vendors themselves, sometimes they buy this from others. But it happens, and it is growing in frequency.
https://www.codethink.co.uk/news/trustable-software.html (Linux) https://access.redhat.com/en/compliance/iso-26262-asil-b (Linux) https://www.lynx.com/case-studies/secure-linux-medical-devic... (Linux) https://developer.arm.com/Tools%20and%20Software/Arm%20Compi... (clang/llvm)
There is tons more. Basically any compiler for safety-relevant embedded stuff is either clang or gcc under the hood. Linux is frequently encountered when the real-time requirements aren't too strict. With Linux also comes the usual Linux ecosystem of OSS libs and services. It won't look like your normal desktop OS, but quite a lot in that area is OSS.
Nothing at all from microsoft (except a useless BS certification "you can use Azure Devops as a code repo to store you ASIL-D code...").
Don't forget that microsoft is the only cloud provider who regularly has so much downtime and eye popping exploits against its cloud infra.
For those cases I have an Invalid CVE tag. No need to worry too much on inbeciles
Nick Wellnhofer is stepping away from libxml2 after a decade of unpaid maintenance. He’s forking it under the AGPL, but that will probably scare off most corporate users.
Meanwhile libxml2 is still everywhere. Without someone with real backing, a core piece of infrastructure is about to go unmaintained.
Once again, the open-source funding problem is laid bare: the internet runs on the unpaid evenings of a few people until they burn out (add relevant reference from XKCD, obviously).
(A)GPL licences are the way to go if you want to maintain the sanity long term…
I moved all my private projects to strong Copyleft licenses and not regretting it a bit.
"Expected effort required to maintain implementation" should be an evaluation criteria when selecting technologies. Thousand page RFCs do not make sustainable standards in the long run. Most committee designed specs end up in this category. People are impressed by complexity but actively pursuing simplicity is what we should be doing.
With not enough time to develop an alternative and too many application ecosystems relying on this library, I think it's a matter of time before a large company forks the library to fix security issues with it now that they have no choice but to do the work themselves. At least until IBM and Google figure out a way to move away from this library.
Ironically, IBM and Google 100% could just pay for it to be maintained under current licensing. (But won't.)
True. It'd be illuminating to know how far and wide it is used. It was always been my go-to library for parsing XML in a number of languages.
https://xkcd.com/2347
This isn't the library or feature browsers wanted to drop? I think I saw a hacker news post about it
That was xslt, which is part of the xml toolset but not specific to libxml2.
[flagged]
Maybe my human interaction interfacing software has a glitch but I am having a hard time parsing this content. Do I detect a hint of sarcasm? Please add a '/s' at the end of your future posts to aid my very archaic and vintage brain matter.
Jia Tan was the alias of the hacker(s) who infiltrated xz to plant a backdoor. He/They were in the project for 2 years I believe, and so had "significant experience" "maintaining" open source software.
Thanks for the info; I read the news but did not remember the name of the person.
"Jia Tan" was the name of the person (or group) who became a maintainer of libxz and sneaked in a vulnerability targeting OpenSSH.
*liblzma
Maybe _my_ software has a glitch but was your comment also sarcastic? Be sure to add an /s next time...
No I had a hard time understanding as I was not aware of the person in question, no sarcasm.
All Internet comments are to be assumed sarcastic until proven otherwise. Bombcar’s law.
[flagged]
[flagged]
I don't think this is about money but about will.
> Since commercial users of libxml2 are completely unwilling to fund further development
Where's that a quote from? I can't find it in the linked post (https://discourse.gnome.org/t/stepping-down-as-libxml2-maint...)
https://gitlab.gnome.org/GNOME/libxml2/-/issues/976#note_253...