> The patents are referred to as Standards Essential Patents, or SEPs. Because of this arrangement, a vendor like Nordic probably can’t open source their modem firmware even if they wanted to.
Why would a SEP license forbid publishing modem firmware? The IP[0] involved with the modem firmware is copyrighted and trade secret material; publishing the firmware does not (and cannot) grant a patent license to those licensed standards-essential patents. It only imperils your implementation of the modem firmware, not the whole patent spec.
To wit, Cisco publishes an H.264 codec with BSD-licensed code, even though H.264 is under a shitload of patents that they are licensing. OpenH264 does ship with an additional patent sublicense that applies if you are using their build and installing it in a specific way, but that is particular to MPEG-LA's licensing structure[1]. Publishing an implementation of a patent you are licensing does not automatically sublicense the patent.
[0] Laws that grant the ability to dictate the conduct of your competitors
[1] Cisco is abusing(?) the MPEG-LA royalty fee cap to shield FOSS entities from having to take a license, basically, by paying for them.
> Why would a SEP license forbid publishing modem firmware?
Contracts and money.
> The IP[0] involved with the modem firmware is copyrighted and trade secret material; publishing the firmware does not (and cannot) grant a patent license to those licensed standards-essential patents. It only imperils your implementation of the modem firmware, not the whole patent spec.
Publishing the source to the firmware would allow people to make use of patented tech without paying for it even though it would be illegal to do so. Also, your license may simply stipulate that you cannot publish or share your code in anyway outside the company.
> To wit, Cisco publishes an H.264 codec with BSD-licensed code, even though H.264 is under a shitload of patents that they are licensing.
> Cisco is abusing(?) the MPEG-LA royalty fee cap to shield FOSS entities from having to take a license, basically, by paying for them.
As you point out here, Cisco is paying for it. They publish open code, and to ensure the LA doesn't come after them, they're paying additional money.
There's also the possibility of trade secrets being involved. An example hypothetical: Maybe Qualcomm has tons of lab data showing signal performance and integrity for any given channel, band, power level, general environment (urban, suburban, rural), device position, device altitude, etc.
You might want to use that data, and algorithms derived from it, to improve your device's battery life. That data obviously cannot be patented, and while the lab data would be copyrighted and not part of the finished product, Qualcomm would understandably be mighty particular about any open-source code implementing the conclusions.
I suspect a big reason is that there’s middleware: some of this stuff is so complex, that like video games, there’s probably a cottage industry of companies selling partially implemented bits and pieces, or licensing across companies of source code, or companies only licensing the patents if you agree to never release code. There’s also the IP angle: if you do manage to figure it out, there’s no honor helping Huawei.
> I suspect a big reason is that there’s middleware
I’d bet it’s simpler. Saying “yes” to the ‘can we share this outside of this company?’ Question means a lot of work and meetings and red tape and somebody is going to have to rope in legal and … and … and …
True, but who owns most of the patents? Some of the most litigious companies on earth (Qualcomm, Broadcom, Cisco). How many legal cases are more expensive than patent litigation?
Also, it’s very simple: Imagine you approach Qualcomm. The script probably goes something like this:
“Sure, we’ll license you that patent, for $0.92 per device, a $25K per month maintenance fee, and compliance with our terms and conditions - chiefly, don’t license this patent to anyone else, and don’t share the source code of your implementation.”
I honestly just figured it was spectrum licensing. You can’t sell devices that can broadcast at power on just any old frequency. At a hardware level these modems are capable of going well outside the bounds of mobile device regulations. See all the ‘turn xyz into a software defined radio’ hacks that have been done before on other devices on the market.
So the firmware is locked down. To do otherwise will break spectrum licensing as the device then becomes capable of doing things outside the original licensed purpose.
That's probably it, the FCC won't let you sell a device that has a user-accessible method to violate wireless regulations. At most, they could release the source code but not the signing keys so end-users could inspect but not change the firmware.
As part of our IoT offerings, we tried quite hard to build devices with mobile connectivity about 7 years ago. This was just as low power 5G, NB-IoT and similar technologies were going to become a thing. We gave up because it was too much effort for little return, and it was better to focus on doing things with data, rather than collecting it. Even getting prototypes up with powered fanless PCs (SBCs) and Mini-PCIe or M.2 modems was harder than it should have been - you would think it would be easy with off-the-shelf devices and drivers.
Small Internet connected devices are still needed, despite the perception that IoT is dead. Mobile networks and the modem supply chain are definitely holding the market back. On the plus side, for stationary powered devices most people are happy to connect to wi-fi. For low power devices, LoRa, with private gateways, seems to be a standard. Mobile that is used in outdoor vehicle and asset tracking is still stuck with fighting with modems as per OP.
You might want to take another look, I went through a similar process a couple of years ago, and am now doing it again because our original equipment supplier decided they'd like a life (RIP PC Engines). There's a thriving industry of companies providing fanless ARM based machines either shipping with LTE modems, or with M2 & SIM slots on the board to provide your own.
We install in industrial environments where an accessible internet connection is far from a given, having LTE on all our devices means that we can almost always give the device a way to call home. I can strongly recommend Compulab's devices, which you can purchase as a fully assembled unit that just needs a SIM card put in it (I can't recommend the Linux distribution they run, but you're free to flash them with whatever you'd like).
> There's a thriving industry of companies providing fanless ARM based machines either shipping with LTE modems, or with M2 & SIM slots on the board to provide your own
When I did this about 10 years ago we had quite a number to pick from. Build your own was possible. But that was only because of our organization had the capability and expertise to do it. We settled on 3 off the shelf ones. That was dependent on cost and number of I/O the customer was needing.
DiY basically is 'first make your own computer with the ARM/MIPS/x86 chipsets' then 'spin your own special firmware for it'. Then 'build your own ground up linux distro or similar with compiler chain and SDK to work with it'. You may be able to get someone to sell/give you a reference layout/SDK. Then after all of that. You are now ready to add in a modem. Also prepare for the certifications of all the mobile networks you want to run on. Plus software for you to interact with the cell modem. Oh also you need to work on getting yourself provisioned correctly in the mobile networks. When you do for five devices it is a couple hours of playing with an API. But you probably want hundreds of them so be prepared for managing that, plus billing. Oh also you need to manage EoL for your parts. Many IoT installations are looking at you hanging around for 10+ years.
> Also prepare for the certifications of all the mobile networks you want to run on
AFAIK the modem maker actually does this. You would rather check that the chip is certified for what you need before you buy.
> also you need to work on getting yourself provisioned correctly in the mobile networks
AFAIK also, this is a service provided by "virtual operators", companies that pass deals with the main operators and provide advanced services for machine-to-machine uses, for instance.
But perhaps these didn't exist 10 years ago. Things really started to move where I operate when they EoL'd copper wires.
Thanks... I'll have another look. I always assumed that device/modem supply would catch up, but they always seemed held back by the established mobile network operators. Also, covid-related supply-chain issues stopped a lot of products in their tracks. We would waited up to a year for modems in some cases. I suppose that has all flushed through the system by now.
> The number one reason we see customers opt to not use cellular when
> another connectivity option is on the table is due to concerns around
> cost and reliability.
For "reliability" read "availability". Cellular networks are profitable because carriers serve customers that are easy to serve and leave the rest alone. Vehicle 2 Vehicle communications mediated by cellular is a non-starter because people drive a lot in places where cell phones don't work. Wireless internet has been a false hope that's spoiled realistic plans for universal service because it's only affordable in markets that are already served. Large-scale government and corporate IoT programs find that 20% or more of stations are in places without service, etc.
On top of this, there are markets that are not well covered due to local politics. I can go just a few blocks away into a neighborhood that has plenty of density to be interesting to the major carriers but has basically no usable cellular data. A couple years ago, there was a big fight between Crown Castle (big cellular network contractor) and the local government. Crown Castle wanted to build cell towers. The residents wanted service but didn’t want cell towers in their back yard. The government tried, and failed, to avoid issuing permits, because federal law sensibly prevents local government from effectively prohibiting the construction of cell towers.
Win? No! The local government is very proud of itself for having pulled a rabbit out of its hat. It turns out that a cell tower is useless without backhaul, and the government rolled out so much red tape that Crown Castle determined that they would never recover the costs of installing the backhaul and abandoned the entire project. The local government somehow considers this to be a victory.
What I remember in the 3G era was that my MVNO phone [1] worked great in medium sized cites like Rochester, NY but poorly in places with lower density and even worse in the highest density places like NYC and Hollywood. 4G had the capacity to serve super density urban areas really well.
[1] I've got a suspicion that Tracfones ride the back of the bus and get worse service than you'd get with a premium-priced plan.
I think the miserable state of cellular modems ultimately comes from the power dynamics of private spectrum. The 3GPP protocols are complex because they're not really beholden to the implementers, but rather to the network operators. The operators are fine with additional complexity if it serves their interests, and the cost will just be passed on to the OEMs. The network operators have all the power because they're the ones with the licenses.
WiFi tends to be simpler in part because the protocol authors are working more in the interest of the implementers, since it's really they who decide whether to adopt or not. Obviously a gross simplification but I think it's at the heart of the problem.
Long time ago i worked on WiMAX devices which was a competing standard for the term “4g” from ieee and while it was simpler it wasn’t a whole lot simpler and performance sucked, so there’s that
Hey folks, author here. Appreciate the comments and discussion on the post! Happy to continue the discussion or answer any follow-on questions folks have about our investigation and resolution.
> The patents are referred to as Standards Essential Patents, or SEPs. Because of this arrangement, a vendor like Nordic probably can’t open source their modem firmware even if they wanted to.
Why would a SEP license forbid publishing modem firmware? The IP[0] involved with the modem firmware is copyrighted and trade secret material; publishing the firmware does not (and cannot) grant a patent license to those licensed standards-essential patents. It only imperils your implementation of the modem firmware, not the whole patent spec.
To wit, Cisco publishes an H.264 codec with BSD-licensed code, even though H.264 is under a shitload of patents that they are licensing. OpenH264 does ship with an additional patent sublicense that applies if you are using their build and installing it in a specific way, but that is particular to MPEG-LA's licensing structure[1]. Publishing an implementation of a patent you are licensing does not automatically sublicense the patent.
[0] Laws that grant the ability to dictate the conduct of your competitors
[1] Cisco is abusing(?) the MPEG-LA royalty fee cap to shield FOSS entities from having to take a license, basically, by paying for them.
> Why would a SEP license forbid publishing modem firmware?
Contracts and money.
> The IP[0] involved with the modem firmware is copyrighted and trade secret material; publishing the firmware does not (and cannot) grant a patent license to those licensed standards-essential patents. It only imperils your implementation of the modem firmware, not the whole patent spec.
Publishing the source to the firmware would allow people to make use of patented tech without paying for it even though it would be illegal to do so. Also, your license may simply stipulate that you cannot publish or share your code in anyway outside the company.
> To wit, Cisco publishes an H.264 codec with BSD-licensed code, even though H.264 is under a shitload of patents that they are licensing. > Cisco is abusing(?) the MPEG-LA royalty fee cap to shield FOSS entities from having to take a license, basically, by paying for them.
As you point out here, Cisco is paying for it. They publish open code, and to ensure the LA doesn't come after them, they're paying additional money.
There's also the possibility of trade secrets being involved. An example hypothetical: Maybe Qualcomm has tons of lab data showing signal performance and integrity for any given channel, band, power level, general environment (urban, suburban, rural), device position, device altitude, etc.
You might want to use that data, and algorithms derived from it, to improve your device's battery life. That data obviously cannot be patented, and while the lab data would be copyrighted and not part of the finished product, Qualcomm would understandably be mighty particular about any open-source code implementing the conclusions.
I suspect a big reason is that there’s middleware: some of this stuff is so complex, that like video games, there’s probably a cottage industry of companies selling partially implemented bits and pieces, or licensing across companies of source code, or companies only licensing the patents if you agree to never release code. There’s also the IP angle: if you do manage to figure it out, there’s no honor helping Huawei.
> I suspect a big reason is that there’s middleware
I’d bet it’s simpler. Saying “yes” to the ‘can we share this outside of this company?’ Question means a lot of work and meetings and red tape and somebody is going to have to rope in legal and … and … and …
Or, “no, get back to work”.
True, but who owns most of the patents? Some of the most litigious companies on earth (Qualcomm, Broadcom, Cisco). How many legal cases are more expensive than patent litigation?
Also, it’s very simple: Imagine you approach Qualcomm. The script probably goes something like this:
“Sure, we’ll license you that patent, for $0.92 per device, a $25K per month maintenance fee, and compliance with our terms and conditions - chiefly, don’t license this patent to anyone else, and don’t share the source code of your implementation.”
I honestly just figured it was spectrum licensing. You can’t sell devices that can broadcast at power on just any old frequency. At a hardware level these modems are capable of going well outside the bounds of mobile device regulations. See all the ‘turn xyz into a software defined radio’ hacks that have been done before on other devices on the market.
So the firmware is locked down. To do otherwise will break spectrum licensing as the device then becomes capable of doing things outside the original licensed purpose.
That's probably it, the FCC won't let you sell a device that has a user-accessible method to violate wireless regulations. At most, they could release the source code but not the signing keys so end-users could inspect but not change the firmware.
As part of our IoT offerings, we tried quite hard to build devices with mobile connectivity about 7 years ago. This was just as low power 5G, NB-IoT and similar technologies were going to become a thing. We gave up because it was too much effort for little return, and it was better to focus on doing things with data, rather than collecting it. Even getting prototypes up with powered fanless PCs (SBCs) and Mini-PCIe or M.2 modems was harder than it should have been - you would think it would be easy with off-the-shelf devices and drivers.
Small Internet connected devices are still needed, despite the perception that IoT is dead. Mobile networks and the modem supply chain are definitely holding the market back. On the plus side, for stationary powered devices most people are happy to connect to wi-fi. For low power devices, LoRa, with private gateways, seems to be a standard. Mobile that is used in outdoor vehicle and asset tracking is still stuck with fighting with modems as per OP.
You might want to take another look, I went through a similar process a couple of years ago, and am now doing it again because our original equipment supplier decided they'd like a life (RIP PC Engines). There's a thriving industry of companies providing fanless ARM based machines either shipping with LTE modems, or with M2 & SIM slots on the board to provide your own.
We install in industrial environments where an accessible internet connection is far from a given, having LTE on all our devices means that we can almost always give the device a way to call home. I can strongly recommend Compulab's devices, which you can purchase as a fully assembled unit that just needs a SIM card put in it (I can't recommend the Linux distribution they run, but you're free to flash them with whatever you'd like).
> There's a thriving industry of companies providing fanless ARM based machines either shipping with LTE modems, or with M2 & SIM slots on the board to provide your own
When I did this about 10 years ago we had quite a number to pick from. Build your own was possible. But that was only because of our organization had the capability and expertise to do it. We settled on 3 off the shelf ones. That was dependent on cost and number of I/O the customer was needing.
DiY basically is 'first make your own computer with the ARM/MIPS/x86 chipsets' then 'spin your own special firmware for it'. Then 'build your own ground up linux distro or similar with compiler chain and SDK to work with it'. You may be able to get someone to sell/give you a reference layout/SDK. Then after all of that. You are now ready to add in a modem. Also prepare for the certifications of all the mobile networks you want to run on. Plus software for you to interact with the cell modem. Oh also you need to work on getting yourself provisioned correctly in the mobile networks. When you do for five devices it is a couple hours of playing with an API. But you probably want hundreds of them so be prepared for managing that, plus billing. Oh also you need to manage EoL for your parts. Many IoT installations are looking at you hanging around for 10+ years.
> Also prepare for the certifications of all the mobile networks you want to run on
AFAIK the modem maker actually does this. You would rather check that the chip is certified for what you need before you buy.
> also you need to work on getting yourself provisioned correctly in the mobile networks
AFAIK also, this is a service provided by "virtual operators", companies that pass deals with the main operators and provide advanced services for machine-to-machine uses, for instance.
But perhaps these didn't exist 10 years ago. Things really started to move where I operate when they EoL'd copper wires.
Seconding this about Compulab. If they made a box with a Marvell CN9130 it would be really nice.
Thanks... I'll have another look. I always assumed that device/modem supply would catch up, but they always seemed held back by the established mobile network operators. Also, covid-related supply-chain issues stopped a lot of products in their tracks. We would waited up to a year for modems in some cases. I suppose that has all flushed through the system by now.
Why mobile connectivity instead of Bluetooth?
As an end user, I prefer my devices use Bluetooth and a hub device like a phone or laptop, rather than each one have a direct mobile connection.
> The number one reason we see customers opt to not use cellular when > another connectivity option is on the table is due to concerns around > cost and reliability.
For "reliability" read "availability". Cellular networks are profitable because carriers serve customers that are easy to serve and leave the rest alone. Vehicle 2 Vehicle communications mediated by cellular is a non-starter because people drive a lot in places where cell phones don't work. Wireless internet has been a false hope that's spoiled realistic plans for universal service because it's only affordable in markets that are already served. Large-scale government and corporate IoT programs find that 20% or more of stations are in places without service, etc.
On top of this, there are markets that are not well covered due to local politics. I can go just a few blocks away into a neighborhood that has plenty of density to be interesting to the major carriers but has basically no usable cellular data. A couple years ago, there was a big fight between Crown Castle (big cellular network contractor) and the local government. Crown Castle wanted to build cell towers. The residents wanted service but didn’t want cell towers in their back yard. The government tried, and failed, to avoid issuing permits, because federal law sensibly prevents local government from effectively prohibiting the construction of cell towers.
Win? No! The local government is very proud of itself for having pulled a rabbit out of its hat. It turns out that a cell tower is useless without backhaul, and the government rolled out so much red tape that Crown Castle determined that they would never recover the costs of installing the backhaul and abandoned the entire project. The local government somehow considers this to be a victory.
What I remember in the 3G era was that my MVNO phone [1] worked great in medium sized cites like Rochester, NY but poorly in places with lower density and even worse in the highest density places like NYC and Hollywood. 4G had the capacity to serve super density urban areas really well.
[1] I've got a suspicion that Tracfones ride the back of the bus and get worse service than you'd get with a premium-priced plan.
I think the miserable state of cellular modems ultimately comes from the power dynamics of private spectrum. The 3GPP protocols are complex because they're not really beholden to the implementers, but rather to the network operators. The operators are fine with additional complexity if it serves their interests, and the cost will just be passed on to the OEMs. The network operators have all the power because they're the ones with the licenses.
WiFi tends to be simpler in part because the protocol authors are working more in the interest of the implementers, since it's really they who decide whether to adopt or not. Obviously a gross simplification but I think it's at the heart of the problem.
Vendors wrote those standards though, not the operators. They are complex because it’s a defensive moat against competition imho
They are complex because, usually, they are a union of everything every vendor in the consortium has done and/or wants to do.
Looking at the overview of the architecture on the network side [1] tells the tale, I think.
My take is that telecom operators forcefully had to evolve from Morse and copper to IP and radio, hacking stuff left and right every step of the way.
[1] https://en.wikipedia.org/wiki/IP_Multimedia_Subsystem#Archit...
Long time ago i worked on WiMAX devices which was a competing standard for the term “4g” from ieee and while it was simpler it wasn’t a whole lot simpler and performance sucked, so there’s that
Wow that was quite the technical deep dive! I bet there's not one person in the world who understands all the interactions in this system.
I too greatly enjoyed the article. Love those deep troubleshooting write-ups!
Hey folks, author here. Appreciate the comments and discussion on the post! Happy to continue the discussion or answer any follow-on questions folks have about our investigation and resolution.
[dead]