linsomniac 3 months ago

As someone who has gone through the ipfwadm -> ipchains -> iptables history, I would generally be pretty meh about another firewalling change. But, I have high hopes that it'll give me the ability to do reasonable traffic shaping in Linux.

I once set up a FreeBSD box to do shaping for an ISP's entire DSL customer base, and it was a couple commands and worked brilliantly! By contrast, I've dabbled with Linux shaping for ~2 decades, and the best I've gotten is one recipe that works well for one limited use case. Maybe it's just me, but I've never been able to make it do my bidding, despite several occasions sitting down and giving it the old college try.

For other kernel paths, I'm really interested in using it to monitor for file modifications in the way that inotify is not very good. And I often resort to strace to figure out low level what is going on, it might be really good for that too.

Looks like an interesting book!

  • wahern 3 months ago

    > And I often resort to strace to figure out low level what is going on, it might be really good for that too.

    BPF is a double-edged sword in that regard. It's being used to write better tracing tools for normal kernel facilities, but facilities which migrate to BPF become more opaque and unstructured than ever.

  • navinsylvester 3 months ago

    > I'm really interested in using it to monitor for file modifications in the way that inotify is not very good

    Totally agree. Have had horrible experiences with inotify. I was able to modify and get a good alternative to find new files.

    It still isn't polished and haven't tested it enough so will release it back to community when its ready for beta.

  • nimmen 3 months ago

    That's my impression also, another few decades and linux might get PF, yay... But that is good change, tried picking up IPTABLES multiples times, but always go back to *BSD or dedicated hardware

    • linsomniac 3 months ago

      I like iptables fairly well, though it does take some planning to ensure it can remain speedy. In fact, the only way I could make shaping in Linux work involved using iptables to tag the packets for tc to then shape. Used that for our mirror server, among other things. Even had a fairly slick setup where our core routers would, every few seconds, report their bandwidth usage to the mirror server, which would adjust it's shaping to consume all our committed bandwidth, without interfering with customers traffic or costing us more bandwidth charges.

  • Fnoord 3 months ago

    Wondershaper [1] should make it easy on Linux. On *BSD there is AltQ for this which also has PF integration (or well, back when I looked into this in 2004).

    I'm curious to hear which use cases didn't work out for you.

    Regarding BPF, is it possible to convert IPT rules to BPF for backwards compatibility?


    • linsomniac 3 months ago

      Thanks for that pointer, I had only used wondershaper as provided in the Ubuntu package, which is version 1.1a, and horribly broken. I recently had a system that was downloading around 1TB of archive data, and killing the network for other use, so I set up wondershaper on that VM. It did indeed limit it, but I couldn't get it to limit it to anything other than the 768kbps it seemed built to do. I really needed more like 50-100Mbps. I'll have to try version 1.4 on github.

  • sargun 3 months ago

    I think the last thing that's holding Linux back is the root qdisc lock. It becomes very problematic on high PPS workloads and complicated shaper configurations.

    Jumbo frames largely solve the problem, but it's the biggest hurdle we've found.

  • Wheaties466 3 months ago

    why not use tc for linux traffic shaping? you can even use bpf.

    • linsomniac 3 months ago

      I have never been successful at using tc or the recipes in LARTC or other resources to set up anything more than a basic shaping setup. And even basic ones took hours of messing around to get working. Maybe that's just me?

      • _jal 3 months ago

        Not just you. I once spent multiple days trying to get it working, and eventually did - by installing FreeBSD and using pf.

zaphar 3 months ago

I've been a fan of dtrace for a long time and so I've been meaning to learn BPF to get similar functionality native to the linux kernel. I may pick this book up.

TwoNineFive 3 months ago

It's an advertisement for a book. Cilium is on there and that's a name I associate with blogverts.

There are eleven exclamation marks on that page.

BPF is the new paradigm for synergistic application of strategic opportunities in the use of acronyms.

dustfinger 3 months ago

Thanks for sharing Jessie! I am definitely going to order this book :-)

EDIT: Why did I get down voted? I am not affiliated with this book, the authors of the book, or the OP in anyway. It just so happens that I will find this book very useful, so I thanked the OP for sharing.

It is sad we live in a world where there is so much manipulation that we are suspicious of each other's sincerity. I guess I understand where the downvoter was coming from :-(

  • monocasa 3 months ago

    This sort of comment isn't looked highly upon on HN as it doesn't really add to the discussion. It's what the upvote button is for.

    • mirashii 3 months ago

      And in addition, from the guidelines:

      > Please don't comment about the voting on comments. It never does any good, and it makes boring reading.

    • dustfinger 3 months ago

      Fair enough. I will remember for next time.