chalkycrimp 34 minutes ago

Good chance I'm not the only one thinking this: share the code

codekansas 46 minutes ago

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

datadrivenangel 1 hour ago

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

  • datadrivenangel 1 hour ago

    it only cost them $10,000 in Claude API usage. :)

    • hmate9 1 hour ago

      You get that much usage in a monthly $200 subscription.

      • dofm 42 minutes ago

        And a kitten.

      • dvh 25 minutes ago

        So who's paying the cost?

        • exe34 24 minutes ago

          Corporate customers

  • risyachka 53 minutes ago

    Fpga is software-driven, model can verify its output easily.

    Zero surprise it did it

saejox 25 minutes ago

i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.