What kind of CPU is the CPU part? The link doesn't tell. Is it like ARM or RISC V and can it run a general purpose OS like Linux?
Do you plugin in DDR memory somewhere for the 480GB, or is this already on the board?
EDIT: found answer to my own question in the datasheet: "The NVIDIA Grace CPU combines 72 Neoverse V2 Armv9 cores with up to 480GB of server-class LPDDR5X memory with ECC."
Quoting from https://www.nvidia.com/en-us/data-center/grace-cpu-superchip...
> The NVIDIA Grace CPU Superchip uses the NVIDIA® NVLink®-C2C technology to deliver 144 Arm® Neoverse V2 cores and 1 terabyte per second (TB/s) of memory bandwidth.
> High-performance CPU for HPC and cloud computing Superchip design with up to 144 Arm Neoverse V2 CPU cores with Scalable Vector Extensions (SVE2)
> World’s first LPDDR5X with error-correcting code (ECC) memory, 1TB/s total bandwidth
> 900 gigabyte per second (GB/s) coherent interface, 7X faster than PCIe Gen 5
> NVIDIA Scalable Coherency Fabric with 3.2TB/s of aggregate bisectional bandwidth
> 2X the packaging density of DIMM-based solutions
> 2X the performance per watt of today’s leading CPU
This is about the "Grace Hopper Superchip", which has one of the Grace CPU's replaced with a GPU. Thus, 72 Neoverse V2 cores.
https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-su...
72 Neoverse V2 Armv9 cores.
Not sure how one interfaces with it, but it presumably runs an approved Linux distro, with a web server at best.
it's a normal chip like your x64 chip.you install ARM variant of your linux distribution on it and run it natively.
source: I have one
Could one in theory then game on it by installing Steam on linux? And run all the LLM's with the largest models you want straight from github repositories?
Yes, its no different from any other server. If it was a desktop computer, you will connect your HDMI to it to a monitor and put GPUs on its PCIe slots and attach hard drives to NVMe.
Where does one buy one anyway? What does it cost? (I know it'll be some huge amount, just curious)
"If you have to ask..."
It is many, many thousands.
I think more like many tens of thousands.
These types of items are generally only sold to large buyers with established partnerships. Eventually some stock gets allocated to wholesalers like ShopBLT[0] and CDW[1], which break tradition and sell directly to consumers rather than only selling to retailers/system integrators.
These are for the 80GB versions, currently priced at $30,000 per GPU. It will likely be many months before this 96GB version is available to prosumers, if it ever is at all.
0: http://www.shopblt.com/cgi-bin/shop/shop.cgi?action=thispage...
1: https://www.cdw.com/product/nvidia-h100-gpu-computing-proces...
Some of the older generations run roughly the cost of 3/2 house in a medium COL city.
Ex - the HGX A100 platforms sold as single servers usually ran around 150k, but could get up above 200k depending on loadout.
Just getting an H100 (just the gpu) right now is ~40k new.
There is a reason nvidia's stock is doing so well...
LTT posted a video a few days ago from Computex talking a bit in depth about it