Nice breakdown. I've been wondering why it's so complicated to program on the GPU compared to the CPU. Is it a language paradigm issue or maybe a hardware design limitation? I always think of this when looking at the graphic APIs, espeically Vulkan.
It's not more complicated, you have things like CUDA which is basically C/C++ that compiles to GPU.
But GPUs have thousands of cores, which means if you want any benefit, the smallest data type you must use is something like array[n * 1024], or your GPU is wasted doing nothing. So you must shape your programs into that memory model or you're just wasting your time (and GPU).
Right, that makes sense. There's a ton of overhead in the setup though, I'm assuming that's because everything runs on the CPU by default. Is this then an OS design choice? Thanks for your input.
There can be overhead, though GPU memory + drivers are some of the most optimized software in the world.
Some OS/hardware have separate GPU memory (Nvidia), some have unified memory (Macos, modern consoles), but moving memory is usually not the problem.
Things don't "run on CPU by default", it's up to the programmer where to run it. But the CPU and GPU paradigm of "running a program" are very different, since on GPU you're running SIMD (i.e. running your code on arrays) or else there is no point to use a GPU.
part of the reason is historical cruft & history path dependency. GPUs didn't start as general computing device for parallelism but a device for the specific purpose of accelerating graphics with very specific fixed functions for that job and only that. with time that fixed pipeline became more complex and more configurable, not fixed/more flexible and hacks were in parallel used to make use of it for tasks outside graphics those hacks were later made into something more formal (CUDA/OpenCL).
CUDA being the winner (closed source, not really usable on non-nvidia GPUs) between those 2 and nvidia not wanting to open their competitive advantage to the competition for the niche of non gamings uses / scientific computing (and later AI, which became a huge win for nvidia).
vulkan came around after as this open thing (based on a previous attempt by AMD called mantle) that's trying to break down the leaky abstractions and the divide between language for graphics & video games and separate language/frameworks for gpu non-graphic computing with the promise of giving more control and more room for squeezing performance from the hardware at the cost of more complexity with the idea that you'll probably build your own abstractions on top of it for your subset of applications.
If GPU were from the get go designed as a parallelism tool first and video game tools second the hardware architecture and the language would probably be different than today and simpler (and if the competition between the duopoly didn't sabotage things)
don't forget that the drivers for GPUs would until recently get updates to patch specific video games for performance issues (not sure if they still do that ?)
Good summary, thanks. Sounds like it's all still pretty early, even for graphics but much more for parallel compute, which seems the direction the hardware is going in. I'm sure there is a CUDA equivalent for CPU but I would guess it's all the way back in the 70s or earlier. Hopefully open standards gets some traction.
It's partially historical; I'm sure if we redesigned GPGPU today from the ground up, we'd have an entirely different pipeline. And partially due to hardware vendor lock-in; you can't really design an alternative to Vulkan/d3d12 even if you had the skill and drive to do so.
But also partly in the fact that the GPU is specialized for certain operations over a CPU. It wants and need to obtain data in a different way from a CPU to maximize throughput. So language constructs are designed around making sure developers stay on this fast path. The complexity can be seen as some intentional gatekeeping to signal that this kind of programming is only for specific kinds of problems.
GLES isn't a bad way to teach graphics but I wish there was a middle ground between it and VK.
The worst part of GL is the mysterious global state you will never understand without studying the spec. The world could use an API that's basically GL but without global hidden singleton (per thread!) BS behind the scenes.
I've learned more graphics writing software rasterizers and ray tracers than using any APIs. If the goal is to teach GPU rendering specifically, then OpenGL is indeed more amenable to teaching than the others. But if the goal is to teach graphics and rendering concepts, then OpenGL et al are just miserable and counter-productive.
Disagree.
Trying to express all of graphics and rendering through the lens of OpenGL will of course fail and doing 3D rendering itself via raytracing is simply not possible in rasterization focused OpenGL without dirty workarounds. That is obvious.
A fundamental shift in thinking that got me into high performance graphics programming and parallelized image processing, the "Aha" Moment, is being forced to express how to calculate the final pixel independently, without having access to neighboring pixels of the same output buffer, confined to the straight jacket of the fragment shader. OpenGL with it's combination of hand-holding and relative simplicity as compared to Vulkan and limits of a predefined shader pipeline as compared to a pure software implementation is exactly the thing that I needed to get into graphics programming. It's an excellent jump in point I think.
Nice breakdown. I've been wondering why it's so complicated to program on the GPU compared to the CPU. Is it a language paradigm issue or maybe a hardware design limitation? I always think of this when looking at the graphic APIs, espeically Vulkan.
It's not more complicated, you have things like CUDA which is basically C/C++ that compiles to GPU.
But GPUs have thousands of cores, which means if you want any benefit, the smallest data type you must use is something like array[n * 1024], or your GPU is wasted doing nothing. So you must shape your programs into that memory model or you're just wasting your time (and GPU).
That's >50% of the problem.
Right, that makes sense. There's a ton of overhead in the setup though, I'm assuming that's because everything runs on the CPU by default. Is this then an OS design choice? Thanks for your input.
There can be overhead, though GPU memory + drivers are some of the most optimized software in the world.
Some OS/hardware have separate GPU memory (Nvidia), some have unified memory (Macos, modern consoles), but moving memory is usually not the problem.
Things don't "run on CPU by default", it's up to the programmer where to run it. But the CPU and GPU paradigm of "running a program" are very different, since on GPU you're running SIMD (i.e. running your code on arrays) or else there is no point to use a GPU.
part of the reason is historical cruft & history path dependency. GPUs didn't start as general computing device for parallelism but a device for the specific purpose of accelerating graphics with very specific fixed functions for that job and only that. with time that fixed pipeline became more complex and more configurable, not fixed/more flexible and hacks were in parallel used to make use of it for tasks outside graphics those hacks were later made into something more formal (CUDA/OpenCL).
CUDA being the winner (closed source, not really usable on non-nvidia GPUs) between those 2 and nvidia not wanting to open their competitive advantage to the competition for the niche of non gamings uses / scientific computing (and later AI, which became a huge win for nvidia).
vulkan came around after as this open thing (based on a previous attempt by AMD called mantle) that's trying to break down the leaky abstractions and the divide between language for graphics & video games and separate language/frameworks for gpu non-graphic computing with the promise of giving more control and more room for squeezing performance from the hardware at the cost of more complexity with the idea that you'll probably build your own abstractions on top of it for your subset of applications.
If GPU were from the get go designed as a parallelism tool first and video game tools second the hardware architecture and the language would probably be different than today and simpler (and if the competition between the duopoly didn't sabotage things)
don't forget that the drivers for GPUs would until recently get updates to patch specific video games for performance issues (not sure if they still do that ?)
Good summary, thanks. Sounds like it's all still pretty early, even for graphics but much more for parallel compute, which seems the direction the hardware is going in. I'm sure there is a CUDA equivalent for CPU but I would guess it's all the way back in the 70s or earlier. Hopefully open standards gets some traction.
It's partially historical; I'm sure if we redesigned GPGPU today from the ground up, we'd have an entirely different pipeline. And partially due to hardware vendor lock-in; you can't really design an alternative to Vulkan/d3d12 even if you had the skill and drive to do so.
But also partly in the fact that the GPU is specialized for certain operations over a CPU. It wants and need to obtain data in a different way from a CPU to maximize throughput. So language constructs are designed around making sure developers stay on this fast path. The complexity can be seen as some intentional gatekeeping to signal that this kind of programming is only for specific kinds of problems.
GLES isn't a bad way to teach graphics but I wish there was a middle ground between it and VK.
The worst part of GL is the mysterious global state you will never understand without studying the spec. The world could use an API that's basically GL but without global hidden singleton (per thread!) BS behind the scenes.
Sounds like a great way to make sure students understand fundamental concepts.
The linked webpage refuses keyboard navigation e.g. cursors, pageup/down and home/end seem to do nothing.
Awful.
I think it's because of the webgl demo at top stealing focus
Specifically, it registers a keydown handler on the <html> that swallows everything.
I've learned more graphics writing software rasterizers and ray tracers than using any APIs. If the goal is to teach GPU rendering specifically, then OpenGL is indeed more amenable to teaching than the others. But if the goal is to teach graphics and rendering concepts, then OpenGL et al are just miserable and counter-productive.
Disagree. Trying to express all of graphics and rendering through the lens of OpenGL will of course fail and doing 3D rendering itself via raytracing is simply not possible in rasterization focused OpenGL without dirty workarounds. That is obvious.
A fundamental shift in thinking that got me into high performance graphics programming and parallelized image processing, the "Aha" Moment, is being forced to express how to calculate the final pixel independently, without having access to neighboring pixels of the same output buffer, confined to the straight jacket of the fragment shader. OpenGL with it's combination of hand-holding and relative simplicity as compared to Vulkan and limits of a predefined shader pipeline as compared to a pure software implementation is exactly the thing that I needed to get into graphics programming. It's an excellent jump in point I think.
The way I’m interpreting their post is, you could far more easily teach things like introductory projective geometry without a GPU.