OpenCL 1.0: The Road to Pervasive GPU Computing
by Derek Wilson on December 31, 2008 6:40 PM EST- Posted in
- GPUs
Why is Parallel Computing Hard?
There are plenty of issues with parallel programming. Breaking up the problem is often the most important and complex step, especially when the parallelism is not obvious. As we are rooted in a world of sequential programming, conceptualizing the parallelization of tasks that lend themselves to sequential programming is tough. This can require not only the reworking of code, but redesigning the entire process of solving a problem.
Even in problems that lend themselves to parallelism, exploiting the parallelism can be tough. Even if you know the best and fastest algorithm for solving a data parallel problem, it isn't always possible to translate that to an efficient program. For instance, if I want to multiply two matrices with 100k x 100k dimentions, I can't just spawn all the threads I would need. If I were using POSIX threads to calculate one cell of the result matrix each, I would spend more time creating threads and allocating resources than actually doing the computation. I've got to take the resources I have and use them to the best of my ability. Though I can do matrix multiplication in parallel, I have to be careful about how I break up the problem and I can't exploit all the parallelism possible because of the tools I normally work with.
We are also limited in terms of hardware resources. With only a few processors available for general purpose programming, even if the software overhead weren't an issue we couldn't actually get any speed up from parallelizing beyond a certain point. This not only means that we can't exploit tons of parallelism even if the algorithm lends itself to it and this discourages programmers from thinking in terms of parallelism.
How Does OpenCL Help?
What if we had not only a pool of hardware resources hundreds wide that could handle thousands of threads in flight at a time with no software overhead? Well, we do: it's called a GPU. And if we could use the GPU for processing, then we could spawn a bunch of threads and really chew through the matrix multiplication we talked about earlier (or whatever). We might still have to be concerned about how many hardware resources we have in order to best map the problem to the specific device in the system. And we still have the problem of actually spawning, managing and running threads on the GPU hardware.
But what if we could write a special function, called a kernel, that can instantly be spawned hundreds or thousands or millions of times and run on different data all without needing to handle creating and managing all the threads ourselves. And what if we didn't need to worry about how to break up our problem and left actually determining how to handle allocating threads to the runtime? Well, now we have a solution: that's OpenCL.
The GPU is the vehicle for exploiting data parallelism. But before now our vehicle has run like a train on a track called real-time 3D graphics acceleration. OpenCL removes the track and the limitations and builds in a steering wheel developers can use to take the GPU (and other parallel devices) anywhere a programmer can imagine.
Facebook
Twitter
RSS
37 Comments
View All Comments
DerekWilson - Wednesday, December 31, 2008 - link
that is not possible -- when using the CPU to process data you need to copy it off the GPU ... when using the GPU to do processing, you need to copy data onto the GPU.What you don't need to do is to worry about copying data from an OpenGL buffer that resides on the GPU to another buffer in order to work on it with OpenCL.
In DX11, you can share buffers between the Pixel Shader and the Compute Shader. Both of these are processed on the graphics card. You can do graphics work and general purpose compute work on the same data ... this is useful for effects physics, visualization of calculations, or complex shaders that might not be possible in the constraints of HLSL.
With OpenGL + OpenCL, you can do the same thing -- share data between graphics buffers and OpenCL buffers. But these buffers reside on the GPU.
In both DX11 and OpenCL, data must be moved off the GPU to process it with the CPU.
If OpenGL and OpenCL did not have binary level buffer compatibility, worst case we would need to copy OpenGL buffers off the GPU, convert them, copy them into OpenCL buffers in the correct format, and then re-upload the data to the GPU. Alternately, we could modify the buffer on the GPU, but that would still require processing power and incur a performance penalty.
Jaybus - Friday, January 2, 2009 - link
I think kevinkreiser was advocating a shared memory architecture, where CPU and GPU could access the same physical RAM, so that there would be no need to copy buffers. However, I disagree with such an approach, because that is only eliminating the buffer copy overhead by forcing the use of a global mutex or some other method of shared memory arbitration. The bottleneck would then become memory contention, offsetting any performance gained by eliminating the copy.Loki726 - Friday, January 2, 2009 - link
PCIe latency is incredibly large compared to memory copy latency. For example, a synchronous copy of a single byte from CPU memory to GPU memory on an 8800GT using CUDA takes around 100k cpu cycles to complete, where non-cached CPU memory copies of the same size are in the order of 100-1000s of cycles. PCIe transfers only become fast when copying large chunks of data.You are right that a shared memory architecture would require synchronization via some mechanism (mutex or other), but this would still be much faster than a DMA copy over PCIe for small data sizes if it was implemented correctly. There is no reason it should be any slower than sharing data between two threads in an SMP.
I think the reason why no one builds systems like this is because low latency access to a shared DRAM would require complex protocols between the GPU and CPU memory controllers to ensure memory consistency and coherence, and no one builds CPUs and GPUs that closely integrated.
DerekWilson - Saturday, January 3, 2009 - link
Some people built / build systems like this -- they are called game consoles ;-)Loki726 - Saturday, January 3, 2009 - link
Good point Derek. The Xbox 360 supports tightly integrated CPU-GPU communication:"The bus design and the CPU L2 provide added support that allows the GPU to read directly from the CPU L2 cache."[1]
[1] Andrews, J. and Baker, N. 2006. Xbox 360 System Architecture. IEEE Micro 26, 2 (Mar. 2006), 25-37
Wwhat - Monday, January 5, 2009 - link
Whatever happened to the hypertransport bus on motherboards and making graphics card for it? That would nicely cover both issues, plus since intel also is going in that direction they might agree with AMD at some far point in the future on a universal direct CPU transport bus connector.Or perhaps the graphicscard makers should consider making a universal socket on their graphicscards that connects to the motherboard to a dedicated connector designed for DMA between a shared memory space with the CPU, a cache designed for shared GPU/CPU use, the advantage would be that people would yet again be forced to buy a new motherboard, and chipset, and that will keep the money rolling in ;]
Personally I think they should sit down in some room alone with themselves and think a bit until they realise having everybody doing their own propriety interfaces and systems is NOT a nice and positive and helpful and even economical way to go about thing and that making a plan then talking in a group with the 'opposition' and then tweaking it before releasing isn't such a bad idea and might actually lead to MORE profit and innovation.
Loki726 - Monday, January 5, 2009 - link
The interface you are thinking of is called HTX and there are some specialized products that use it. Hypertransport may be an open spec, but the memory transfer and coherence protocols used by AMD are not open. So it is not possible for a third party vendor to sit down and implement an HTX card that could work cooperatively with an AMD processor without negotiating a license from AMD. Intel's equivalent Quickpath is similar, but not even an open spec. PCIe is not an open spec either, but is controlled by a consortium that offers third parties pretty much equal opportunities to obtain a license.Someone correct me if I'm wrong, but I'm not sure if dramatically reduced CPU/GPU memory copy latency would be useful for graphics applications. Games seem to scale just fine with the PCIe. Obviously there will be specific cases where it will be useful, but in general, the industry hasn't had a problem getting huge speedups over CPUs without it.