Application Development Strategy: Provide Good Performance on GPUs

The GPU provides best performance on computations that can be run on numerous data objects simultaneously in parallel. This typically involves performing operations using large numbers (thousands or tens of thousands) of concurrent threads.

• Data must be transferred from the host to the GPU and back in order to employ it. These transfers are costly in terms of performance and should be minimized.
• The amount of work performed at each invocation of the GPU needs to justify the cost of moving data to and from the GPU. Code that transfers data for brief use by a small number of threads will see no performance benefit (more likely, an overall performance decrease).
• Data should be kept on the GPU as long as possible. As transfers should be minimized, applications that can perform multiple consecutive operations on the same data need to leave the data on the GPU for these subsequent operations. Application redesign will likely be needed to accomplish this goal.