Nodepp GPU: Accelerating C++ with GPU and Nodepp

In the past, the Graphics Processing Unit (GPU) was primarily seen as a specialized piece of hardware for rendering images and videos on a screen. However, its architecture, built on thousands of parallel cores, makes it an incredibly powerful tool for a much wider range of computational tasks. This is known as General-Purpose computing on Graphics Processing Units (GPGPU). This article will walk you through a simple C++ example of using a GPU to perform element-wise matrix multiplication, showcasing how this powerful technology can be applied to common programming problems for significant performance gains. We will explore the **Nodepp** framework and a custom Nodepp-gpu library that simplifies the process, allowing developers to harness the parallel processing capabilities of modern GPUs without needing to write low-level graphics code.

Why Use a GPU for Computation?

Before diving into the code, it’s essential to understand the fundamental difference between a Central Processing Unit (CPU) and a GPU. A CPU is optimized for sequential tasks, processing a few instructions very quickly. A GPU, on the other hand, is built with thousands of smaller, more efficient cores designed to handle many simple tasks simultaneously. This architecture makes GPUs perfectly suited for highly parallelizable problems, such as processing image pixels, training machine learning models, or, as we will see, performing matrix operations.

A task like multiplying two large matrices, where each element’s calculation is independent of the others, is an ideal candidate for GPU acceleration. Instead of a CPU processing each multiplication one by one, a GPU can perform all multiplications at the same time.

[embed]GitHub - NodeppOfficial/nodepp-gpu: Nodepp GPU: A C++ GPU compute layer for high-performance image… Nodepp GPU: A C++ GPU compute layer for high-performance image processing, leveraging Raylib and GLSL shaders for…github.com

Dissecting the Code

The following C++ code uses the Nodepp framework and a custom Nodepp-gpu library to harness the power of a GPU.

The first step is to start the GPU machine. This initializes the necessary hardware and software environment for GPU computation. If this fails, the program will throw an error, as the rest of the operations depend on it.

if( !gpu::start_machine() ) 
  { throw except_t("Failed to start GPU machine"); }

The core of the program is the GPU kernel. This is a small program, written in a GLSL-like language, that defines the work to be done for each parallel thread. Think of it as a function that will be executed for every single element in our output matrix simultaneously. The kernel retrieves the values from the input matrices at the current coordinate (uv), multiplies them, and returns the result.

gpu::gpu_t gpu ( GPU_KERNEL(
    vec2 idx = uv / vec2( 2, 2 );
    float color_a = texture( image_a, idx ).x;
    float color_b = texture( image_b, idx ).x;
    float color_c = color_a * color_b;
    return vec4( vec3( color_c ), 1. );
));

Next, we define our input matrices. In this example, we create two 2times2 matrices with floating-point values using the gpu::matrix_t class.

gpu::matrix_t matrix_a( 2, 2, ptr_t<float>({
    10., 10.,
    10., 10.,
}) );

gpu::matrix_t matrix_b( 2, 2, ptr_t<float>({
    .1, .2,
    .4, .3,
}) );

Before execution, we need to configure the GPU instance. We specify the dimensions and format of the output matrix and then bind our input matrices to the kernel’s named inputs ("image_a" and "image_b").

gpu.set_output(2, 2, gpu::OUT_DOUBLE4);
gpu.set_input (matrix_a, "image_a");
gpu.set_input (matrix_b, "image_b");

Finally, we execute the kernel by calling gpu(). This triggers the parallel computation on the GPU. We then retrieve the resulting data and print each element to the console. The expected output would be the element-wise product of the two matrices.

for( auto x: gpu().data() ) 
   { console::log(x); }

The Broader Impact

While this example uses small 2 times 2 matrices, the true power of this approach becomes evident with larger data sets. For a 1000 times 1000 matrix, a CPU would perform one million multiplications in a sequential loop. A GPU, with its parallel architecture, can perform these calculations in a fraction of the time. This is the core principle behind high-performance computing in fields like scientific research, data analytics, and machine learning.

An Asynchronous Approach to GPU Computation

While the previous example demonstrates a synchronous workflow, many real-world applications require processing data from external sources, such as a web server. In these cases, an asynchronous approach is far more efficient, as it allows the program to continue executing other tasks while waiting for data to arrive.

The Nodepp framework provides a powerful way to handle these asynchronous operations. In the following example, an image is fetched from a URL, and once the data is fully received, a GPU-accelerated convolution filter is applied to it.

#include <nodepp/nodepp.h>
#include <nodepp/https.h>
#include <gpu/gpu.h>

using namespace nodepp;

void gpu_convolution( string_t data, string_t ext ){

    gpu::gpu_t gpu( GPU_KERNEL( // Define a GPU kernel

        vec2 pixel_size= 1.0 / size; // Calculate pixel size for sampling
        vec4 sum = vec4( 0.0 );      // Accumulator for convolution
        vec2 uv_norm = uv / size;    // Normalized UV coordinates

        for( int y=0; y<3; y++ ){    // Loop through 3x3 filter rows
        for( int x=0; x<3; x++ ){    // Loop through 3x3 filter columns

            // Calculate centered image offset
            vec2 image_offset = vec2(x, y) - 1.0;
            vec2 image_coord  = uv_norm + image_offset*pixel_size;
            vec3 image_val    = texture( image, image_coord ).xyz;

            // Calculate filter texture coordinates
            vec2 fltr_coord = (vec2(x, y) + 0.5) / 3.0;
            vec3 fltr_val   = texture( fltr, fltr_coord ).xyz;

            // Multiply image and filter values, add to sum
            sum += vec4( image_val * fltr_val, 0.0 );

        }}

        // Remap convolution result for visualization
        float remapped_x = (sum.x + 2.0) / 4.0;

        // Clamp result to [0, 1] range
        remapped_x = clamp(remapped_x, 0.0, 1.0);

        // Return final grayscale color
        return vec4( remapped_x, remapped_x, remapped_x, 1.0 );
    ));

    gpu::matrix_t image ( data, ext );          // Load input image
    gpu::matrix_t filter( 3, 3, ptr_t<float>({ // Create a 3x3 filter matrix (vertical edge detector)
        1., 0., -1.,
        1., 0., -1.,
        1., 0., -1.,
    }));

    gpu.set_output( image.width(), image.height(), gpu::OUT_UCHAR4 ); // Set output dimensions and format
    gpu.set_input ( filter, "fltr"  ); // Bind filter matrix to kernel
    gpu.set_input ( image , "image" ); // Bind image matrix to kernel
    gpu.set_input ( gpu::uvec2_t({     // Pass image size as a uniform
        image.width (),
        image.height()
    }), "size" );

    gpu::save_canvas( gpu().get(), "output.png" ); // Execute kernel and save output

}

void onMain(){

    gpu::start_machine();

    fetch_t args; ssl_t ssl;
    args.url     = "https://deep-image.ai/blog/content/images/size/w1600/2022/08/magic-g1db898374_1920.jpg";
    args.headers = header_t({ { "Host", url::host(args.url) } });
    args.method  = "GET";

    https::fetch( args, &ssl )

    .then([=]( https_t cli ){
        auto data = stream::await( cli );
        gpu_convolution( data, "jpg" );
    })

    .fail([=]( except_t err ){
        console::error( err.data() );
    });

    // gpu::stop_machine(); nodepp automaticaly stop machine at close

}

In this code, https::fetch initiates an asynchronous request. The .then() block is executed only after the request is successful and the entire data stream is awaited. This prevents the program from freezing while waiting for the data, a crucial aspect of high-performance and responsive applications.

original image

post-processed image

[embed]GitHub - NodeppOfficial/nodepp-gpu: Nodepp GPU: A C++ GPU compute layer for high-performance image… Nodepp GPU: A C++ GPU compute layer for high-performance image processing, leveraging Raylib and GLSL shaders for…github.com

Conclusion

The use of GPUs for general-purpose computing is a game-changer that has redefined the possibilities of modern applications. By offloading parallel tasks to the GPU, we can achieve significant performance gains that are simply not possible with a traditional CPU-only approach. This fundamental shift in computing is driving innovation across a multitude of fields.

For developers, frameworks like Nodepp and Nodepp-gpu are crucial, as they provide a high-level, accessible entry point into GPGPU. They abstract away the complexities of low-level graphics APIs and hardware-specific code, allowing programmers to focus on the logic of their parallel algorithms. This opens the door for a new generation of high-performance applications, from real-time data processing and scientific simulations to machine learning and artificial intelligence. The C++ code discussed here serves as a clear and concise blueprint for how to leverage this powerful technology. As hardware continues to evolve, the integration of GPGPU into everyday programming will only become more seamless and essential for building the next generation of powerful and efficient software.

Thanks for reading! If you enjoy reading this post, got help, knowledge, inspiration, and motivation through it. And if you want to support me — you can **“buy me a coffee.”** Your support really makes a difference ❤️