Performance Optimization Using SIMD

Cloth simulation runs at 32 FPS with SIMD disabled.

Cloth simulation runs at 38 FPS with SIMD enabled.

What is SIMD (Single Instruction, Multiple Data)?

SIMD is a parallel processing method where a single instruction operates on multiple data elements simultaneously. This means the CPU can compute several data points at once, significantly increasing processing speed.

Performance Improvement Rates When Applying SIMD (Best Case)

SIMD vectorization delivers significant performance boosts across many domains:

• Vector and matrix operations: Using high-performance SIMD instructions like Intel AVX-512 can accelerate computations by 4 to over 16 times, commonly applied in scientific computing and image processing.
• Multimedia processing: Technologies such as Intel SSE, AVX, and ARM NEON provide 2 to 10 times speedup, often used to optimize media codecs.
• Physics simulation: For tasks like cloth simulation and collision detection, SIMD can yield 3 to 8 times acceleration.
• Machine learning inference: On mobile CPUs, NEON instructions combined with FP16 acceleration achieve 2 to 6 times performance improvement, especially beneficial in mobile environments.
• Game graphics optimization: Game engines leverage SIMD for physics and animation calculations, resulting in 3 to 10 times faster performance in many cases.

SIMD performance depends on factors such as processing width (128, 256, 512 bits), operation types, memory bandwidth, and data alignment. Proper alignment and padding of data to multiples of SIMD width are critical; otherwise, inefficient memory access and mismatched data sizes can degrade performance.

Factors Limiting SIMD Performance

Memory Bottlenecks

While SIMD instructions enable very fast parallel computation, performance is often limited by how quickly data can be fetched from and stored to memory. Frequent cache misses cause CPUs to stall waiting for data, negating SIMD benefits. Designing data layouts for good cache locality is essential.

Memory Alignment Issues

SIMD instructions usually operate efficiently on data aligned to fixed boundaries (e.g., 16, 32, 64 bytes). Misaligned data forces additional memory accesses or complex handling, resulting in performance drops. Proper memory alignment is a fundamental prerequisite for efficient SIMD usage.

Data Size and Padding

When the number of data elements isn’t a multiple of the SIMD width, leftover elements must be processed using scalar operations, reducing overall speed.

Branching and Conditionals

SIMD processes multiple data points in parallel. If each data element follows different branch paths, SIMD masks are used to select results, introducing overhead. High rates of branch mispredictions also flush CPU pipelines, harming performance.

Data Dependencies and Operation Order

SIMD excels when processing independent data in parallel. Strong dependencies between operations or required sequential processing limit parallelization, restricting SIMD’s effectiveness.

Small Data Sizes and Overheads

If data sets are too small, SIMD registers are underutilized. The overhead of function calls and SIMD setup can outweigh benefits, making SIMD counterproductive on tiny workloads.

Compiler Optimization Limits

Modern compilers auto-generate SIMD code but may struggle with complex or non-standard code patterns. Developers often need to manually optimize with explicit SIMD instructions beyond compiler automation.

Data Shuffle Costs

Data rearrangement (shuffle) within SIMD registers is expensive. Excessive shuffles degrade throughput, so minimizing unnecessary data movement in SIMD algorithms is important.

FP16 / Low Precision Arithmetic Limitations

FP16 operations reduce memory usage and accelerate computation but require hardware support. Without native FP16 hardware, software emulation slows performance. Lower precision may also introduce accuracy errors, requiring costly compensation.

Constraints and Considerations for SIMD Implementation

• SIMD fits vectorized operations well but struggles with conditional branching and data-dependent complex logic like collision processing.
• Data must be aligned and laid out to match SIMD widths (128, 256, 512 bits).
• Algorithms must minimize dependencies and be designed for parallelization/vectorization.

Introduction to Intel ISPC and Its Benefits

[ISPC Tutorial — Intel Developer Site, Official ISPC Website]

ISPC simplifies SIMD programming by allowing parallel processing using CPU vector instructions without manually writing complex SIMD code.

• SPMD Model: ISPC follows Single Program Multiple Data programming, running the same program on multiple data elements simultaneously, where each SIMD lane handles one element.
• Supported Architectures: Intel SSE, AVX, AVX2, AVX-512, ARM NEON SIMD
• Advantages: C-like syntax with a gentle learning curve Reduces SIMD programming complexity Compiler automatically generates optimized SIMD code Efficiently utilizes CPU SIMD vector operations
• Example ISPC code:

export void add_float_arrays(uniform float *a, uniform float *b, uniform float *result, uniform int count) {
    foreach (i = 0 ... count) {
        result[i] = a[i] + b[i];
    }
}

SIMD Instruction Sets and Supported Architectures

• SSE (Streaming SIMD Extensions) Architecture: Intel x86 family Vector size: 128-bit Simultaneous float operations: 4 (float32), 8 (float16) Representative supported devices: Intel Pentium 4 and later desktop/laptop CPUs Features: Early SIMD instruction set improved up to SSE4, providing basic parallel processing capabilities
• AVX (Advanced Vector Extensions) Architecture: Intel x86 family Vector size: 256-bit Simultaneous float operations: 8 (float32), 16 (float16) Representative supported devices: Intel Core i series (since 2011) Features: Registers twice as wide as SSE, enabling faster parallel processing
• AVX-512 Architecture: Intel x86 family Vector size: 512-bit Simultaneous float operations: 16 (float32), 32 (float16) Representative supported devices: Intel Xeon, Core i9 10th generation and later Features: Registers twice as wide as AVX, optimized for large-scale parallel processing
• NEON Architecture: ARM family Vector size: 128-bit Simultaneous float operations: 4 (float32), 8 (float16) Representative supported devices: Qualcomm Snapdragon, Samsung Exynos and other mobile SoCs Features: Mobile-friendly with high power efficiency, widely used in smartphones and tablets

FP16 Support Status in Chipsets

FP16 hardware support on desktop CPUs is very limited compared to server CPUs and GPUs. Intel and AMD’s latest generations offer partial FP16 acceleration, with full AVX-512 FP16 mainly on server processors. Most of Mobile CPUs support FP16 hardware acceleration using ARM NEON SIMD, especially on recent ARM-based chipsets.

• Intel Xeon Scalable (Ice Lake, Sapphire Rapids): Server-grade x86 CPUs with hardware support for AVX-512 FP16 operations Alder Lake, Raptor Lake: AVX-512 FP16 support mainly limited to server models; desktop versions provide software emulation or partial acceleration
• AMD EPYC Milan, Genoa: Server-grade x86 CPUs with hardware FP16 acceleration through AVX-512 and extended features Ryzen 7000 series: Hardware FP16 acceleration available on AVX-512 based server EPYC models
• ARM-based Chipsets Cortex-A78, Cortex-X2, Neoverse V1: ARMv8.2+ architecture with FP16 operations supported via NEON SIMD Apple Silicon M1, M2 series: Full hardware FP16 acceleration Qualcomm Snapdragon 8 Gen 2 and above: Hardware FP16 support Apple A15, A16 Bionic: Full hardware FP16 operations support
• GPU and AI Accelerators NVIDIA Ampere (A100), Hopper (H100): CUDA architecture GPUs with full FP16 support and Tensor Core acceleration NVIDIA RTX 30, 40 series: FP16 support including Tensor Cores Google TPU v3, TPU v4: AI-dedicated accelerators with full FP16 support

Example of SIMD Optimization Application

Data Alignment and Structuring : Memory access patterns are critical to maximizing SIMD performance. Store position, velocity, acceleration data in Structure of Arrays (SoA) format to enable efficient loading into SIMD registers.

Vectorization Example :

// AoS: Structure for a single point
struct Point {
    float x, y, z;   // Position
    float vx, vy, vz; // Velocity
};

// Array of N points in AoS format
Point points[N];

// SoA: Separate arrays for each field
struct PointsSoA {
    float* x;
    float* y;
    float* z;
    float* vx;
    float* vy;
    float* vz;
    int size;
    PointsSoA(int n) : size(n) {
        x = new float[n];
        y = new float[n];
        z = new float[n];
        vx = new float[n];
        vy = new float[n];
        vz = new float[n];
    }
    ~PointsSoA() {
        delete[] x; delete[] y; delete[] z;
        delete[] vx; delete[] vy; delete[] vz;
    }
};

// Conversion from AoS to SoA
void AoS_To_SoA(const Point* aos, PointsSoA& soa, int n) {
    for (int i = 0; i < n; ++i) {
        soa.x[i] = aos[i].x;
        soa.y[i] = aos[i].y;
        soa.z[i] = aos[i].z;
        soa.vx[i] = aos[i].vx;
        soa.vy[i] = aos[i].vy;
        soa.vz[i] = aos[i].vz;
    }
}

SIMD optimization example :

// Example of SIMD optimization (using SSE/AVX)
void updatePositions(PointsSoA& soa, float dt, int n) {
    int i = 0;
    for (; i <= n - 8; i += 8) {
        // Load vectorized data into SIMD registers
        __m256 px = _mm256_load_ps(&soa.x[i]);
        __m256 py = _mm256_load_ps(&soa.y[i]);
        __m256 pz = _mm256_load_ps(&soa.z[i]);
        __m256 vx = _mm256_load_ps(&soa.vx[i]);
        __m256 vy = _mm256_load_ps(&soa.vy[i]);
        __m256 vz = _mm256_load_ps(&soa.vz[i]);
        __m256 dt_vec = _mm256_set1_ps(dt);
        // SIMD operation (position += velocity * dt)
        px = _mm256_fmadd_ps(vx, dt_vec, px);
        py = _mm256_fmadd_ps(vy, dt_vec, py);
        pz = _mm256_fmadd_ps(vz, dt_vec, pz);
        // Store results
        _mm256_store_ps(&soa.x[i], px);
        _mm256_store_ps(&soa.y[i], py);
        _mm256_store_ps(&soa.z[i], pz);
    }
    // Process remaining elements after vectorized loop
    for (; i < n; ++i) {
        soa.x[i] += soa.vx[i] * dt;
        soa.y[i] += soa.vy[i] * dt;
        soa.z[i] += soa.vz[i] * dt;
    }
}

Example of Code for AVX-512 FP16

When FP16 (16-bit floating point) is supported, it is possible to compute 32 half-precision floats simultaneously. In AVX-512 instructions, different functions are used depending on the precision of the operands: functions with the suffix _ps are for 32-bit floats, and functions with _ph are for 16-bit half floats.

For example, for float operations, _mm512_add_ps is used, and for half-precision operations, _mm512_add_ph is used.

Since the C++ standard does not define a native 16-bit floating point type, uint16_t is used to store data, and hardware instructions or library functions are used for conversion and computation.

When using half-precision floats, position changes within a 1-meter range can be distinguished roughly at 1-millimeter accuracy, but in a 10-meter range, the error can be about 1 centimeter, so half-precision floats are not suitable for representing movement or positions in large spaces.

#include <immintrin.h> // AVX-512 (Intel)

void updatePositionsFP16_AVX512(uint16_t* positions, const uint16_t* velocities, float deltaTime, int count)
{
    int i = 0;
    int totalCount = count * 3;               // total number of float16 elements (x, y, z)
    int simdCount = (totalCount / 32) * 32;  // align to multiple of 32 (SIMD batch)
    int remainderCount = totalCount - simdCount; // remainder for scalar processing
    __m512h deltaTimeVec = _mm512_set1_ph(_cvtss_sh(deltaTime, 0));

    // SIMD loop (AVX-512 FP16)
    for (; i < simdCount; i += 32)
    {
        __m512h pos = _mm512_loadu_ph(&positions[i]);        // load 32 half floats
        __m512h vel = _mm512_loadu_ph(&velocities[i]);
        __m512h deltaVel = _mm512_mul_ph(vel, deltaTimeVec); // velocity * deltaTime
        pos = _mm512_add_ph(pos, deltaVel);                   // position + delta velocity
        _mm512_storeu_ph(&positions[i], pos);
    }
    // Scalar remainder processing
    for (; i < totalCount; ++i)
    {
        float p = _cvtsh_ss(positions[i]);
        float v = _cvtsh_ss(velocities[i]);
        p += v * deltaTime;
        positions[i] = _cvtss_sh(p, 0);
    }
}

void updatePositions_AVX512(float* positions, const float* velocities, float deltaTime, int count)
{
    int i = 0;
    int totalCount = count * 3;
    int simdCount = (totalCount / 16) * 16;  // 512 bits / 32 bits = 16 floats
    __m512 deltaTimeVec = _mm512_set1_ps(deltaTime);

    for (; i < simdCount; i += 16)
    {
        __m512 pos = _mm512_loadu_ps(&positions[i]);
        __m512 vel = _mm512_loadu_ps(&velocities[i]);
        __m512 deltaVel = _mm512_mul_ps(vel, deltaTimeVec);
        pos = _mm512_add_ps(pos, deltaVel);
        _mm512_storeu_ps(&positions[i], pos);
    }
    // Scalar remainder
    for (; i < totalCount; ++i)
    {
        positions[i] += velocities[i] * deltaTime;
    }
}

SIMD in Unreal Engine

• Unreal Engine supports various SIMD instruction sets (SSE, AVX, NEON, etc.) depending on the platform.
• Vector and matrix classes such as FVector, FVector4, FMatrix are vectorized to leverage SIMD operations.
• Inside the engine, SIMD optimizations are applied in systems like Niagara particle simulation and Cloth simulation.
• Example code snippet:

#include "Math/VectorRegister.h"
#include "Math/UnrealMathVectorCommon.h"

void UpdatePositionsSIMD(FClothPointsSoA& Points, float DeltaTime)
{
    const int32 NumPoints = Points.Num();
    const int32 VectorSize = 4;  // 4 floats per VectorRegister
    VectorRegister DeltaTimeVec = VectorLoadFloat1(&DeltaTime);
    int32 i = 0;

    for (; i <= NumPoints - VectorSize; i += VectorSize)
    {
        VectorRegister XVec = VectorLoad(&Points.X[i]);
        VectorRegister YVec = VectorLoad(&Points.Y[i]);
        VectorRegister ZVec = VectorLoad(&Points.Z[i]);
        VectorRegister VXVec = VectorLoad(&Points.VX[i]);
        VectorRegister VYVec = VectorLoad(&Points.VY[i]);
        VectorRegister VZVec = VectorLoad(&Points.VZ[i]);

        // position += velocity * deltaTime
        XVec = VectorMultiplyAdd(VXVec, DeltaTimeVec, XVec);
        YVec = VectorMultiplyAdd(VYVec, DeltaTimeVec, YVec);
        ZVec = VectorMultiplyAdd(VZVec, DeltaTimeVec, ZVec);

        VectorStore(XVec, &Points.X[i]);
        VectorStore(YVec, &Points.Y[i]);
        VectorStore(ZVec, &Points.Z[i]);
    }
    // Scalar remainder
    for (; i < NumPoints; ++i)
    {
        Points.X[i] += Points.VX[i] * DeltaTime;
        Points.Y[i] += Points.VY[i] * DeltaTime;
        Points.Z[i] += Points.VZ[i] * DeltaTime;
    }
}

SIMD Optimization in Cloth Simulation

Cloth simulation is a core element for realistic character immersion and natural movement in games and real-time visual effects. However, simulating cloth physics requires heavy computation, and optimization is essential for real-time performance.

SIMD (Single Instruction Multiple Data) is a CPU instruction set that can process multiple data points in parallel with a single instruction. Typically, SIMD can perform 4 to 32 float operations simultaneously, making it very effective for vector operation performance improvements.

For example, in the loop calculating spring constraints, loading 4 or 8 point positions at once and computing them in parallel drastically reduces computation time.

Computational Bottleneck in Cloth Simulation

Most cloth simulations use a point-based physics model. The cloth consists of thousands of points, each holding position, velocity, acceleration, etc. These points connect like springs to form physical constraints, which produce realistic cloth movement.

Every frame involves:

• Updating each point’s position and velocity
• Computing and resolving spring constraints between connected points
• Handling collision detection and response

These computations are mostly vector math repeated for all points, making the structure highly suitable for parallelization.

Results

In our cloth simulator development, I’m progressively applying SIMD optimization. In the main bottleneck loop, I confirmed about a 30% frame rate improvement after SIMD was applied.

This allows simulating more cloth in real-time while maintaining simulation quality.

Before/After SIMD code comparison :

// Non-SIMD code snippets
float ErrorLength = currentLength - LinkBaseLength;

// Disassembly for Non-SIMD code
// xmm0 = currentLength
00007FFCC2DAE732  movss       xmm0,dword ptr [rsp+64h]    
// xmm0 -= LinkBaseLength
00007FFCC2DAE738  subss       xmm0,dword ptr [rsp+130h]  
// ErrorLength = xmm0
00007FFCC2DAE741  movss       dword ptr [rsp+220h],xmm0  

// SIMD code snippets
VectorRegister errorLengthx4 = VectorSubtract(currentLengthx4, linkBaseLengthx4);

// Disassembly for SIMD code
// xmm0 = currentLengthx4 (4 floats)
00007FFCC2DB6166  movaps      xmm0,xmmword ptr [rsp+1210h]  
// xmm0 -= linkBaseLengthx4 (4 floats)
00007FFCC2DB616E  subps       xmm0,xmmword ptr [rsp+1220h]  
// save as errorLengthx4 (4 floats)
00007FFCC2DB6176  movaps      xmmword ptr [rsp+1230h],xmm0  
00007FFCC2DB617E  movaps      xmm0,xmmword ptr [rsp+1230h]  
00007FFCC2DB6186  movaps      xmmword ptr [rsp+1240h],xmm0  
00007FFCC2DB618E  movaps      xmm0,xmmword ptr [rsp+1240h]  
00007FFCC2DB6196  movaps      xmmword ptr [rsp+800h],xmm0 

Cloth simulation runs at 32 FPS with SIMD disabled.

Cloth simulation runs at 38 FPS with SIMD enabled.

Demo : https://x.com/3devnote/status/1926157530494902532