C++ on icyveins7's blog

Avoid being baited by your printf statements in CUDA kernels

Mon, 23 Mar 2026 20:00:00 +0800

Behaviour of printf on device is not the same as on host!

If you’re a printf aficionado like me, then you use printf for debugging. A lot. In fact I previously wrote a small logger that uses printf called spfLogger. I do enjoy the flexibility of tuning every single width/precision with a few characters, and it’s something I haven’t yet seen C++ be able to emulate with as little irritation.

I haven’t gotten around to writing code using newer std::format or println yet, so the jury’s out on that.

Iterative algorithms with CUDA

Sat, 29 Nov 2025 20:00:00 +0800

Because not everything can be made un-iterative..

There’s a pretty large class of algorithms that have been developed for CPUs that involve, optimally, an iterative solver.

Usually, this is defined by some notion of convergence; some variable is updated, and then at the end of each iteration it is checked to determine whether further iterations are required.

This is all fine and dandy in CPU-land, but in GPUs this almost always makes the flow awkward.

Stumbling into a sweep line algorithm's edge case

Wed, 10 Sep 2025 20:00:00 +0800

Mistakes that you should learn from, lesson 1..

I recently wrote some code for interval merging. Every leetcoder can probably recite 15 algorithms to do this in their sleep, but I figured it out myself in this case, and in the process fell into an edge case.

Here’s to writing down your mistakes.

Interval merging

The generic interval merging problem provides you with a list of start/stop pairs which denote individual intervals. Each interval is then to be merged with any other overlapping intervals; an overlap is defined by at least 1 element being shared between 2 intervals.

Some tips for integrating small bits of CUDA code into larger codebases

Wed, 13 Aug 2025 20:00:00 +0800

Some lessons from general C++ come in handy here..

In some recent work I integrated some CUDA code I developed into a larger existing codebase, written for the CPU. Essentially, my module(s) would accelerate and replace some existing functionality, but was only a small cog in the machine.

This is likely applicable to many others, so hopefully the lessons I document here will be concisely useful to those who chance upon this post.

AtomicMinFloat; overloading integer-only atomics for floating-point numbers in CUDA

Sun, 16 Mar 2025 20:00:00 +0800

Convincing you (and myself) that with some minor edits, we can still use atomicMin for floats in CUDA..

CUDA has a set of atomic functions for safely updating the same memory address with different threads. There’s a whole list of them in the programming guide here.

However, for atomicMin (and also atomicMax), there isn’t an overload that works with floats (or doubles , but who uses those in CUDA anyway..). For this discussion we’ll just focus on atomicMin, but all the points we discuss can be inverted to explain atomicMax.

Gotta go (randomly) fast, thrust vs cuRAND

Mon, 10 Mar 2025 20:00:00 +0800

How fast can you generate (pseudo-)random numbers on the GPU? How fast can you generate (pseudo-)random numbers on the GPU?

I recently had to answer this question. Well, not exactly this question, but this was a key part of it.

A quick google search will bring up two common methods (libraries) when trying to do this in CUDA: thrust and cuRAND. Thrust is known to be a lot easier to set up; no need to write the kernel code and nitty gritty details, so I started with that.

When circumstances don’t allow you to use unique_ptr

Sun, 23 Feb 2025 20:00:00 +0800

Some background on my single header file memory manager and why I made it..

Picture this. You are working on a codebase with some ~~lazy~~ ~~inept~~ questionable decisions from external forces. You are given a class to work on. For the purposes of this post let’s call it MyClass.

The class will have its methods invoked in the following order:

MyClass() i.e. the constructor.
setup(…)
run(…)
teardown()
Repeat 2-4 many times.
~MyClass() i.e. the destructor.

This class will be reused in multiple different scenarios repeatedly, with different input data.

Cross-platform trigonometric SIMD and how the C ABI confused me

Thu, 30 Jan 2025 20:00:00 +0800

My small journey in discovering libmvec functions and the C problems I encountered..

I was recently working on making an existing Windows-only library of SIMD functions cross-platform i.e. making everything work in Linux. Here I’m going to highlight some problems I encountered in the process; hopefully it will help someone if they encounter something similar too.

The Windows (MSVC) function

Intrinsics are usually tied to compiler implementations. The one that gave me issues was a bunch of trigonometric functions. Let’s use the SSE version that works on floats; in MSVC this was implemented as a simple

Esoteric Errors: 'hidden symbol is referenced by DSO'

Fri, 18 Oct 2024 20:00:00 +0800

Hidden linkage isn’t something I’d encountered until now, so maybe this will help someone else too..

A new series of posts

I recently started work on a new codebase - one that is very large and has multiple moving components, from a frontend desktop UI (not my business) to the hardware interface (also not my business) and the backend processing (this one’s my business).

The standard build process had been setup on a remote server, and most people were ok with the process of connecting to it with tangible amounts of latency, but I was not. So my stubborn ass decided to figure out how to build it myself on a local machine. In the process, I encountered a ridiculous number of build errors; they were using custom Makefiles, but they had been generated by some other tool (possibly from Windows, even though it was being built in Linux, since there was a Windows VS solution as well).

Who knew a simple logger class would be this complicated?

Mon, 09 Sep 2024 20:00:00 +0800

Writing a printf-based C++ logger class was more of a journey than I originally thought..

I recently had to work with a codebase where the build process was so convoluted it couldn’t be run and debugged from within a Visual Studio instance, despite being a Visual Studio solution.

This was primarily because it had a bunch of Java components, which were mainly used for the UI, and that prevented it from being run as a standard C++ application within the debugger (or maybe you could? I couldn’t find a way..)

Some notes on 2D real-to-complex Fourier transforms

Mon, 15 Jul 2024 20:00:00 +0800

IPP in particular has some very niche ways of packing R2C DFT output, but otherwise there’s a few pointers here to keep in mind for how they are implemented in most libraries.

It’s pretty well known that the output of a Fourier transform of real inputs has symmetric properties. This is due to the fact that real waves consist of two conjugate pairs of complex exponentials.

What may be a bit less obvious (at least to me, when examining some programming libraries) is exactly how many useful output elements there are, and under which scenarios. In particular, I looked at IPP (which has some special packed structure), and cuFFT/NumPy/SciPy (which follow the FFTW structure I think).

CRTP, method chaining, and static polymorphism

Sat, 04 May 2024 20:00:00 +0800

Yes, it’s yet another blogpost about CRTP and how it’d be useful..

The 1st issue: method chaining

Have you seen a billion other blogposts about CRTP? Yes, so have I. But maybe there’s a reason for all of them; it wasn’t really apparent when reading them previously why it would be useful and/or why I would ever need it.

But recently, while writing some simple templated code for ufl, I had the bright idea of trying to make method chaining possible for the class. That’s when my templated class and its derived friend implementation started to fall apart. I fixed this by using CRTP for the first time.

Clang and Eigen's alternatives to complex multiplication SIMD

Mon, 11 Mar 2024 20:00:00 +0800

Clang isn’t much better than MSVC for complex number multiplication, while Eigen is equivalent to GCC but uses slightly different instructions.

Complex Number Multiplication Is Probably Not Common Enough

I think I might have bashed MSVC too much on the last post; trying to vectorise our simple 4-element complex number multiplication using clang with AVX instructions delivers similarly poor results:

#include <complex>

void cmul(
 const std::complex<float>* __restrict__ x,
 const std::complex<float>* __restrict__ y,
 std::complex<float>* __restrict__ z
){
 for (int i = 0; i < 4; ++i)
 z[i] = x[i] * y[i];
}

void mul(
 const float* __restrict__ x,
 const float* __restrict__ y,
 float* __restrict__ z
){
 for (int i = 0; i < 8; ++i)
 z[i] = x[i] * y[i];
}

cmul(std::complex<float> const*, std::complex<float> const*, std::complex<float>*): # @cmul(std::complex<float> const*, std::complex<float> const*, std::complex<float>*)
 vmovsd xmm0, qword ptr [rdi] # xmm0 = mem[0],zero
 vmovsd xmm1, qword ptr [rsi] # xmm1 = mem[0],zero
 vbroadcastss xmm2, xmm0
 vmovshdup xmm0, xmm0 # xmm0 = xmm0[1,1,3,3]
 vshufps xmm3, xmm1, xmm1, 225 # xmm3 = xmm1[1,0,2,3]
 vmulps xmm0, xmm3, xmm0
 vfmaddsub231ps xmm0, xmm1, xmm2 # xmm0 = (xmm1 * xmm2) +/- xmm0
 vmovlps qword ptr [rdx], xmm0
 vmovsd xmm0, qword ptr [rdi + 8] # xmm0 = mem[0],zero
 vmovsd xmm1, qword ptr [rsi + 8] # xmm1 = mem[0],zero
 vbroadcastss xmm2, xmm0
 vmovshdup xmm0, xmm0 # xmm0 = xmm0[1,1,3,3]
 vshufps xmm3, xmm1, xmm1, 225 # xmm3 = xmm1[1,0,2,3]
 vmulps xmm0, xmm3, xmm0
 vfmaddsub231ps xmm0, xmm1, xmm2 # xmm0 = (xmm1 * xmm2) +/- xmm0
 vmovlps qword ptr [rdx + 8], xmm0
 vmovsd xmm0, qword ptr [rdi + 16] # xmm0 = mem[0],zero
 vmovsd xmm1, qword ptr [rsi + 16] # xmm1 = mem[0],zero
 vbroadcastss xmm2, xmm0
 vmovshdup xmm0, xmm0 # xmm0 = xmm0[1,1,3,3]
 vshufps xmm3, xmm1, xmm1, 225 # xmm3 = xmm1[1,0,2,3]
 vmulps xmm0, xmm3, xmm0
 vfmaddsub231ps xmm0, xmm1, xmm2 # xmm0 = (xmm1 * xmm2) +/- xmm0
 vmovlps qword ptr [rdx + 16], xmm0
 vmovsd xmm0, qword ptr [rdi + 24] # xmm0 = mem[0],zero
 vmovsd xmm1, qword ptr [rsi + 24] # xmm1 = mem[0],zero
 vbroadcastss xmm2, xmm0
 vmovshdup xmm0, xmm0 # xmm0 = xmm0[1,1,3,3]
 vshufps xmm3, xmm1, xmm1, 225 # xmm3 = xmm1[1,0,2,3]
 vmulps xmm0, xmm3, xmm0
 vfmaddsub231ps xmm0, xmm1, xmm2 # xmm0 = (xmm1 * xmm2) +/- xmm0
 vmovlps qword ptr [rdx + 24], xmm0
 ret
mul(float const*, float const*, float*): # @mul(float const*, float const*, float*)
 vmovups ymm0, ymmword ptr [rsi]
 vmulps ymm0, ymm0, ymmword ptr [rdi]
 vmovups ymmword ptr [rdx], ymm0
 vzeroupper
 ret

Compiled with clang 18.1.0, with -O3 -ffast-math -march=x86-64-v3, this still refuses to vectorise the cmul function correctly, only using the xmm registers. I included the normal real-valued float multiplication to check that clang is indeed able to vectorise that. Note that you still need the __restrict__ keywords for the vectorisation to work. Using -ffast-math doesn’t seem to do anything for us in the real float vectorisation, but it does make the complex-valued vectorisation less verbose.

MSVC's terrible auto-vectoriser for AVX

Sat, 24 Feb 2024 20:00:00 +0800

MSVC has extremely lackluster auto-vectorisation, so I handrolled intrinsic calls by backtranslating GCC’s output.

The Motivation

I recently decided I wanted to spend some time understanding intrinsics and SIMD at a deeper level.

In developing code for my project ffs, I wanted to make sure that the code was running with at least AVX instructions (because that’s my target architecture, and honestly very few computers don’t have AVX these days..).

This led me down a path of discovery; first I discovered the amazing-ness that is godbolt.org, then I joined their discord, where I then asked for some help with understanding basic .asm compiler output.

Getting IPP to work on non-Intel chips

Mon, 19 Feb 2024 20:00:00 +0800

Intel Performance Primitives is not guaranteed to work on non-Intel chips, but there are some ways around it..

Over the last year or so I’ve written a C++ wrapper of header-only templates around a library known as Intel Performance Primitives (IPP). I often use this for its signal processing library, which is - at least by my measurements - one of the fastest, if not the fastest one around. It also has the benefit of having almost everything I need in one place: FFT/DFTs, math array processing, low-pass filtering etc. You can see my templates at the ipp_ext repository.