https://icyveins7.github.io/tags/cache/Recent content in Cache on icyveins7's blogHugoen-usThis work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.Wed, 02 Sep 2026 20:00:00 +0800https://icyveins7.github.io/posts/2026/09/large-pinned-host-allocations-in-cuda/Wed, 02 Sep 2026 20:00:00 +0800https://icyveins7.github.io/posts/2026/09/large-pinned-host-allocations-in-cuda/<blockquote> <p>Sometimes, you just can&rsquo;t fit everything in VRAM.</p> </blockquote> <p>In most CUDA projects, the advice is usually to transfer all inputs, outputs, and temporary scratch space to the device. Working completely inside VRAM is <em>fast</em>, and avoids both the complexity and the throughput hit that comes with over-PCIe transfers being scattered throughout your hot path.</p> <p>All of this is correct, and I adhere to this myself as much as possible. Sometimes, though, you simply just can&rsquo;t do this, and recently I was left with no choice and had to start migrating stuff back to host memory.</p>