This repository has been archived on 2026-08-19. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
ROCm-Research-Archive/README.md
T
2026-08-20 00:45:43 +02:00

26 KiB

This was my try on implementing ROCm to the BC250. It is discontinued as of March 22th 2026 (Latest Commit). This Repository is a sanitized re-upload of my findings based off of Dani's research including the Patches that are now applyed to project-ariel. This Repository is now a public Archive under the GPL-2.0 License and can be used as such.

ROCm on AsRock BC-250 — Complete Installation Guide

Hardware: AsRock BC-250 (AMD Cyan Skillfish / gfx1013)

Target ROCm: 7.2.0 | OS: CachyOS (Arch-based) | Shell: fish / bash

Bootloader: Limine | Compiler: Clang (LLVM)

Author: Fabian | Last Updated: 2026-03-17

Honorable Mention: Dani (Creator of first KIQ Timeout fix)

This is not working Standalone.


Table of Contents

  1. Hardware Overview
  2. Prerequisites
  3. Step 1 — Install ROCm Packages
  4. Step 2 — User Group Configuration
  5. Step 3 — Environment Variables
  6. Step 4 — Kernel Boot Parameters
  7. Step 5 — Modprobe Configuration
  8. Step 6 — Build Patched amdgpu Kernel Module
  9. Step 7 — Install Module & Reboot
  10. Step 8 — Post-Reboot Verification
  11. Step 9 — HIP Compute Tests
  12. Cyan Skillfish Governor (GPU Clock Management)
  13. GPU Watchdog (Safety Monitor)
  14. HIP Programming Guidelines for BC-250
  15. Kernel Module Rebuild After Updates
  16. Technical Reference — Why This GPU Needs Patches
  17. Troubleshooting
  18. File Inventory

1. Hardware Overview

Text Only
GPU:              AMD BC-250 (Cyan Skillfish)
Device ID:        0x13FE (Vendor: 0x1002 AMD)
Architecture:     RDNA 1.5 (GFX 10.1.3 / gfx1013)
ROCm Target:      gfx10-1-generic (auto-mapped), runs as gfx1010 via override
Compute Units:    24 (reported by some tools as 12 due to SIMD config)
Wavefront Size:   32 (RDNA-style)
Memory:           ~14.75 GB shared system GDDR6 (NO dedicated VRAM)
Memory Type:      APU-style unified memory (system RAM shared with CPU)
PCIe:             Internal SoC fabric (Completion Timeout: Not Supported)

Why This GPU Is Special

The BC-250 is a cryptocurrency mining board repurposed as a compute accelerator. It behaves like an APU — there is no dedicated VRAM; all GPU memory operations use system RAM. This has critical implications:

Behavior Explanation
No standard hipMalloc Standard device memory allocation targets non-existent VRAM
hipMallocManaged required Unified memory works on shared RAM
hipHostMalloc works Pinned host memory is safe for APU
GPU reset = system crash Resetting the GPU corrupts shared system RAM
SDMA engine unreliable Must disable via HSA_ENABLE_SDMA=0
GFXOFF is fatal GPU enters power-save and can't wake — CPU hangs on MMIO read
KIQ ring is broken KIQ commands cause fence timeouts → system hang

2. Prerequisites

  • CachyOS (or Arch-based distro) with a working internet connection
  • sudo access for the installation user
  • paru or another AUR helper (for the GPU governor)
  • Sufficient disk space (~2 GB for kernel source + build artifacts)

Install base build dependencies:

Fish
sudo pacman -S --needed --noconfirm base-devel bc python

3. Step 1 — Install ROCm Packages

Install all 16 required ROCm 7.2.0 packages:

Fish
sudo pacman -S --needed --noconfirm \
  rocm-core hsa-rocr rocminfo rocm-smi-lib rocm-device-libs \
  rocm-llvm comgr hip-runtime-amd rocm-hip-runtime \
  rocm-opencl-runtime rocblas hipblas rocrand rocm-cmake \
  rocm-language-runtime hipblas-common

You can also run the provided script:

Fish
bash scripts/01_install_rocm_packages.sh

Package List

Package Description
rocm-core ROCm core (version files)
hsa-rocr HSA Runtime API
rocminfo ROCm system info tool
rocm-smi-lib ROCm SMI library
rocm-device-libs ROCm device libraries
rocm-llvm ROCm LLVM/Clang compiler (~4.5 GB)
comgr AMDGPU Code Object Manager
hip-runtime-amd HIP Runtime (AMD backend)
rocm-hip-runtime Meta-package for HIP runtime
rocm-opencl-runtime ROCm OpenCL runtime
rocblas ROCm BLAS library
hipblas ROCm BLAS marshalling library
hipblas-common hipBLAS common files
rocrand ROCm random number generator
rocm-cmake ROCm CMake modules
rocm-language-runtime ROCm language runtime meta

4. Step 2 — User Group Configuration

Add your user to the render and video groups:

Fish
sudo usermod -aG render,video $USER

You must log out and back in (or reboot) for group changes to take effect.

Verify:

Fish
groups $USER
# Should include: render video


5. Step 3 — Environment Variables

The BC-250 requires specific environment variables for ROCm to function. With the v3 kernel patches (Step 6), only a minimal set is needed.

For fish shell

Add to ~/.config/fish/config.fish:

Fish
# === ROCm Configuration for AMD BC-250 ===
set -gx PATH /opt/rocm/bin $PATH
set -gx LD_LIBRARY_PATH /opt/rocm/lib $LD_LIBRARY_PATH
set -gx ROCM_PATH /opt/rocm
set -gx HSA_OVERRIDE_GFX_VERSION 10.1.0
set -gx HIP_VISIBLE_DEVICES 0
set -gx HSA_ENABLE_SDMA 0
set -gx HSA_TOOLS_LIB ""
set -gx HSA_TOOLS_REPORT_LOAD_FAILURE 0

For bash shell

Add to ~/.bashrc:

Bash
# === ROCm Configuration for AMD BC-250 ===
export PATH="/opt/rocm/bin:$PATH"
export LD_LIBRARY_PATH="/opt/rocm/lib:$LD_LIBRARY_PATH"
export ROCM_PATH=/opt/rocm
export HSA_OVERRIDE_GFX_VERSION=10.1.0
export HIP_VISIBLE_DEVICES=0
export HSA_ENABLE_SDMA=0
export HSA_TOOLS_LIB=""
export HSA_TOOLS_REPORT_LOAD_FAILURE=0

You can also run the provided script:

Fish
bash scripts/02_configure_environment.sh

Variable Reference

Variable Value Why Required
HSA_OVERRIDE_GFX_VERSION 10.1.0 Maps gfx1013 → gfx1010 (closest supported target)
HIP_VISIBLE_DEVICES 0 Explicit GPU device selection
HSA_ENABLE_SDMA 0 SDMA engine is broken on gfx1013, use shader DMA
HSA_TOOLS_LIB "" Prevent profiling tools from destabilizing GPU
HSA_TOOLS_REPORT_LOAD_FAILURE 0 Suppress tool load warnings

Variables You Must NOT Set

These were pre-v3 workarounds that severely hurt performance:

Variable Why It's Harmful
GPU_MAX_HW_QUEUES=1 Serializes all GPU ops to 1 queue — severe slowdown
HIP_LAUNCH_BLOCKING=1 Forces synchronous kernel launches — prevents pipelining
HSA_DISABLE_FRAGMENT_ALLOCATOR=1 Not needed with v3 patches

6. Step 4 — Kernel Boot Parameters

Limine Bootloader

Edit /etc/default/limine and add these parameters to the KERNEL_CMDLINE:

Text Only
amdgpu.gpu_recovery=1 amdgpu.noretry=0 amdgpu.dc=0 amdgpu.lockup_timeout=120000 amdgpu.ppfeaturemask=0xfff73ef7

Apply changes:

Fish
sudo limine-update

GRUB Bootloader (if applicable)

Edit /etc/default/grub and add to GRUB_CMDLINE_LINUX_DEFAULT:

Text Only
amdgpu.gpu_recovery=1 amdgpu.noretry=0 amdgpu.dc=0 amdgpu.lockup_timeout=120000 amdgpu.ppfeaturemask=0xfff73ef7

Apply changes:

Bash
sudo grub-mkconfig -o /boot/grub/grub.cfg

Parameter Reference

Parameter Value Purpose
amdgpu.gpu_recovery=1 Enabled CRITICAL: Auto-recover from GPU hangs
amdgpu.noretry=0 Retry enabled Allow page fault retry (required for shared memory)
amdgpu.dc=0 Display disabled Disable display controller (headless — prevents IRQ errors)
amdgpu.lockup_timeout=120000 120 seconds Time before declaring GPU hung (allows heavy compute)
amdgpu.ppfeaturemask=0xfff73ef7 Custom mask Disable GFXOFF + DeepSleep + ULV (see below)

ppfeaturemask Calculation

Text Only
Default value:  0xfff7bfff
BC-250 value:   0xfff73ef7

Bits disabled:
  Bit 15 (0x8000) — PP_GFXOFF_MASK:          GPU enters unrecoverable power-save
  Bit  8 (0x0100) — PP_ULV_MASK:             Ultra-low voltage may destabilize GPU
  Bit  3 (0x0008) — PP_SCLK_DEEP_SLEEP_MASK: Deep clock sleep prevents GPU wake

7. Step 5 — Modprobe Configuration

Copy the provided config file:

Fish
sudo cp config/amdgpu.conf /etc/modprobe.d/amdgpu.conf

Or create manually — file /etc/modprobe.d/amdgpu.conf:

Text Only
# AMD BC-250 (Cyan Skillfish / gfx1013) — ROCm Stability Parameters
# noretry=0              — Allow page fault retry (critical for shared memory / APU)
# gpu_recovery=1         — Enable GPU recovery on timeout
# sched_hw_submission=2  — Limit concurrent HW submissions (prevent queue overload)
# ppfeaturemask=0xfff73ef7 — Disable GFXOFF (bit 15), DeepSleep (bit 3), ULV (bit 8)
options amdgpu noretry=0 gpu_recovery=1 sched_hw_submission=2 ppfeaturemask=0xfff73ef7

8. Step 6 — Build Patched amdgpu Kernel Module

This is the most critical step. The BC-250 has a hardware/firmware bug where the KIQ (Kernel Interface Queue) ring causes fatal GPU hangs. The v3 patch applies 8 surgical modifications across 3 kernel source files to bypass KIQ and add dead-GPU detection.

Why a Custom Module is Needed

The stock amdgpu driver routes TLB (Translation Lookaside Buffer) invalidation commands through the KIQ ring. On the BC-250, this causes:

  1. KIQ fence timeouts → GPU becomes unresponsive
  2. GFXOFF power-save → GPU can't wake, CPU hangs on MMIO read (no PCIe timeout)
  3. System freeze → No recovery possible, hard power-off required

The v3 patch has three protection layers: - Layer 1: Disable GFXOFF for Cyan Skillfish (gfx_v10_0.c) - Layer 2: KIQ bypass + dead-GPU detection (gmc_v10_0.c, amdgpu_gmc.c) - Layer 3: Boot parameters disable GFXOFF/DeepSleep/ULV (Step 4)

CRITICAL: Use CachyOS Kernel Source (Not Vanilla!)

You MUST build from the CachyOS kernel source, not vanilla kernel.org source. CachyOS patches their kernel with struct layout changes that are incompatible with vanilla source headers. Building from vanilla kernel source will produce a module that causes a NULL pointer dereference at boot in drm_vma_offset_add during gmc_v10_0_sw_init.

Build Process

The provided build script handles everything:

Fish
# Edit scripts/04_build_patched_module.sh and verify your kernel version first!
bash scripts/04_build_patched_module.sh

Option B: Manual Step-by-Step

1. Identify Your Kernel Version
Fish
uname -r
# Example output: 6.19.6-2-cachyos

Note both the kernel version (e.g., 6.19.6) and the full string (e.g., 6.19.6-2-cachyos).

2. Find the Matching CachyOS Source

Go to https://github.com/CachyOS/linux/releases and find the release matching your kernel.

For example, kernel 6.19.6-2-cachyos → release cachyos-6.19.6-1 or similar.

Fish
mkdir -p ~/kernel-build
cd ~/kernel-build

# Download (replace URL with your matching version)
curl -L -o cachyos-source.tar.gz \
  "https://github.com/CachyOS/linux/archive/refs/tags/YOUR_TAG.tar.gz"

# Extract
tar xf cachyos-source.tar.gz
cd linux-*  # or whatever the extracted directory is named
3. Prepare Build Environment
Fish
set KVER (uname -r)

# Copy the running kernel's config and build files
cp /usr/lib/modules/$KVER/build/.config .
cp /usr/lib/modules/$KVER/build/Module.symvers .

# Copy localversion files (these set the kernel version string)
cp /usr/lib/modules/$KVER/build/localversion.* . 2>/dev/null

# Prepare the build system
make LLVM=1 olddefconfig
make LLVM=1 modules_prepare

Note: LLVM=1 is mandatory — CachyOS kernels are compiled with Clang, not GCC. Using GCC will produce an incompatible module.

4. Apply the Patches

Copy the three patch scripts from this guide's patches/ directory to the BC-250, then run them:

Fish
set AMDGPU_DIR (pwd)/drivers/gpu/drm/amd/amdgpu

python3 patches/patch1_gfxoff.py $AMDGPU_DIR
python3 patches/patch2_gmc.py $AMDGPU_DIR
python3 patches/patch3_amdgpu_gmc.py $AMDGPU_DIR

Verify the patches were applied:

Fish
grep -c "BC-250" $AMDGPU_DIR/gfx_v10_0.c    # Should be ≥ 4
grep -c "BC-250" $AMDGPU_DIR/gmc_v10_0.c     # Should be ≥ 14
grep -c "BC-250" $AMDGPU_DIR/amdgpu_gmc.c    # Should be ≥ 9

5. Build the Module
Fish
make LLVM=1 -j(nproc) M=drivers/gpu/drm/amd/amdgpu modules

This takes 3-10 minutes depending on CPU. The output should end with:

Text Only
LD [M]  drivers/gpu/drm/amd/amdgpu/amdgpu.ko

6. Strip and Compress
Fish
# Strip debug symbols: ~600MB → ~30MB
strip --strip-debug drivers/gpu/drm/amd/amdgpu/amdgpu.ko

# Compress: ~30MB → ~4.5MB
zstd -19 -f drivers/gpu/drm/amd/amdgpu/amdgpu.ko

9. Step 7 — Install Module & Reboot

1. Backup the Original Module
Fish
set KVER (uname -r)
set MODULE_DIR /usr/lib/modules/$KVER/kernel/drivers/gpu/drm/amd/amdgpu

sudo cp $MODULE_DIR/amdgpu.ko.zst $MODULE_DIR/amdgpu.ko.zst.original
2. Install the Patched Module
Fish
sudo cp drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst $MODULE_DIR/amdgpu.ko.zst
sudo depmod -a
3. Rebuild Initramfs

For Limine:

Fish
sudo limine-update

For GRUB/mkinitcpio:

Fish
sudo mkinitcpio -P

4. Verify Before Reboot
Fish
# Check the installed module has all 9 BC-250 strings
zstd -d -c $MODULE_DIR/amdgpu.ko.zst | strings | grep "BC-250"

Expected output (9 strings):

Text Only
BC-250: GFXOFF disabled to prevent GPU power-state hangs
BC-250: GPU unreachable (MMIO returned 0xFFFFFFFF)...
BC-250: GPU died during sem acquire (0xFFFFFFFF)
BC-250: GPU died during TLB flush ACK wait (0xFFFFFFFF)
BC-250: GPU unreachable in fw_reg_write_reg_wait...
BC-250: GPU died during reg_write_reg_wait (0xFFFFFFFF)
BC-250 KIQ bypass active
BC-250 KIQ bypass active in fw_reg_write_reg_wait
BC-250: MMIO reg write/wait timeout

5. Reboot
Fish
sudo reboot
Emergency Rollback

If the system doesn't boot or the GPU crashes, restore the original module:

Fish
set KVER (uname -r)
set MODULE_DIR /usr/lib/modules/$KVER/kernel/drivers/gpu/drm/amd/amdgpu

sudo cp $MODULE_DIR/amdgpu.ko.zst.original $MODULE_DIR/amdgpu.ko.zst
sudo depmod -a
sudo limine-update  # or: sudo mkinitcpio -P
sudo reboot

10. Step 8 — Post-Reboot Verification

After reboot, run the verification script:

Fish
bash scripts/06_verify_installation.sh

Or check manually:

Check dmesg for v3 Module

Fish
sudo dmesg | grep -E "BC-250|GFXOFF|out-of-tree"

Expected:

Text Only
amdgpu: loading out-of-tree module taints kernel.
amdgpu 0000:01:00.0: amdgpu: BC-250: GFXOFF disabled to prevent GPU power-state hangs

Check for Errors

Fish
# KIQ timeout errors (should be 0)
sudo dmesg | grep -ci "timeout waiting for kiq fence"

# GPU unreachable events (should be 0)
sudo dmesg | grep -ci "GPU unreachable\|GPU died"

# NULL pointer / BUG (should be 0)
sudo dmesg | grep -ci "BUG\|NULL pointer"

Check Device Nodes

Fish
ls /dev/dri/
# Expected: card0  renderD128  (and by-path/)

ls /dev/kfd
# Expected: /dev/kfd (character device)

Check ppfeaturemask

Fish
cat /sys/module/amdgpu/parameters/ppfeaturemask
# Expected: 0xfff73ef7

Check rocminfo

Fish
rocminfo | grep -E "Name:|Marketing|gfx|Done"

Expected output includes:

Text Only
  Name:                    gfx1010
  Marketing Name:          AMD BC-250
*** Done ***


11. Step 9 — HIP Compute Tests

Compile and run the provided HIP test programs:

Compile Tests

Fish
cd tests/
hipcc --offload-arch=gfx1010 -o hip_probe hip_probe.cpp
hipcc --offload-arch=gfx1010 -o hip_minimal_test hip_minimal_test.cpp
hipcc --offload-arch=gfx1010 -o hip_vector_add hip_vector_add.cpp

Run Tests (in order)

Fish
# Test 1: Device probe (no compute)
./hip_probe

# Test 2: Minimal kernel compute (32 values)
./hip_minimal_test

# Test 3: Full vector add with managed memory (65536 elements, sin²+cos²=1.0)
./hip_vector_add

Sequential Stress Test

The critical test — on unpatched kernels, the second consecutive HIP run crashes the system:

Fish
for i in (seq 1 5)
    echo "=== Round $i ==="
    ./hip_minimal_test
    sleep 2
end

All 5 rounds should pass. If any round hangs or crashes, the module patch may not be applied correctly.

You can also run the comprehensive test script:

Fish
bash scripts/07_run_hip_tests.sh

12. Cyan Skillfish Governor (GPU Clock Management)

The BC-250's GPU clock/voltage management is handled by the cyan-skillfish-governor service. The kernel patches handle GFXOFF/power-state prevention; the governor handles DPM clock scaling.

Installation

Fish
paru -S cyan-skillfish-governor

Enable & Start

Fish
sudo systemctl enable --now cyan-skillfish-governor

Verify

Fish
systemctl is-active cyan-skillfish-governor
# Expected: active

cat /sys/class/drm/card0/device/pp_dpm_sclk
# Expected: Shows clock levels with governor managing transitions

Safe Operating Points

Clock Voltage Use Case
1000 MHz 700 mV Idle
1500 MHz 900 mV Light load
2000 MHz 1000 mV Compute (max recommended)

13. GPU Watchdog (Safety Monitor)

A safety watchdog script monitors kernel logs for KIQ timeouts and auto-kills GPU processes before the timeout cascade crashes the system. The window to act is ~10 seconds after first timeout.

Usage

Fish
# Monitor and auto-kill on KIQ timeout
bash scripts/gpu_watchdog.sh

# Monitor only (no kill)
bash scripts/gpu_watchdog.sh --dry-run

# Kill after 2 KIQ timeouts instead of 1
bash scripts/gpu_watchdog.sh --max-kiq 2

Running as a Background Monitor

Fish
nohup bash scripts/gpu_watchdog.sh > /tmp/gpu_watchdog.log 2>&1 &

14. HIP Programming Guidelines for BC-250

Memory Allocation

Function Safe? Notes
hipMallocManaged() YES Preferred — unified memory for APU
hipHostMalloc() YES Pinned host memory — safe
hipHostMallocCoherent YES Cache-coherent host memory
hipMalloc() CAUTION Works with v3 patches but may be slower
hipMemcpy() CAUTION Works with v3 patches

Critical Rules

  1. Never call hipDeviceReset() — triggers KIQ queue teardown, crashes the system
  2. Use _exit(0) instead of return 0 in HIP programs — bypasses C++ static destructors that trigger HIP runtime cleanup, which can cause KIQ fence timeouts
  3. Never call rocm-smi repeatedly — GPU management queries can destabilize the device
  4. Run a single long-lived daemon for GPU workloads when possible — each process startup/exit cycles TLB flush paths

Example: Safe HIP Program Exit

C++
#include <hip/hip_runtime.h>
#include <unistd.h>  // _exit()

int main() {
    // ... your HIP code ...

    // CRITICAL: Use _exit() to avoid HIP runtime destructor crash
    printf("Done\n");
    fflush(stdout);
    _exit(0);  // Bypasses C++ destructors and HIP cleanup
}

15. Kernel Module Rebuild After Updates

Any CachyOS kernel update will replace your patched module. After updating the kernel, you must rebuild and reinstall the patched module.

Quick Rebuild Steps

Fish
set OLD_KVER "6.19.6-2-cachyos"    # Previous kernel
set NEW_KVER (uname -r)             # New kernel after update

# If kernel version changed, download new CachyOS source and repeat Step 6
# If only pkgrel changed (e.g., -2-cachyos → -3-cachyos), you may be able to
# just copy the module and rebuild initramfs:

set MODULE_DIR /usr/lib/modules/$NEW_KVER/kernel/drivers/gpu/drm/amd/amdgpu
sudo cp /usr/lib/modules/$OLD_KVER/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst $MODULE_DIR/
sudo depmod -a
sudo limine-update

Note: This copy approach only works if the kernel ABI hasn't changed between versions. If the module fails to load, you must do a full rebuild from the new kernel source.


16. Technical Reference — Why This GPU Needs Patches

The KIQ Problem

KIQ (Kernel Interface Queue) is a privileged ring buffer used by the amdgpu driver for TLB invalidation and GPU management. On the BC-250, KIQ commands cause fence timeouts — the GPU never acknowledges the command, and the CPU waits indefinitely.

There are exactly 4 code paths that use the KIQ ring at runtime:

# Function File Patched?
1 gmc_v10_0_flush_gpu_tlb() gmc_v10_0.c YES — goto use_mmio
2 amdgpu_gmc_flush_gpu_tlb_pasid() amdgpu_gmc.c YES — direct callout
3 amdgpu_gmc_fw_reg_write_reg_wait() amdgpu_gmc.c YES — MMIO write+poll
4 amdgpu_gfx_enable/disable_kcq() amdgpu_gfx.c No (boot only, works fine)

The GFXOFF Problem

After a HIP process exits, the GPU enters GFXOFF (a power-saving state). When the next process starts and triggers a TLB flush, the GPU is unresponsive. MMIO reads via readl() inside a spinlock-protected loop hang the CPU indefinitely because the BC-250's internal PCIe fabric has no completion timeout (DevCap2: Completion Timeout: Not Supported).

The v3 Three-Layer Solution

Layer Mechanism Purpose
1 — GFXOFF Disable gfx_v10_0.c: PP_GFXOFF_MASK cleared for IP 10.1.3 Prevents GPU from entering unrecoverable power-save
2 — KIQ Bypass + Dead-GPU gmc_v10_0.c + amdgpu_gmc.c: MMIO instead of KIQ, 0xFFFFFFFF checks Avoids KIQ ring entirely; detects dead GPU before hanging
3 — Boot Parameters ppfeaturemask=0xfff73ef7 Belt-and-suspenders: disables GFXOFF+DeepSleep+ULV at power management level

Patch Summary (8 modifications across 3 files)

# File Function Type Purpose
1 gfx_v10_0.c gfx_v10_0_check_gfxoff_flag v3 Disable GFXOFF for IP 10.1.3
2 gmc_v10_0.c gmc_v10_0_flush_gpu_tlb v2 KIQ bypass → goto use_mmio
3 gmc_v10_0.c gmc_v10_0_flush_gpu_tlb v3 Pre-spinlock 0xFFFFFFFF health check
4 gmc_v10_0.c gmc_v10_0_flush_gpu_tlb v3 Sem acquire loop dead-GPU bail
5 gmc_v10_0.c gmc_v10_0_flush_gpu_tlb v3 ACK-wait loop dead-GPU bail
6 gmc_v10_0.c gmc_v10_0_hw_init v2 flush_pasid_uses_kiq = false
7 amdgpu_gmc.c amdgpu_gmc_flush_gpu_tlb_pasid v2 Direct MMIO TLB flush
8 amdgpu_gmc.c amdgpu_gmc_fw_reg_write_reg_wait v2+v3 MMIO write+poll + dead-GPU detection

Why Mining OS (Kernel 5.10) Worked Without Patches

In kernel 5.10 + AMDGPU-PRO 22.20, TLB flush was simpler and more localized. The centralized amdgpu_gmc_flush_gpu_tlb_pasid() in amdgpu_gmc.c and the deferred work amdgpu_vm_tlb_fence_work() didn't exist yet. KIQ failures returned -ETIME gracefully instead of cascading to GPU death. Modern kernels (6.x) refactored TLB flushing into a shared centralized path that defaults to KIQ for all hardware — which breaks BC-250.


17. Troubleshooting

System freezes after HIP program runs

Cause: v3 module not loaded, or initramfs not rebuilt after installation.

Fix: Boot from recovery, restore original module, verify initramfs was rebuilt:

Fish
sudo cp /usr/lib/modules/(uname -r)/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst.original \
       /usr/lib/modules/(uname -r)/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst
sudo depmod -a
sudo limine-update  # MUST rebuild initramfs!
sudo reboot

NULL pointer dereference at boot (drm_vma_offset_add)

Cause: Module was built from vanilla kernel.org source instead of CachyOS source.

Fix: Rebuild from CachyOS-specific kernel source (see Step 6).

rocminfo shows no GPU / "No HSA GPU agents found"

Cause: Environment variables not set, or /dev/kfd doesn't exist.

Fix:

Fish
# Check env vars
echo $HSA_OVERRIDE_GFX_VERSION  # Should be 10.1.0

# Check /dev/kfd exists
ls -la /dev/kfd

# Check user groups
groups  # Should include 'render'

"KIQ timeout" messages in dmesg

Cause: Stock (unpatched) module is loaded instead of v3.

Fix: Verify the v3 module is installed:

Fish
zstd -d -c /usr/lib/modules/(uname -r)/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst \
  | strings | grep "BC-250"
# Should show 9 BC-250 strings

Module version mismatch after kernel update

Cause: CachyOS updated the kernel, replacing the patched module.

Fix: Rebuild the module from matching CachyOS source (see Section 15).


18. File Inventory

Text Only
BC250_ROCm_Guide/
├── README.md                          ← This guide
├── config/
│   └── amdgpu.conf                   ← Modprobe configuration
├── patches/
│   ├── patch1_gfxoff.py              ← Layer 1: Disable GFXOFF
│   ├── patch2_gmc.py                 ← Layer 2: KIQ bypass + dead-GPU (gmc_v10_0.c)
│   └── patch3_amdgpu_gmc.py          ← Layer 2: KIQ bypass + dead-GPU (amdgpu_gmc.c)
├── scripts/
│   ├── 01_install_rocm_packages.sh   ← Package installation
│   ├── 02_configure_environment.sh   ← Environment variables + modprobe + boot params
│   ├── 04_build_patched_module.sh    ← Download source, patch, build module
│   ├── 05_install_module.sh          ← Install module + rebuild initramfs
│   ├── 06_verify_installation.sh     ← Post-reboot verification
│   ├── 07_run_hip_tests.sh           ← Compile and run HIP tests
│   └── gpu_watchdog.sh               ← KIQ timeout safety monitor
└── tests/
    ├── hip_probe.cpp                 ← Device diagnostic (6 steps)
    ├── hip_minimal_test.cpp          ← Minimal kernel compute test
    └── hip_vector_add.cpp            ← Full managed-memory vector add test

Based on research and patches by Dani (2026-02-22), adapted and verified on CachyOS kernel 6.19.6-2-cachyos with ROCm 7.2.0.