Fabian/ROCm-Research-Archive
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Uploaded sanitized BC250/ROCm Repository.

Browse Source
This commit is contained in:
Fabian
2026-08-20 00:45:43 +02:00
parent 7d2184f1e8
commit d7d22e93b3
678 changed files with 65963 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Danis ROCm Kernel Patch Research/ZImage_Documentation.md
+572
View File
@@ -0,0 +1,572 @@
# Z-Image-Turbo on AMD BC-250 — Complete Guide
> **Hardware**: AMD BC-250 (Cyan Skillfish, gfx1013→gfx1010, 24 CUs, shared RAM)
> **Backend**: ROCm 7.2.0 / HIP (hipBLAS)
> **OS**: CachyOS, kernel 6.18.8-3-cachyos with v3 kernel patches
> **Server**: [stable-diffusion.cpp-restapi](https://github.com/leejet/stable-diffusion.cpp) (sdcpp-restapi)
---
## 1. Overview
Z-Image-Turbo is a Stable Diffusion model optimized for fast image generation. On the BC-250, it runs via the ROCm/HIP backend using `sdcpp-restapi` as an HTTP server. The server provides a REST API and a WebUI for image generation.
### Architecture
```
[User / WebUI] → HTTP :8080 → [sdcpp-restapi] → [stable-diffusion.cpp] → [HIP/hipBLAS] → [AMD BC-250 GPU]
```
### Pipeline Phases
Each image generation goes through three phases:
| Phase | What it does | Time (BC-250) |
|-------|-------------|---------------|
| **Text-Encoding** (CLIP) | Encodes the text prompt into embeddings | ~0.5–0.6s |
| **Sampling** (Diffusion) | Iterative denoising (8 steps with Euler sampler) | ~73–78s |
| **VAE Decode** | Decodes latent space back to pixel image | ~14s |
| **Total** | End-to-end generation | **~82–92s** |
---
## 2. Directory Layout
```
~/stable-diffusion.cpp-restapi/
├── build/
│ ├── bin/sdcpp-restapi # Server binary
│ └── config.json # Server configuration
├── src/ # Server source code
└── build/_deps/stable-diffusion-src/ # SD library (fetched via CMake)
~/sd-models/
├── diffusion_models/
│ └── z_image_turbo-Q5_K_S.gguf # Main diffusion model (5.2 GB, Q5_K_S quantized)
├── vae/
│ └── ae.safetensors # VAE decoder (335 MB, f32)
├── llm/
│ ├── Qwen3-4B-Instruct-2507-Q5_K_S.gguf # LLM for prompt enhancement (2.8 GB)
│ ├── Qwen3-8B-Q4_K_M.gguf # Alternative LLM (5.0 GB)
│ └── gpt-oss-20b-Q4_K_M.gguf # Large LLM (11.6 GB) — too big for BC-250
├── lora/
├── clip/
├── controlnet/
├── esrgan/
└── taesd/
~/sd-outputs/ # Generated images output directory
~/start-zimage.sh # Launch script
~/VibeROCm/ZImage_Documentation.md # This document
```
---
## 3. Building from Source
```bash
cd ~/stable-diffusion.cpp-restapi
mkdir -p build && cd build
cmake .. -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DSD_HIP=ON \
-DSDCPP_WEBUI=ON \
-DCMAKE_PREFIX_PATH=/opt/rocm
ninja -j$(nproc --all)
```
**Requirements**: ROCm 7.2.0, CMake, Ninja, hipBLAS/rocBLAS
---
## 4. Configuration
### Server Config (`~/stable-diffusion.cpp-restapi/build/config.json`)
```json
{
"server": {
"host": "0.0.0.0",
"port": 8080,
"threads": 8
},
"paths": {
"diffusion_models": "/home/dars/sd-models/diffusion_models",
"vae": "/home/dars/sd-models/vae",
"llm": "/home/dars/sd-models/llm",
"lora": "/home/dars/sd-models/lora",
"clip": "/home/dars/sd-models/clip",
"controlnet": "/home/dars/sd-models/controlnet",
"esrgan": "/home/dars/sd-models/esrgan",
"taesd": "/home/dars/sd-models/taesd",
"output": "/home/dars/sd-outputs"
},
"sd_defaults": {
"n_threads": 10,
"keep_clip_on_cpu": true,
"keep_vae_on_cpu": true,
"flash_attn": false,
"offload_to_cpu": true,
"free_params_immediately": true
}
}
```
### Environment Variables (set in `start-zimage.sh`)
#### Required
| Variable | Value | Why |
|----------|-------|-----|
| `HSA_OVERRIDE_GFX_VERSION` | `10.1.0` | BC-250 (gfx1013) needs gfx1010 spoof for ROCm |
| `HSA_ENABLE_SDMA` | `0` | SDMA engine has hardware bugs on gfx1013 |
| `HIP_VISIBLE_DEVICES` | `0` | Select the BC-250 GPU |
| `ROCM_PATH` | `/opt/rocm` | ROCm installation path |
| `HSA_TOOLS_LIB` | `""` | Disable profiling tools (stability) |
| `HSA_TOOLS_REPORT_LOAD_FAILURE` | `0` | Suppress tool warnings |
#### Explicitly Unset (performance-critical!)
These old workaround variables were used during development and **destroy performance** if set. The start script explicitly unsets them:
| Variable | Effect if set | Impact |
|----------|--------------|--------|
| `GPU_MAX_HW_QUEUES=1` | Serializes all GPU operations to 1 queue | Severe slowdown |
| `HIP_LAUNCH_BLOCKING=1` | Forces synchronous kernel launches | Prevents pipelining |
| `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` | Uses hipMallocManaged (page fault overhead) | +20% slower |
| `GGML_HIP_HOST_ALLOC=1` | Uses hipHostMalloc (zero-copy over PCIe) | +40% slower |
| `GGML_CUDA_NO_PINNED=1` | Disables pinned memory pools | Not needed with v3 patches |
| `GGML_HIP_NO_COARSE_GRAIN=1` | Disables coarse-grain memory | Not needed with v3 patches |
| `HSA_DISABLE_FRAGMENT_ALLOCATOR=1` | Disables memory fragment allocator | Not needed with v3 patches |
---
## 5. Starting the Server
### Using the Launch Script (recommended)
```bash
bash ~/start-zimage.sh
```
Or in background with logging:
```bash
nohup bash ~/start-zimage.sh > /tmp/zimage.log 2>&1 &
```
The script handles:
- GPU health check (aborts if KIQ fence timeout detected — reboot needed)
- Unsetting old workaround variables
- Setting correct ROCm environment
- Starting the server
### Manual Start (if needed)
```bash
export HSA_OVERRIDE_GFX_VERSION=10.1.0
export HSA_ENABLE_SDMA=0
export HSA_TOOLS_LIB=""
export HIP_VISIBLE_DEVICES=0
export ROCM_PATH=/opt/rocm
export PATH="/opt/rocm/bin:$PATH"
export LD_LIBRARY_PATH="/opt/rocm/lib"
# CRITICAL: unset old workarounds
unset GPU_MAX_HW_QUEUES HIP_LAUNCH_BLOCKING
unset GGML_CUDA_ENABLE_UNIFIED_MEMORY GGML_HIP_HOST_ALLOC
unset GGML_CUDA_NO_PINNED GGML_HIP_NO_COARSE_GRAIN
unset HSA_DISABLE_FRAGMENT_ALLOCATOR
cd ~/stable-diffusion.cpp-restapi/build
./bin/sdcpp-restapi --config config.json
```
### Endpoints
| Endpoint | Description |
|----------|-------------|
| `http://localhost:8080/ui` | Web UI |
| `http://localhost:8080` | API root |
| `ws://localhost:8081` | WebSocket (live progress) |
---
## 6. Loading Models
After the server starts, you must load a model before generating images.
### Load Z-Image-Turbo (recommended command)
```bash
curl -s -X POST http://localhost:8080/models/load \
-H "Content-Type: application/json" \
-d '{
"model_name": "z_image_turbo-Q5_K_S.gguf",
"model_type": "diffusion",
"vae": "ae.safetensors",
"llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf",
"options": {
"flash_attn": false
}
}'
```
**Expected response** (takes ~25 seconds):
```json
{
"success": true,
"message": "Model loaded successfully",
"model_name": "z_image_turbo-Q5_K_S.gguf",
"model_type": "diffusion",
"loaded_components": {
"vae": "ae.safetensors",
"llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf"
}
}
```
### Model Load API Reference
**POST** `/models/load`
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `model_name` | string | **Yes** | Filename of the diffusion model (must be in `diffusion_models/` dir) |
| `model_type` | string | **Yes** | `"diffusion"` for SD models |
| `vae` | string | Yes | VAE filename (from `vae/` dir) |
| `llm` | string | Optional | LLM for prompt enhancement (from `llm/` dir) |
| `options` | object | Optional | Model options (see below) |
### Model Load Options
| Option | Type | Default | Recommended | Description |
|--------|------|---------|-------------|-------------|
| `flash_attn` | bool | `false` | **`false`** | Flash attention — gfx1010 does NOT support it |
| `keep_clip_on_cpu` | bool | `true` | `true` | CLIP runs fine on CPU, saves GPU memory |
| `keep_vae_on_cpu` | bool | `false` | `false` | VAE on GPU is faster |
| `offload_to_cpu` | bool | `false` | `false` | CPU offload hurts performance |
| `vae_decode_only` | bool | `true` | `true` | Only need decode for txt2img |
| `free_params_immediately` | bool | `false` | `false` | Frees memory faster but slower load |
| `vae_conv_direct` | bool | `false` | **`false`** | Direct convolutions — model load never finishes! |
| `diffusion_conv_direct` | bool | `false` | **`false`** | Direct convolutions — model load never finishes! |
| `n_threads` | int | `-1` (auto) | `-1` | CPU threads for compute |
| `enable_mmap` | bool | `true` | `true` | Memory-mapped file loading |
### Unload Model
```bash
curl -s -X POST http://localhost:8080/models/unload
```
---
## 7. Generating Images
### Via API (curl)
**Step 1: Submit a generation job**
```bash
curl -s -X POST http://localhost:8080/txt2img \
-H "Content-Type: application/json" \
-d '{
"prompt": "blonde woman",
"width": 512,
"height": 1024,
"steps": 8,
"cfg_scale": 1,
"sampler": "euler",
"scheduler": "smoothstep",
"seed": 42
}'
```
Response:
```json
{
"job_id": "e95872f8-15cb-48ce-a10f-19f2de430d9e",
"position": 1,
"status": "pending"
}
```
**Step 2: Poll for completion**
```bash
curl -s http://localhost:8080/queue/<job_id>
```
Response when done:
```json
{
"status": "completed",
"duration": "92.2s",
"outputs": ["image_filename.png"]
}
```
### Via WebUI
Open `http://localhost:8080/ui` in a browser.
### Generation Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `prompt` | string | Required | Text description of the image |
| `negative_prompt` | string | `""` | What to avoid |
| `width` | int | 512 | Image width (pixels) |
| `height` | int | 512 | Image height (pixels) |
| `steps` | int | 20 | Sampling steps (8 is optimal for Z-Image-Turbo) |
| `cfg_scale` | float | 7.0 | Classifier-free guidance scale (1.0 for turbo models) |
| `sampler` | string | `"euler"` | Sampling method |
| `scheduler` | string | `"smoothstep"` | Noise schedule |
| `seed` | int | random | Seed for reproducibility (-1 = random) |
### Recommended Settings for Z-Image-Turbo
```json
{
"steps": 8,
"cfg_scale": 1,
"sampler": "euler",
"scheduler": "smoothstep"
}
```
Z-Image-Turbo is a distilled model — 8 steps is the sweet spot. More steps don't improve quality.
---
## 8. Performance Benchmarks
All benchmarks: 512×1024 image, 8 steps, Euler sampler, smoothstep scheduler, seed 42, prompt "blonde woman".
### Best Result: ~82s (measured warm run, 3rd consecutive generation)
### Systematic Benchmark Matrix
| Config | Text-Enc | Sampling | VAE Decode | Total | vs Baseline |
|--------|----------|----------|------------|-------|-------------|
| **Clean (cold, 1st gen)** | 0.54s | 77.4s | 13.9s | **92s** | Baseline |
| **Clean (warm, 2nd gen)** | — | — | — | **85s** | -8% |
| **Clean (warm, 3rd gen)** | — | — | — | **82s** | -11% |
| `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` | 0.52s | 82.7s | 26.1s | **109s** | +18% slower |
| `GGML_HIP_HOST_ALLOC=1` | 0.55s | 98.3s | 29.8s | **129s** | +40% slower |
| `conv_direct=true` (both) | — | — | — | **N/A** | Model load stalls (>13 min, aborted) |
| Vulkan backend (RADV) | — | — | — | **~79s** | Reference |
### Key Findings
1. **Clean config is best**: No GGML environment variables. The v3 kernel patches make all old workarounds unnecessary.
2. **Warm vs cold run**: First generation after model load takes ~92s. Subsequent runs improve to ~85s (2nd) → ~82s (3rd+) as GPU caches warm up.
3. **GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 hurts**: `hipMallocManaged` causes page fault overhead → +18% slower sampling, +88% slower VAE.
4. **GGML_HIP_HOST_ALLOC=1 worst**: Zero-copy over PCIe is terrible → +27% slower sampling, +114% slower VAE.
5. **conv_direct is broken**: Setting `vae_conv_direct=true` and/or `diffusion_conv_direct=true` causes the model load to hang indefinitely (13+ minutes, never completes). These bypass rocBLAS im2col+GEMM path for direct convolutions, but the compute graph build is too expensive for this model size (244 VAE f32 tensors + 180 diffusion q5_K tensors).
6. **ROCm vs Vulkan gap**: ROCm (82–92s) vs Vulkan (79s). ROCm warm runs are very close to Vulkan (~82s vs ~79s, only ~4% gap). The remaining overhead is ggml compute graph scheduling, not GPU compute itself (~10s actual GPU time vs ~67s framework overhead in sampling).
### Timing Breakdown
The 92s total breaks down as:
```
Text-Encoding (CLIP on CPU): 0.5s ( 0.5%) — NOT a bottleneck
Sampling (8 steps diffusion): 77.4s ( 84.1%) — main bottleneck
├── Actual GPU compute: ~10s ( 10.9%)
└── ggml framework overhead: ~67s ( 73.2%) — graph scheduling, kernel launches
VAE Decode: 13.9s ( 15.1%)
```
The 67s ggml overhead is an upstream limitation of the stable-diffusion.cpp framework, not something fixable via configuration.
---
## 9. Tested Configurations (Full History)
All the environment variable combinations tested during optimization:
### Config 1: "All Workarounds" (pre-v3 era)
```bash
GPU_MAX_HW_QUEUES=1
HIP_LAUNCH_BLOCKING=1
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1
GGML_HIP_HOST_ALLOC=1
GGML_CUDA_NO_PINNED=1
GGML_HIP_NO_COARSE_GRAIN=1
HSA_DISABLE_FRAGMENT_ALLOCATOR=1
```
**Result**: ~155s initially, ~82s with warm cache. These variables were needed before v3 kernel patches but are harmful now.
### Config 2: Clean (current optimal)
```bash
# NO GGML variables set at all
# Only ROCm basics:
HSA_OVERRIDE_GFX_VERSION=10.1.0
HSA_ENABLE_SDMA=0
HIP_VISIBLE_DEVICES=0
```
**Result**: 92s cold / ~82s warm (3rd gen). **This is the recommended config.**
### Config 3: UNIFIED_MEMORY only
```bash
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1
```
**Result**: 109s (+18%). hipMallocManaged page faults add overhead.
### Config 4: HIP_HOST_ALLOC only
```bash
GGML_HIP_HOST_ALLOC=1
```
**Result**: 129s (+40%). Zero-copy over PCIe kills performance.
### Config 5: Conv Direct
```bash
# Model load options:
vae_conv_direct=true
diffusion_conv_direct=true
```
**Result**: Model load never completes (>13 minutes at 100% CPU, aborted). Not viable.
---
## 10. Troubleshooting
### Server won't start
```
[ERROR] GPU is in a broken state (KIQ fence timeout detected)
```
**Fix**: Reboot the system. The BC-250 GPU cannot recover from KIQ timeouts.
```bash
sudo reboot
```
### Model load fails
```json
{"error": "Main model not found: ''"}
```
**Fix**: Use `model_name` field (not `model` or `model_path`). Value must be the **filename only** (not full path). The server looks in the configured `diffusion_models/` directory.
### Generation hangs or crashes
- **Do NOT run** `rocm-smi`, `clinfo`, or any other GPU query tool while the server is running
- **Do NOT start** multiple GPU processes simultaneously
- If the server crashes, **reboot before restarting** — the GPU state may be corrupted
### Old environment variables leaking from .bashrc
If generation is unexpectedly slow (>100s), check for leftover variables:
```bash
env | grep -E "GGML_|GPU_MAX|HIP_LAUNCH"
```
If any of the old workaround vars are set, unset them:
```bash
unset GPU_MAX_HW_QUEUES HIP_LAUNCH_BLOCKING
unset GGML_CUDA_ENABLE_UNIFIED_MEMORY GGML_HIP_HOST_ALLOC
unset GGML_CUDA_NO_PINNED GGML_HIP_NO_COARSE_GRAIN
unset HSA_DISABLE_FRAGMENT_ALLOCATOR
```
### flash_attn errors
```
gfx1010 does not support flash attention
```
**Fix**: Always load with `"flash_attn": false`. The gfx1010 architecture does not have the required hardware. This is already the default in the patched code.
---
## 11. Quick Reference
### Start Server + Load Model + Generate Image (one-liner workflow)
```bash
# 1. Start server
nohup bash ~/start-zimage.sh > /tmp/zimage.log 2>&1 &
sleep 5
# 2. Load model
curl -s -X POST http://localhost:8080/models/load \
-H "Content-Type: application/json" \
-d '{
"model_name": "z_image_turbo-Q5_K_S.gguf",
"model_type": "diffusion",
"vae": "ae.safetensors",
"llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf",
"options": {"flash_attn": false}
}'
# 3. Generate image
JOB=$(curl -s -X POST http://localhost:8080/txt2img \
-H "Content-Type: application/json" \
-d '{
"prompt": "blonde woman",
"width": 512, "height": 1024,
"steps": 8, "cfg_scale": 1,
"sampler": "euler", "scheduler": "smoothstep"
}' | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
echo "Job: $JOB"
# 4. Poll until done
while true; do
STATUS=$(curl -s http://localhost:8080/queue/$JOB | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('status','?'))")
echo "Status: $STATUS"
[ "$STATUS" = "completed" ] && break
sleep 10
done
# 5. Check result
curl -s http://localhost:8080/queue/$JOB | python3 -m json.tool
```
### Check Server Logs
```bash
tail -f /tmp/zimage.log
```
### Check Timing Breakdown
```bash
grep -a "get_learned\|sampling completed\|decode_first\|generate_image completed" /tmp/zimage.log | tail -4
```
---
## 12. Important Warnings
1. **Do NOT run `rocm-smi`** or any GPU monitoring tool while the server is running — it can crash the GPU
2. **Reboot after any crash** — the BC-250 GPU state cannot be recovered without a full reboot
3. **Do NOT set old GGML environment variables** — they are not needed with v3 kernel patches and severely hurt performance
4. **Do NOT use `conv_direct`** — model load hangs indefinitely
5. **Do NOT use `flash_attn: true`** — gfx1010 does not support flash attention
6. **One GPU process at a time** — the BC-250 cannot handle concurrent GPU workloads