ROCm-Research-Archive/_TestScripts/Danis ROCm Kernel Patch Research/ZImage_Documentation.md

Z-Image-Turbo on AMD BC-250 — Complete Guide¶

Hardware: AMD BC-250 (Cyan Skillfish, gfx1013→gfx1010, 24 CUs, shared RAM) Backend: ROCm 7.2.0 / HIP (hipBLAS) OS: CachyOS, kernel 6.18.8-3-cachyos with v3 kernel patches Server: stable-diffusion.cpp-restapi (sdcpp-restapi)

---

1. Overview¶

Z-Image-Turbo is a Stable Diffusion model optimized for fast image generation. On the BC-250, it runs via the ROCm/HIP backend using sdcpp-restapi as an HTTP server. The server provides a REST API and a WebUI for image generation.

Architecture¶

Text Only

[User / WebUI] → HTTP :8080 → [sdcpp-restapi] → [stable-diffusion.cpp] → [HIP/hipBLAS] → [AMD BC-250 GPU]

Pipeline Phases¶

Each image generation goes through three phases:

Phase	What it does	Time (BC-250)
Text-Encoding (CLIP)	Encodes the text prompt into embeddings	~0.5–0.6s
Sampling (Diffusion)	Iterative denoising (8 steps with Euler sampler)	~73–78s
VAE Decode	Decodes latent space back to pixel image	~14s
Total	End-to-end generation	~82–92s

---

2. Directory Layout¶

Text Only

~/stable-diffusion.cpp-restapi/
├── build/
│   ├── bin/sdcpp-restapi          # Server binary
│   └── config.json                # Server configuration
├── src/                           # Server source code
└── build/_deps/stable-diffusion-src/  # SD library (fetched via CMake)

~/sd-models/
├── diffusion_models/
│   └── z_image_turbo-Q5_K_S.gguf  # Main diffusion model (5.2 GB, Q5_K_S quantized)
├── vae/
│   └── ae.safetensors             # VAE decoder (335 MB, f32)
├── llm/
│   ├── Qwen3-4B-Instruct-2507-Q5_K_S.gguf  # LLM for prompt enhancement (2.8 GB)
│   ├── Qwen3-8B-Q4_K_M.gguf                # Alternative LLM (5.0 GB)
│   └── gpt-oss-20b-Q4_K_M.gguf             # Large LLM (11.6 GB) — too big for BC-250
├── lora/
├── clip/
├── controlnet/
├── esrgan/
└── taesd/

~/sd-outputs/                      # Generated images output directory

~/start-zimage.sh                  # Launch script
~/VibeROCm/ZImage_Documentation.md # This document

---

3. Building from Source¶

Bash

cd ~/stable-diffusion.cpp-restapi
mkdir -p build && cd build
cmake .. -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DSD_HIP=ON \
  -DSDCPP_WEBUI=ON \
  -DCMAKE_PREFIX_PATH=/opt/rocm
ninja -j$(nproc --all)

Requirements: ROCm 7.2.0, CMake, Ninja, hipBLAS/rocBLAS

---

4. Configuration¶

Server Config (~/stable-diffusion.cpp-restapi/build/config.json)¶

JSON

{
    "server": {
        "host": "0.0.0.0",
        "port": 8080,
        "threads": 8
    },
    "paths": {
        "diffusion_models": "/home/dars/sd-models/diffusion_models",
        "vae": "/home/dars/sd-models/vae",
        "llm": "/home/dars/sd-models/llm",
        "lora": "/home/dars/sd-models/lora",
        "clip": "/home/dars/sd-models/clip",
        "controlnet": "/home/dars/sd-models/controlnet",
        "esrgan": "/home/dars/sd-models/esrgan",
        "taesd": "/home/dars/sd-models/taesd",
        "output": "/home/dars/sd-outputs"
    },
    "sd_defaults": {
        "n_threads": 10,
        "keep_clip_on_cpu": true,
        "keep_vae_on_cpu": true,
        "flash_attn": false,
        "offload_to_cpu": true,
        "free_params_immediately": true
    }
}

Environment Variables (set in start-zimage.sh)¶

Required¶

Variable	Value	Why
HSA_OVERRIDE_GFX_VERSION	10.1.0	BC-250 (gfx1013) needs gfx1010 spoof for ROCm
HSA_ENABLE_SDMA	0	SDMA engine has hardware bugs on gfx1013
HIP_VISIBLE_DEVICES	0	Select the BC-250 GPU
ROCM_PATH	/opt/rocm	ROCm installation path
HSA_TOOLS_LIB	""	Disable profiling tools (stability)
HSA_TOOLS_REPORT_LOAD_FAILURE	0	Suppress tool warnings

Explicitly Unset (performance-critical!)¶

These old workaround variables were used during development and destroy performance if set. The start script explicitly unsets them:

Variable	Effect if set	Impact
GPU_MAX_HW_QUEUES=1	Serializes all GPU operations to 1 queue	Severe slowdown
HIP_LAUNCH_BLOCKING=1	Forces synchronous kernel launches	Prevents pipelining
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1	Uses hipMallocManaged (page fault overhead)	+20% slower
GGML_HIP_HOST_ALLOC=1	Uses hipHostMalloc (zero-copy over PCIe)	+40% slower
GGML_CUDA_NO_PINNED=1	Disables pinned memory pools	Not needed with v3 patches
GGML_HIP_NO_COARSE_GRAIN=1	Disables coarse-grain memory	Not needed with v3 patches
HSA_DISABLE_FRAGMENT_ALLOCATOR=1	Disables memory fragment allocator	Not needed with v3 patches

---

5. Starting the Server¶

Using the Launch Script (recommended)¶

Bash

bash ~/start-zimage.sh

Or in background with logging:

Bash

nohup bash ~/start-zimage.sh > /tmp/zimage.log 2>&1 &

The script handles: - GPU health check (aborts if KIQ fence timeout detected — reboot needed) - Unsetting old workaround variables - Setting correct ROCm environment - Starting the server

Manual Start (if needed)¶

Bash

export HSA_OVERRIDE_GFX_VERSION=10.1.0
export HSA_ENABLE_SDMA=0
export HSA_TOOLS_LIB=""
export HIP_VISIBLE_DEVICES=0
export ROCM_PATH=/opt/rocm
export PATH="/opt/rocm/bin:$PATH"
export LD_LIBRARY_PATH="/opt/rocm/lib"

# CRITICAL: unset old workarounds
unset GPU_MAX_HW_QUEUES HIP_LAUNCH_BLOCKING
unset GGML_CUDA_ENABLE_UNIFIED_MEMORY GGML_HIP_HOST_ALLOC
unset GGML_CUDA_NO_PINNED GGML_HIP_NO_COARSE_GRAIN
unset HSA_DISABLE_FRAGMENT_ALLOCATOR

cd ~/stable-diffusion.cpp-restapi/build
./bin/sdcpp-restapi --config config.json

Endpoints¶

Endpoint	Description
http://localhost:8080/ui	Web UI
http://localhost:8080	API root
ws://localhost:8081	WebSocket (live progress)

---

6. Loading Models¶

After the server starts, you must load a model before generating images.

Load Z-Image-Turbo (recommended command)¶

Bash

curl -s -X POST http://localhost:8080/models/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "z_image_turbo-Q5_K_S.gguf",
    "model_type": "diffusion",
    "vae": "ae.safetensors",
    "llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf",
    "options": {
      "flash_attn": false
    }
  }'

Expected response (takes ~25 seconds):

JSON

{
  "success": true,
  "message": "Model loaded successfully",
  "model_name": "z_image_turbo-Q5_K_S.gguf",
  "model_type": "diffusion",
  "loaded_components": {
    "vae": "ae.safetensors",
    "llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf"
  }
}

Model Load API Reference¶

POST /models/load

Field	Type	Required	Description
model_name	string	Yes	Filename of the diffusion model (must be in diffusion_models/ dir)
model_type	string	Yes	"diffusion" for SD models
vae	string	Yes	VAE filename (from vae/ dir)
llm	string	Optional	LLM for prompt enhancement (from llm/ dir)
options	object	Optional	Model options (see below)

Model Load Options¶

Option	Type	Default	Recommended	Description
flash_attn	bool	false	false	Flash attention — gfx1010 does NOT support it
keep_clip_on_cpu	bool	true	true	CLIP runs fine on CPU, saves GPU memory
keep_vae_on_cpu	bool	false	false	VAE on GPU is faster
offload_to_cpu	bool	false	false	CPU offload hurts performance
vae_decode_only	bool	true	true	Only need decode for txt2img
free_params_immediately	bool	false	false	Frees memory faster but slower load
vae_conv_direct	bool	false	false	Direct convolutions — model load never finishes!
diffusion_conv_direct	bool	false	false	Direct convolutions — model load never finishes!
n_threads	int	-1 (auto)	-1	CPU threads for compute
enable_mmap	bool	true	true	Memory-mapped file loading

Unload Model¶

Bash

curl -s -X POST http://localhost:8080/models/unload

---

7. Generating Images¶

Via API (curl)¶

Step 1: Submit a generation job

Bash

curl -s -X POST http://localhost:8080/txt2img \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "blonde woman",
    "width": 512,
    "height": 1024,
    "steps": 8,
    "cfg_scale": 1,
    "sampler": "euler",
    "scheduler": "smoothstep",
    "seed": 42
  }'

Response:

JSON

{
  "job_id": "e95872f8-15cb-48ce-a10f-19f2de430d9e",
  "position": 1,
  "status": "pending"
}

Step 2: Poll for completion

Bash

curl -s http://localhost:8080/queue/<job_id>

Response when done:

JSON

{
  "status": "completed",
  "duration": "92.2s",
  "outputs": ["image_filename.png"]
}

Via WebUI¶

Open http://localhost:8080/ui in a browser.

Generation Parameters¶

Parameter	Type	Default	Description
prompt	string	Required	Text description of the image
negative_prompt	string	""	What to avoid
width	int	512	Image width (pixels)
height	int	512	Image height (pixels)
steps	int	20	Sampling steps (8 is optimal for Z-Image-Turbo)
cfg_scale	float	7.0	Classifier-free guidance scale (1.0 for turbo models)
sampler	string	"euler"	Sampling method
scheduler	string	"smoothstep"	Noise schedule
seed	int	random	Seed for reproducibility (-1 = random)

Recommended Settings for Z-Image-Turbo¶

JSON

{
  "steps": 8,
  "cfg_scale": 1,
  "sampler": "euler",
  "scheduler": "smoothstep"
}

Z-Image-Turbo is a distilled model — 8 steps is the sweet spot. More steps don't improve quality.

---

8. Performance Benchmarks¶

All benchmarks: 512×1024 image, 8 steps, Euler sampler, smoothstep scheduler, seed 42, prompt "blonde woman".

Best Result: ~82s (measured warm run, 3^rd consecutive generation)¶

Systematic Benchmark Matrix¶

Config	Text-Enc	Sampling	VAE Decode	Total	vs Baseline
Clean (cold, 1st gen)	0.54s	77.4s	13.9s	92s	Baseline
Clean (warm, 2nd gen)	—	—	—	85s	-8%
Clean (warm, 3rd gen)	—	—	—	82s	-11%
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1	0.52s	82.7s	26.1s	109s	+18% slower
GGML_HIP_HOST_ALLOC=1	0.55s	98.3s	29.8s	129s	+40% slower
conv_direct=true (both)	—	—	—	N/A	Model load stalls (>13 min, aborted)
Vulkan backend (RADV)	—	—	—	~79s	Reference

Key Findings¶

1. Clean config is best: No GGML environment variables. The v3 kernel patches make all old workarounds unnecessary.

2. Warm vs cold run: First generation after model load takes ~92s. Subsequent runs improve to ~85s (2^nd) → ~82s (3^rd+) as GPU caches warm up.

3. GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 hurts: hipMallocManaged causes page fault overhead → +18% slower sampling, +88% slower VAE.

4. GGML_HIP_HOST_ALLOC=1 worst: Zero-copy over PCIe is terrible → +27% slower sampling, +114% slower VAE.

5. conv_direct is broken: Setting vae_conv_direct=true and/or diffusion_conv_direct=true causes the model load to hang indefinitely (13+ minutes, never completes). These bypass rocBLAS im2col+GEMM path for direct convolutions, but the compute graph build is too expensive for this model size (244 VAE f32 tensors + 180 diffusion q5_K tensors).

6. ROCm vs Vulkan gap: ROCm (82–92s) vs Vulkan (79s). ROCm warm runs are very close to Vulkan (~82s vs ~79s, only ~4% gap). The remaining overhead is ggml compute graph scheduling, not GPU compute itself (~10s actual GPU time vs ~67s framework overhead in sampling).

Timing Breakdown¶

The 92s total breaks down as:

Text Only

Text-Encoding (CLIP on CPU):        0.5s  (  0.5%)  — NOT a bottleneck
Sampling (8 steps diffusion):      77.4s  ( 84.1%)  — main bottleneck
  ├── Actual GPU compute:          ~10s   ( 10.9%)
  └── ggml framework overhead:     ~67s   ( 73.2%)  — graph scheduling, kernel launches
VAE Decode:                        13.9s  ( 15.1%)

The 67s ggml overhead is an upstream limitation of the stable-diffusion.cpp framework, not something fixable via configuration.

---

9. Tested Configurations (Full History)¶

All the environment variable combinations tested during optimization:

Config 1: "All Workarounds" (pre-v3 era)¶

Bash

GPU_MAX_HW_QUEUES=1
HIP_LAUNCH_BLOCKING=1
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1
GGML_HIP_HOST_ALLOC=1
GGML_CUDA_NO_PINNED=1
GGML_HIP_NO_COARSE_GRAIN=1
HSA_DISABLE_FRAGMENT_ALLOCATOR=1

Result: ~155s initially, ~82s with warm cache. These variables were needed before v3 kernel patches but are harmful now.

Config 2: Clean (current optimal)¶

Bash

# NO GGML variables set at all
# Only ROCm basics:
HSA_OVERRIDE_GFX_VERSION=10.1.0
HSA_ENABLE_SDMA=0
HIP_VISIBLE_DEVICES=0

Result: 92s cold / ~82s warm (3^rd gen). This is the recommended config.

Config 3: UNIFIED_MEMORY only¶

Bash

GGML_CUDA_ENABLE_UNIFIED_MEMORY=1

Result: 109s (+18%). hipMallocManaged page faults add overhead.

Config 4: HIP_HOST_ALLOC only¶

Bash

GGML_HIP_HOST_ALLOC=1

Result: 129s (+40%). Zero-copy over PCIe kills performance.

Config 5: Conv Direct¶

Bash

# Model load options:
vae_conv_direct=true
diffusion_conv_direct=true

Result: Model load never completes (>13 minutes at 100% CPU, aborted). Not viable.

---

10. Troubleshooting¶

Server won't start¶

Text Only

[ERROR] GPU is in a broken state (KIQ fence timeout detected)

Fix: Reboot the system. The BC-250 GPU cannot recover from KIQ timeouts.

Bash

sudo reboot

Model load fails¶

JSON

{"error": "Main model not found: ''"}

Fix: Use model_name field (not model or model_path). Value must be the filename only (not full path). The server looks in the configured diffusion_models/ directory.

Generation hangs or crashes¶

• Do NOT run rocm-smi, clinfo, or any other GPU query tool while the server is running
• Do NOT start multiple GPU processes simultaneously
• If the server crashes, reboot before restarting — the GPU state may be corrupted

Old environment variables leaking from .bashrc¶

If generation is unexpectedly slow (>100s), check for leftover variables:

Bash

env | grep -E "GGML_|GPU_MAX|HIP_LAUNCH"

If any of the old workaround vars are set, unset them:

Bash

unset GPU_MAX_HW_QUEUES HIP_LAUNCH_BLOCKING
unset GGML_CUDA_ENABLE_UNIFIED_MEMORY GGML_HIP_HOST_ALLOC
unset GGML_CUDA_NO_PINNED GGML_HIP_NO_COARSE_GRAIN
unset HSA_DISABLE_FRAGMENT_ALLOCATOR

flash_attn errors¶

Text Only

gfx1010 does not support flash attention

Fix: Always load with "flash_attn": false. The gfx1010 architecture does not have the required hardware. This is already the default in the patched code.

---

11. Quick Reference¶

Start Server + Load Model + Generate Image (one-liner workflow)¶

Bash

# 1. Start server
nohup bash ~/start-zimage.sh > /tmp/zimage.log 2>&1 &
sleep 5

# 2. Load model
curl -s -X POST http://localhost:8080/models/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "z_image_turbo-Q5_K_S.gguf",
    "model_type": "diffusion",
    "vae": "ae.safetensors",
    "llm": "Qwen3-4B-Instruct-2507-Q5_K_S.gguf",
    "options": {"flash_attn": false}
  }'

# 3. Generate image
JOB=$(curl -s -X POST http://localhost:8080/txt2img \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "blonde woman",
    "width": 512, "height": 1024,
    "steps": 8, "cfg_scale": 1,
    "sampler": "euler", "scheduler": "smoothstep"
  }' | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")

echo "Job: $JOB"

# 4. Poll until done
while true; do
  STATUS=$(curl -s http://localhost:8080/queue/$JOB | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('status','?'))")
  echo "Status: $STATUS"
  [ "$STATUS" = "completed" ] && break
  sleep 10
done

# 5. Check result
curl -s http://localhost:8080/queue/$JOB | python3 -m json.tool

Check Server Logs¶

Bash

tail -f /tmp/zimage.log

Check Timing Breakdown¶

Bash

grep -a "get_learned\|sampling completed\|decode_first\|generate_image completed" /tmp/zimage.log | tail -4

---

12. Important Warnings¶

1. Do NOT run rocm-smi or any GPU monitoring tool while the server is running — it can crash the GPU
2. Reboot after any crash — the BC-250 GPU state cannot be recovered without a full reboot
3. Do NOT set old GGML environment variables — they are not needed with v3 kernel patches and severely hurt performance
4. Do NOT use conv_direct — model load hangs indefinitely
5. Do NOT use flash_attn: true — gfx1010 does not support flash attention
6. One GPU process at a time — the BC-250 cannot handle concurrent GPU workloads