[Bug]: HETERO config is broken for loading larger models in any branch of OpenVINO

[HumerousGorgon]

OpenVINO Version

Latest, built from source in OpenVINO Model Server

Operating System

Other (Please specify in description)

Device used for inference

HETERO

Framework

None

Model used

Qwen/Qwen3-32B, converted to OV format using export_model.py.

Issue description

When attempting to load a converted version of Qwen/Qwen3-32B (converted to INT4, with u8 kv cache precision and a cache size of 6GB) in OpenVINO model server, the entire instance crashes. This issue is not unique to model server, as this same problem also happens on a seperate project, OpenArc, built on top of the OpenVINO backend. This issue was not present some months ago and this EXACT model with the same config (converted with the same command) worked perfectly. The system is an 11600KF with 3 x Intel Arc A770's running on Ubuntu Server 24.04.3

Step-by-step reproduction

1. Install client GPU drivers on a fresh installation of Ubuntu Server 24.04.3 as per documentation.
2. Build OVMS from source.
3. Use the export_model.py script to convert the Qwen/Qwen3-32B model with the following command:
   python export_model.py text_generation --source_model Qwen/Qwen3-32B --config_file_path /media/models/config.json --weight-format int4 --overwrite_models --target_device HETERO:GPU.0,GPU.1 --extra_quantization_params "--awq --group-size 128" --kv_cache_precision u8 --cache_size 6 --model_repository_path /media/models --reasoning_parser qwen3
4. Load OVMS as per instructions.

Relevant log output

[2025-11-25 01:58:36.163][45639][serving][error][servable_initializer.cpp:214] Error during llm node initialization for models_path: /media/models/Qwen/Qwen3-32B/./ exception: Exception from src/inference/src/cpp/core.cpp:114:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/hetero/src/compiled_model.cpp:36:
Standard exception from compilation library: Exception from src/inference/src/dev/plugin.cpp:53:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:163:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/runtime/ocl/ocl_common.hpp:40:
[GPU] clEnqueueNDRangeKernel, error code: -52 CL_INVALID_KERNEL_ARGS

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[HumerousGorgon]

I've just found that this is linked to the fact that attempting to load models into GPUs other than GPU.0 on a multi-gpu system is completely broken and results in segment faults, which are really kernel arg problems as shown above. I believe this is why the hetero config is failing. This needs to be addressed.

[savvadesogle]

The same is on windows 11.
Driver: 8250
OV Model: https://huggingface.co/OpenVINO/Qwen3-8B-int4-ov/

2026

openvino                   2026.0.0.dev20251202
openvino-genai             2026.0.0.0.dev20251202
openvino-telemetry         2025.2.0
openvino-tokenizers        2026.0.0.0.dev20251202

or
2025.4

openvino                   2025.4.0
openvino-genai             2025.4.0.0
openvino-telemetry         2025.2.0
openvino-tokenizers        2025.4.0.0

or
2025.3

openvino                   2025.3.0
openvino-genai             2025.3.0.0
openvino-telemetry         2025.2.0
openvino-tokenizers        2025.3.0.0

2025-12-04 15:11:16,984 - INFO - ModelType.LLM on GPU.1 with {}
2025-12-04 15:11:43,447 - ERROR - Model loading failed for Qwen3-8B-int4-ov
Traceback (most recent call last):
  File "C:\llm\openarc\201\src\server\model_registry.py", line 228, in _load_task
    model_instance = await create_model_instance(load_config)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\llm\openarc\201\src\server\model_registry.py", line 337, in create_model_instance
    await asyncio.to_thread(model_instance.load_model, load_config)
  File "C:\Users\uuk\AppData\Local\Programs\Python\Python311\Lib\asyncio\threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\uuk\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\llm\openarc\201\src\engine\ov_genai\llm.py", line 201, in load_model
    self.model = LLMPipeline(
                 ^^^^^^^^^^^^
RuntimeError: Exception from src\inference\src\cpp\core.cpp:117:
Exception from src\inference\src\dev\plugin.cpp:53:
Check 'false' failed at src\plugins\intel_gpu\src\plugin\program_builder.cpp:163:
[GPU] ProgramBuilder build failed!
Exception from src\plugins\intel_gpu\src\runtime\ocl\ocl_common.hpp:40:
[GPU] clEnqueueNDRangeKernel, error code: -52 CL_INVALID_KERNEL_ARGS

[Wan-Intel]

Hi @HumerousGorgon , @savvadesogle

We are checking with relevant team, and we will provide an update soon.

[HumerousGorgon]

Hi @HumerousGorgon , @savvadesogle

We are checking with relevant team, and we will provide an update soon.

Thank you very much, I appreciate it.
I’ve done extensive testing on this issue and no matter how we try it; bare metal or Docker, it persists. We look forward to hearing your results :)