Skip to content

vllm.model_executor.models.qwen3_next_mtp

Inference-only Qwen3Next MTP model.

Functions:

should_torch_compile_mtp_drafter(vllm_config)

enable_if for MTP drafters that pick their module by spec_step_idx.

The drafter is compiled once with guards dropped, so a compiled multi-module drafter would reuse the first-traced module for every step.

Source code in vllm/model_executor/models/qwen3_next_mtp.py 
44
45
46
47
48
49
50
51
def should_torch_compile_mtp_drafter(vllm_config: VllmConfig) -> bool:
    """`enable_if` for MTP drafters that pick their module by `spec_step_idx`.

    The drafter is compiled once with guards dropped, so a compiled
    multi-module drafter would reuse the first-traced module for every step.
    """
    spec_config = vllm_config.speculative_config
    return spec_config is None or not spec_config.use_multi_module_mtp()