Skip to content

vllm.model_executor.model_loader.reload.lifecycle

Begin and complete a checkpoint-format weight update in the reload mode VLLM_RELOAD_MODE selects (fixed for the process).

Functions:

  • finish_reload –

    Complete the weight update start_reload began.

  • start_reload –

    Begin a weight update in the configured mode.

finish_reload(model, model_config, device)

Complete the weight update start_reload began.

Source code in vllm/model_executor/model_loader/reload/lifecycle.py 
24
25
26
27
28
29
30
31
def finish_reload(
    model: torch.nn.Module, model_config: ModelConfig, device: torch.device
) -> None:
    """Complete the weight update `start_reload` began."""
    if direct_reload_enabled():
        finish_direct_reload(model, model_config, device)
    else:
        finalize_layerwise_reload(model, model_config)

start_reload(model)

Begin a weight update in the configured mode.

Source code in vllm/model_executor/model_loader/reload/lifecycle.py 
16
17
18
19
20
21
def start_reload(model: torch.nn.Module) -> None:
    """Begin a weight update in the configured mode."""
    if direct_reload_enabled():
        begin_direct_reload(model)
    else:
        initialize_layerwise_reload(model)