[Model] Support Deepseek V3.2

[heheda12345]

Purpose

This PR adds the support for the new deepseek v3.2 model. It is a verified stable version that can be used now.

We are also working on a cleaner implementation #25896 which will be the one merged into main branch.

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @heheda12345.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request introduces support for the Deepseek v3.2 model, which involves substantial changes across the codebase. Key additions include updates to the build system for a new version of FlashMLA, new CUDA kernels for custom attention and quantization mechanisms specific to this model, and a new sparse attention backend. The changes are extensive, touching upon model definitions, attention operations, and KV cache management to accommodate the unique architecture of Deepseek v3.2. My review has identified a critical bug in the new Triton utilities for sequence packing and unpacking, as well as a significant performance issue in a data gathering function. I have provided detailed suggestions to address these points. Overall, this is a major and valuable contribution that expands vLLM's model support.

[gemini-code-assist]

The current implementation of cp_gather_indexer_k_quant_cache is inefficient for GPU execution. It iterates over batches in a Python loop and uses list.append followed by torch.cat to build the final tensors. This pattern creates many small, intermediate tensors and can lead to significant overhead from memory allocations and kernel launch latencies, especially with a large number of batches.

A more performant approach is to pre-allocate the destination tensors (dst_value, dst_scale) and write the results directly into the appropriate slices within the loop. This avoids the overhead of list manipulation and multiple concatenations, resulting in a more efficient, vectorized operation.

    for b in range(batch_size):
        s = cu_seq_lens[b + 1] - cu_seq_lens[b]
        if s == 0:
            continue

        start_idx = cu_seq_lens[b]

        tot = cdiv(s, block_size)
        blocks = block_table[b, :tot]

        num_full_blocks = s // block_size
        num_full_tokens = 0
        if num_full_blocks > 0:
            full_blocks_indices = blocks[:num_full_blocks]
            num_full_tokens = num_full_blocks * block_size

            value_full = kv_cache[full_blocks_indices, :block_size * head_dim].reshape(-1, head_dim)
            dst_value[start_idx:start_idx + num_full_tokens].copy_(value_full.view(torch.float8_e4m3fn))

            scale_full = kv_cache[full_blocks_indices, block_size * head_dim:].reshape(-1, 4)
            dst_scale[start_idx:start_idx + num_full_tokens].copy_(scale_full.view(torch.float32))

        remaining = s - num_full_tokens
        if remaining > 0:
            last_block_idx = blocks[-1]

            rem_start_idx = start_idx + num_full_tokens

            value_rem = kv_cache[last_block_idx, :remaining * head_dim].view(-1, head_dim)
            dst_value[rem_start_idx:rem_start_idx + remaining].copy_(value_rem.view(torch.float8_e4m3fn))

            scale_rem = kv_cache[last_block_idx, block_size * head_dim:block_size * head_dim + remaining * 4].view(-1, 4)
            dst_scale[rem_start_idx:rem_start_idx + remaining].copy_(scale_rem.view(torch.float32))

[youkaichao]

close as we will merge #25896