Tracking Issue: DeepSeek V3.2 support

[youkaichao]

The main PR #25896 has been merged, and the nightly wheel can be used.

Following the recipes, you should be able to run it. Recipe: https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2-Exp.html .

[boxiaowave]

only support cuda V12.9?

[youkaichao]

only support cuda V12.9?

cuda 12.x should be supported. just the wheel needs to be built using cuda 12.9

[boxiaowave]

only support cuda V12.9?

cuda 12.x should be supported. just the wheel needs to be built using cuda 12.9

So, though this wheel is built with 12.9, we can still use it under cuda 12.x when x!=9?

[youkaichao]

only support cuda V12.9?

cuda 12.x should be supported. just the wheel needs to be built using cuda 12.9

So, though this wheel is built with 12.9, we can still use it under cuda 12.x when x!=9?

yes.

[seindum]

The main PR is #25869 .

Following the recipes, you should be able to run it.

An initial recipe is available at https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2-Exp.html , and we will keep updating it.

@youkaichao is it intended to not have flashinfer from the initial recipe? Because if not, maybe we could be specific to make the command uv pip install -vvv -e .[flashinfer]?

[youkaichao]

is it intended to not have flashinfer from the initial recipe?

you can install it, but it's optional. plain temperature sampling does not need it.

[Roysky]

Is 2-node (H800*8) deployment supported? If not, when will it be supported?

[Jixin10]

I encountered this issue

  File "/source/vllm/vllm/platforms/cuda.py", line 18, in <module>
    import vllm._C  # noqa
ImportError: /usr/lib64/libc.so.6: version `GLIBC_2.34' not found (required by /source/vllm/vllm/_C.abi3.so)

It seems that the precompiled wheel it built with glibc 2.34? But unfortunately my develop machine has glibc with 2.32... Could you provide a precompiled wheel with lower version glibc? My thanks.

[youkaichao]

It seems that the precompiled wheel it built with glibc 2.34? But unfortunately my develop machine has glibc with 2.32... Could you provide a precompiled wheel with lower version glibc? My thanks.

@Jixin10 the precompiled wheel uses ubuntu 24 i think. you can either trigger a full build to get the wheel yourself, or switch to ubuntu 24 to get a newer glibc. after #25896 is merged, nightly wheel will also work.

[youkaichao]

Is 2-node (H800*8) deployment supported? If not, when will it be supported?

@Roysky it should be supported, do you meet any issues?

[Jixin10]

It seems that the precompiled wheel it built with glibc 2.34? But unfortunately my develop machine has glibc with 2.32... Could you provide a precompiled wheel with lower version glibc? My thanks.

@Jixin10 the precompiled wheel uses ubuntu 24 i think. you can either trigger a full build to get the wheel yourself, or switch to ubuntu 24 to get a newer glibc. after #25896 is merged, nightly wheel will also work.

tsk for reply!

[youkaichao]

@Jixin10 you can try the nightly wheel now, the PR has been merged. The nightly wheel should support multiple glibc versions.