lmsys

/

SGLang-EAGLE3-Qwen3-Next-80B-A3B-Instruct-FP8-SpecForge-Meituan

like 2

Follow

Large Model Systems Organization 856

@frankleeeee

Can you merge your branch feature/eagle3 to main so that we can use this eagle3 spec?

I can see that you implemented the eagle3 feature in qwen3next
https://github.com/sgl-project/sglang/blob/feature/eagle3/python/sglang/srt/models/qwen3_next.py#L1059

I tried it and got worse results than the built-in MTP spec

@frankleeeee

Thanks for this great work! I wonder whether you test the performance using MTP spec in SpecForge? Because in my local test, the preformance of MTP spec is similar(even little better) to the draft model in this repo.

Therefore, providing a comparison between MTP spec and this repo's draft model would be more helpful. Otherwise, I think training a draft model for a LLM that already has an MTP layer may not be particularly meaningful.

is it possible to use by vllm?