Commit Graph
47 Commits
Author SHA1 Message Date
Yuhao YangandMick e14f5ec8a8 [diffusion] refactor: eliminate redundant parameters in req (#16505)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-01-08 11:14:03 +08:00
Yuhao Yang 10174e1114 Revert "[grpc] update api to scheduler in grpc request manager" (#16387) 2026-01-04 22:05:39 -08:00
Yuhao Yang 2138ff48c6 Revert "[FEAT] optimize tensor zmq transfer for multimodal inputs" (#16386) 2026-01-04 22:05:26 -08:00
Yuhao Yang 6c8587b5db [diffusion] fix: align negative prompt with official readme for new model (#16222) 2026-01-02 15:20:56 +08:00
Yuhao YangandMick 4280a18a13 [diffusion] CI: add test for cache-dit (#16204)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-12-31 19:58:10 +08:00
Yuhao YangandMick 39ca57cd28 [diffusion] chore: tiny fix model config (#16159)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-12-30 22:11:38 +08:00
Yuhao Yang 0cd2b719a5 [diffusion] chore: remove useless params (#15925) 2025-12-28 01:01:08 +08:00
Yuhao Yang 29ce7b3612 [diffusion] chore: remove stepvideo code (#15918) 2025-12-27 13:25:05 +08:00
Yuhao Yang ba41080892 [diffusion] model: support qwen-image-edit-2511 (#15458) 2025-12-19 20:06:00 +08:00
Yuhao Yang 3d42b7e7b0 unified management of environment variables for vlm cuda ipc transport (#14501) 2025-12-18 12:28:06 +08:00
Yuhao Yang 01b955ac3d [diffusion] model: support mutli-image input and qwen-image-edit-2509 (#15005) 2025-12-15 16:17:10 +08:00
Yuhao Yang a81cc1b8b3 add transformers version validation for glm-4.6v moe models (#14998) 2025-12-13 10:54:08 -08:00
Yuhao Yang 06b58c5dc5 fix flaky image access in ci by switching to raw content url (#14940) 2025-12-13 10:52:06 -08:00
Yuhao Yang b62fe8504c fix nightly vlm ci : restore original eval for requests without regex (#14875) 2025-12-10 23:13:25 -08:00
Yuhao Yang c1bd5ee8c5 Revert transformers to 4.57.1 (#14801) 2025-12-10 11:04:36 -08:00
Yuhao Yang 02f1e81e2d Revert "fix: checking if tokenizer is in cache before downloading from HF" (#14808) 2025-12-10 01:14:35 -08:00
Yuhao Yang 793c98afaf handling incomplete rope_scaling config ci after transformers upgrade (#14784) 2025-12-09 22:56:16 -08:00
Yuhao Yang 15bc8cbd74 fix rope parameter initialization error caused by transformers v5.0 update (#14745) 2025-12-09 10:51:26 -08:00
Yuhao Yang b9bef31a15 fix: use .get() when accessing strict mem-check env variable (#14657) 2025-12-08 13:25:42 -08:00
Yuhao Yang 8200fb56cb update transformers package version to 5.0.0rc0 (#14356) 2025-12-08 22:46:01 +08:00
Yuhao Yang f72a77038f modify the sgl-kernel to be compatible with transformers 5.x. (#14625) 2025-12-08 00:39:00 -08:00
Yuhao Yang 7b0c7ad163 Revise DP Multi-Modal Encoder Document (#14290) 2025-12-06 12:56:53 +08:00
Yuhao Yang 0b9dbea593 [diffusion] chore: improve z-image (#14104) 2025-12-01 12:26:17 +08:00
Yuhao Yang f03ea34a3d add runtime check for PyTorch 2.9.1 + CuDNN < 9.15 to prevent Conv3d performance issues (#14119) 2025-11-29 10:05:54 -05:00
Yuhao Yang 841eb29d3d [diffusion] model: support z-image (#14067) 2025-11-28 21:48:31 +08:00
Yuhao Yang 5795da5e83 [diffusion] fix: fix the issue where the qwen-edit & wan model produces incorrect output during sequence parallelism (#13922) 2025-11-26 15:58:23 +08:00
Yuhao YangandMick e5c0f59133 [diffusion] CI: send nightly-test outputs of diffusion to slack for correctness monitoring (#13833)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-24 15:22:26 +08:00
Yuhao YangandMick dd303614e0 [diffusion] CI: tinyfix diffusion ci (#13769)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-23 01:03:36 +08:00
Yuhao YangandMick a22de641ef [diffusion] fix: fix hunyuanvideo and add 2-gpu ci test (#13720)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-22 20:54:51 +08:00
Yuhao YangandMick 8c212a2029 [difusion] CI: speed up multimodal_gen ci (#13665)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-21 14:26:33 +08:00
Yuhao Yang 7119d188f6 [CI] re-enable test_vision_openai_server_a ci (#13444) 2025-11-17 18:32:59 -08:00
Yuhao YangandMick 6afe396399 diffusion: support fa4 in fa backend for blackwell (#13263)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-16 21:02:45 +08:00
Yuhao Yang 1240ac13b8 vlm: fix tiny multimodal cache bug (#12984) 2025-11-10 21:30:19 +08:00
Yuhao YangandMick d8736c756a fix multimodal gen issues (#12765)
Co-authored-by: Mick <mickjagger19@icloud.com>
2025-11-07 13:19:49 +08:00
yhyang201 48e9e71930 Add --max-new-tokens CLI flag for MMMU evaluation (#11217) 2025-10-04 17:35:53 -07:00
yhyang201 6130529143 Quick Fix: fix Qwen3-VL launch failure caused by MRotaryEmbedding arg (#10985) 2025-09-30 22:17:05 -07:00
yhyang201 388c05d544 Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py (#10579) 2025-09-18 11:44:43 -07:00
yhyang201 c377923304 [feat] Reduce GPU memory overhead by using weakref (#9673) 2025-08-28 01:09:06 -07:00
a85363c199 [docs] Instructions for bench_serving.py (#9071)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
2025-08-26 18:30:57 -07:00
yhyang201 00da906584 feat: Support DP Attention for step3_vl (#8699) 2025-08-03 19:35:26 +08:00
yhyang201andChayenne 0dfe2491ac Preliminary Support for Qwen3XMLDetector (#8260)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
2025-07-23 06:49:38 +08:00
yhyang201andChang Su dea2b84bc3 [OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
2025-06-20 14:51:21 -07:00
yhyang201 1dffee31ac OAI Server Skeleton & Core Utility Endpoints (#7179) 2025-06-16 20:45:55 -07:00
yhyang201 cec98f1034 [Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745) 2025-05-08 17:52:26 -07:00
yhyang201 92ab0a2055 feat: Add fused moe triton config for qwen3bf16 moe on h20 (#5839) 2025-04-28 09:30:59 -07:00
yhyang201 4db463b1ad [Model] Adding Qwen3 and Qwen3MoE (#4693) 2025-04-18 09:51:29 -07:00
yhyang201 072df75354 Support for Qwen2.5-VL Model in bitsandbytes Format (#5003) 2025-04-14 02:03:40 -07:00