Skip to content

Commit e16fe04

Browse files
slin1237narutolhy
authored andcommitted
[router] add basic usage doc (sgl-project#8640)
1 parent 3924a0c commit e16fe04

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

sgl-router/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,35 @@ pip install -e .
5454
```bash
5555
# Build Rust components
5656
cargo build
57+
```
5758

59+
#### Launch Router with Worker URLs in regular mode
60+
```bash
5861
# Launch router with worker URLs
5962
python -m sglang_router.launch_router \
6063
--worker-urls http://worker1:8000 http://worker2:8000
6164
```
6265

66+
#### Launch Router with Worker URLs in prefill-decode mode
67+
```bash
68+
# Note that the prefill and decode URLs must be provided in the following format:
69+
# http://<ip>:<port> for decode nodes
70+
# http://<ip>:<port> bootstrap-port for prefill nodes, where bootstrap-port is optional
71+
# Launch router with worker URLs
72+
python -m sglang_router.launch_router \
73+
--pd-disaggregation \
74+
--policy cache_aware \
75+
--prefill http://127.0.0.1:30001 9001 \
76+
--prefill http://127.0.0.2:30002 9002 \
77+
--prefill http://127.0.0.3:30003 9003 \
78+
--prefill http://127.0.0.4:30004 9004 \
79+
--decode http://127.0.0.5:30005 \
80+
--decode http://127.0.0.6:30006 \
81+
--decode http://127.0.0.7:30007 \
82+
--host 0.0.0.0 \
83+
--port 8080
84+
````
85+
6386
## Configuration
6487

6588
### Logging

0 commit comments

Comments
 (0)