minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Rename: --hooks to --forward-hooks (#13994)

Browse Source
This commit is contained in:
Liangsheng Yin
2025-11-26 22:26:28 +08:00
committed by GitHub
parent eff6a07c8f
commit 6c190cbda0
6 changed files with 29 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/forward_hooks.md
+11 -11
View File
@@ -20,14 +20,14 @@ Hooks are configured via a `ServerArgs` field:
class ServerArgs:
...
# For forward hooks
hooks: Optional[List[dict[str, Any]]] = None
forward_hooks: Optional[List[dict[str, Any]]] = None
````
In JSON form, a minimal configuration looks like:
```jsonc
{
"hooks": [
"forward_hooks": [
{
"name": "outer_linear_hooks",
"target_modules": ["outer.0", "outer.1"],
@@ -42,7 +42,7 @@ In JSON form, a minimal configuration looks like:
#### Top-level fields
* `hooks` (optional list of objects)
* `forward_hooks` (optional list of objects)
Each element is a hook spec describing:
* Which modules to target
@@ -53,7 +53,7 @@ In JSON form, a minimal configuration looks like:
### Hook spec schema
Each entry in `hooks` is a JSON object with the following shape:
Each entry in `forward_hooks` is a JSON object with the following shape:
```jsonc
{
@@ -149,16 +149,16 @@ The first two cause initialization to fail fast with a descriptive error; the la
Hooks are registered in `ModelRunner.initialize()`:
```python
if server_args.hooks:
register_hooks(self.model, server_args.hooks)
if server_args.forward_hooks:
register_forward_hooks(self.model, server_args.forward_hooks)
```
The actual registration logic is implemented by `register_hooks`:
The actual registration logic is implemented by `register_forward_hooks`:
```python
def register_hooks(model: nn.Module, hook_specs: List[dict[str, Any]]) -> None:
def register_forward_hooks(model: nn.Module, hook_specs: List[dict[str, Any]]) -> None:
"""
hook_specs is a list of dicts from server_args.hooks.
hook_specs is a list of dicts from server_args.forward_hooks.
Attaches forward hooks to the matching modules.
"""
name_to_module = dict(model.named_modules())
@@ -255,7 +255,7 @@ In JSON:
```jsonc
{
"hooks": [
"forward_hooks": [
{
"name": "capture_outer",
"target_modules": ["outer.0", "outer.1"],
@@ -279,7 +279,7 @@ This will:
### Summary
* Define `hooks` as a list of specs in `ServerArgs` to turn on the feature.
* Define `forward_hooks` as a list of specs in `ServerArgs` to turn on the feature.
* Each spec:
docs/advanced_features/server_arguments.md
+1 -1
View File
@@ -403,7 +403,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
## Forward hooks
| Argument | Description | Defaults | Options |
| --- | --- | --- | --- |
| `--hooks` | JSON-formatted list of hook specifications. Each element must include `target_modules` (list of glob patterns matched against `model.named_modules()` names) and `hook_factory` (Python import path to a factory, e.g. `my_package.hooks:make_hook`). An optional `name` field is used for logging, and an optional `config` object is passed as a `dict` to the factory. | `None` | Type: JSON list |
| `--forward-hooks` | JSON-formatted list of forward hook specifications. Each element must include `target_modules` (list of glob patterns matched against `model.named_modules()` names) and `hook_factory` (Python import path to a factory, e.g. `my_package.hooks:make_hook`). An optional `name` field is used for logging, and an optional `config` object is passed as a `dict` to the factory. | `None` | Type: JSON list |
## Debug tensor dumps
| Argument | Description | Defaults | Options |