[model-gateway][docs] Add Classification API documentation (#16182)

Co-authored-by: Chang Su <chang.s.su@oracle.com>
This commit is contained in:
Simo Lin
2025-12-30 10:53:31 -08:00
committed by GitHub
co-authored by Chang Su
parent d17b9e6392
commit 4bc2f2e0bc
2 changed files with 102 additions and 1 deletions
+47
View File
@@ -482,6 +482,53 @@ The HTTP router exposes the full OpenAI-compatible surface area (`/generate`, `/
| `POST /v1/rerank`, `POST /rerank` | Ranking APIs. |
| `POST /v1/classify` | Text classification endpoint. |
### Classification API
The `/v1/classify` endpoint provides text classification using sequence classification models (e.g., `Qwen2ForSequenceClassification`, `BertForSequenceClassification`).
**Request:**
```bash
curl http://localhost:30000/v1/classify \
-H "Content-Type: application/json" \
-d '{
"model": "jason9693/Qwen2.5-1.5B-apeach",
"input": "I love this product!"
}'
```
**Response:**
```json
{
"id": "classify-a1b2c3d4-5678-90ab-cdef-1234567890ab",
"object": "list",
"created": 1767034308,
"model": "jason9693/Qwen2.5-1.5B-apeach",
"data": [
{
"index": 0,
"label": "positive",
"probs": [0.12, 0.88],
"num_classes": 2
}
],
"usage": {
"prompt_tokens": 6,
"completion_tokens": 0,
"total_tokens": 6
}
}
```
**Fields:**
- `label`: Predicted class label (from model's `id2label` config, or `LABEL_N` fallback)
- `probs`: Probability distribution over all classes (softmax of logits)
- `num_classes`: Number of classification classes
**Notes:**
- Classification reuses the embedding backend—the scheduler returns logits which are converted to probabilities via softmax
- Labels come from the model's HuggingFace config (`id2label` field); models without this mapping use generic labels (`LABEL_0`, `LABEL_1`, etc.)
- Both HTTP and gRPC routers support classification
Public health endpoints (`/liveness`, `/readiness`, `/health`, `/health_generate`) reflect registry state; readiness ensures PD workers are paired and IGW has at least one healthy route.
### Tokenization Endpoints