minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

examples: add Chain-of-Verification (CoVe) hallucination reduction demo (#27866)

Browse Source 
Co-authored-by: unknown <liuxiao.209@360buyad.local>
This commit is contained in:
liuxpro
2026-06-12 01:57:33 -07:00
committed by GitHub
co-authored by unknown
parent 371b96e210
commit 83c0007f8d
2 changed files with 279 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/runtime/README.md
+1 -1
View File
@@ -13,7 +13,7 @@ The below examples will mostly need you to start a server in a separate terminal
When enabled, the final (partial) assistant message is removed and its content is used as a prefill so that the model continues that message rather than starting a new turn. See [Anthropic's prefill example](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/prefill-claudes-response#example-structured-data-extraction-with-prefilling) for more context.
* `reward_model.py`: An example how to extract scores from a reward model.
* `vertex_predict.py`: An example how to deploy a model to [Vertex AI](https://cloud.google.com/vertex-ai?hl=en).
* `chain_of_verification.py`: An example of [Chain-of-Verification (CoVe)](https://arxiv.org/abs/2309.11495) to reduce hallucinations. The model drafts an answer, then verifies it in a **fresh, isolated session** (no shared KV-cache) to avoid self-confirmation bias, and refines if needed.
## Engine
The `engine` folder contains that examples that show how to use [Offline Engine API](https://docs.sglang.io/basic_usage/offline_engine_api.html#Offline-Engine-API) for common workflows.