LLM Control Plane

Endpoints

An endpoint is identified by the composite pair endpoint name + runtime type. The same endpoint name can therefore exist once for each runtime type.

Endpoint

Endpoint nameDisplay nameRuntime typeEnabled

Endpoint identity: gb10_1 (vllm)

Base URL

API keyHardware labelMax concurrent connections

Endpoint notes

Model Profiles

Profiles are ordered as endpoint name → runtime type → profile name. Each profile is bound to one exact endpoint/runtime pair.

Model profile

Profile name / exposed model IDDisplay nameEndpointEnabledDefault generation preset

Binding: gb10_1 (vllm) gb10_1_vllm_deepseek_v41_flash_exl3 · record 7a737183de

Discovered upstream modelUpstream model ID

Use Refresh Models From Endpoints to query /v1/models. Discovery is scoped to the exact endpoint name/runtime pair.

QuantizationArchitectureTotal params BActive/effective params B

MoE expert countMoE active expertsMax context tokensSupports visionSupports tools

vLLM host/container binding

These fields bind host actions to this exact model profile.

Host control

Only host-control profiles bound to this model profile's vLLM endpoint name are shown.

Container nameLaunch script pathServed model name

Portmax_num_seqsmax_num_batched_tokensgpu_memory_utilization

kv_cache_dtypevLLM quantization

Measured output tok/secMeasured prefill tok/secMeasured TTFT ms

Model profile notes

Saved Model Records

This list is read directly from /persist/config/model_profiles.json. endpoints.json is never a model-profile source. Routing and /v1/models read this same file authority.

Status	Endpoint / profile	Display name	Upstream	Record
PUBLISHED / routable	`5090 (lm_studio) 5090_lm_studio_huihui_qwen3_8_27b_abliterated`	huihui-qwen3.8-27b-abliterated	`huihui-qwen3.8-27b-abliterated`	`5ba6e97d1b`
PUBLISHED / routable	`5090 (lm_studio) nvidia_5090___qwen3.5-9b-uncensored-hauhaucs-aggressive`	qwen3.5-9b-uncensored-hauhaucs-aggressive via NVIDIA 5090	`qwen3.5-9b-uncensored-hauhaucs-aggressive`	`legacy_f88`
PUBLISHED / routable	`gb10_1 (vllm) gb10_1_vllm_deepseek_ai_deepseek_v4_1_flash`	deepseek-ai/DeepSeek-V4.1-Flash	`deepseek-ai/DeepSeek-V4.1-Flash`	`5690420dc3`
PUBLISHED / routable	`gb10_1 (vllm) gb10_1_vllm_deepseek_v41_flash_exl3`	deepseek-v41-flash-exl3	`deepseek-v41-flash-exl3`	`7a737183de`
PUBLISHED / routable	`gb10_1 (vllm) gb10_1_vllm_deepseek_v4_flash_dspark_abliterated`	deepseek-v4-flash-dspark-abliterated	`deepseek-v4-flash-dspark-abliterated`	`e159342897`
PUBLISHED / routable	`gb10_1 (vllm) gb10_1_vllm_qwen3_8_flash_next`	qwen3.8-flash-next	`qwen3.8-flash-next`	`9afb2d809b`

Endpoint Inventory

Status	Endpoint identity	Display name	Base URL	Discovered upstream models	Profiles	Capacity
offline HttpRequestException	`5090 (lm_studio)`	5090	`http://192.168.1.202:1234/v1`	0	2 5090_lm_studio_huihui_qwen3_8_27b_abliterated, nvidia_5090___qwen3.5-9b-uncensored-hauhaucs-aggressive	0 / 4
offline HttpRequestException	`dell_l40 (lm_studio)`	Dell_L40	`http://192.168.165.249:1234/v1`	0	0	0 / 8
online HTTP 200 OK	`gb10_1 (vllm)`	GB10_1	`http://192.168.10.1:8003/v1`	1 qwen3.8-flash-next	4 gb10_1_vllm_deepseek_ai_deepseek_v4_1_flash, gb10_1_vllm_deepseek_v41_flash_exl3, gb10_1_vllm_deepseek_v4_flash_dspark_abliterated, gb10_1_vllm_qwen3_8_flash_next	0 / 4
offline HttpRequestException	`gb10_1_lm_studio (lm_studio)`	GB10_1_lm_studio	`http://192.168.10.1:1234/v1`	0	0	0 / 1

Current Model Catalog

Ordered by endpoint name, runtime type, then profile name.

Endpoint / profile	Display name	Upstream model	Online	Discovered	Preset	Benchmark	Metadata
`5090 (lm_studio) 5090_lm_studio_huihui_qwen3_8_27b_abliterated`	huihui-qwen3.8-27b-abliterated	`huihui-qwen3.8-27b-abliterated`	False	False	`creative_brainstorming`	70.0 tok/s TTFT 235 ms	total / active B; ; ctx
`5090 (lm_studio) nvidia_5090___qwen3.5-9b-uncensored-hauhaucs-aggressive`	qwen3.5-9b-uncensored-hauhaucs-aggressive via NVIDIA 5090	`qwen3.5-9b-uncensored-hauhaucs-aggressive`	False	False	`creative_brainstorming`	113.4 tok/s TTFT 105 ms	total / active B; ; ctx
`gb10_1 (vllm) gb10_1_vllm_deepseek_ai_deepseek_v4_1_flash`	deepseek-ai/DeepSeek-V4.1-Flash	`deepseek-ai/DeepSeek-V4.1-Flash`	True	False	`glm_5_3_flash_speculative`	15.3 tok/s TTFT 318 ms	total / active B; ; ctx
`gb10_1 (vllm) gb10_1_vllm_deepseek_v41_flash_exl3`	deepseek-v41-flash-exl3	`deepseek-v41-flash-exl3`	True	False	`glm_5_3_flash_speculative`	29.9 tok/s TTFT 430 ms	total / active B; ; ctx
`gb10_1 (vllm) gb10_1_vllm_deepseek_v4_flash_dspark_abliterated`	deepseek-v4-flash-dspark-abliterated	`deepseek-v4-flash-dspark-abliterated`	True	False	`deepseek_speculative`	21.8 tok/s TTFT 298 ms	total / active B; ; ctx
`gb10_1 (vllm) gb10_1_vllm_qwen3_8_flash_next`	qwen3.8-flash-next	`qwen3.8-flash-next`	True	True	`glm_5_3_flash_speculative`	42.8 tok/s TTFT 176 ms	total / active B; ; ctx