llmprobe

stdionone★ 1development

Probe LLM API endpoints and report health metrics including time to first token, latency, and throughput. Check single models or run full config-based health checks.

github ↗

Install

How to run this server

No install command recorded.

server path: llmprobe

Tools

What this server exposes

probe_all

Probe all configured endpoints from probes.yml. Returns TTFT, latency, throughput, and health status for every model. Accepts an optional config parameter for a custom config path.

probe_model

Probe a single model without a config file. Requires provider, model, and api_key_env. Supports optional base_url for OpenAI-compatible endpoints and optional label for display.

list_providers

List all providers and models in the config file with their thresholds. Use this to discover available models before probing.

get_config

Return the full parsed configuration including defaults, providers, models, and thresholds.

Configuration

Environment & auth

authnone

envOPENAI_API_KEY

envANTHROPIC_API_KEY

envAZURE_OPENAI_API_KEY

envAWS_ACCESS_KEY_ID

envAWS_SECRET_ACCESS_KEY

envGROQ_API_KEY

envDEEPSEEK_API_KEY

envTOGETHER_API_KEY

Resources

githubhttps://github.com/Jwrede/llmprobe ↗