Run a vLLM Server on HF Jobs in One Command
HF Jobs now stands up an OpenAI-compatible vLLM endpoint in one command, billed per second (A10G at $1.50/hr). Useful for throwaway endpoints: one-off evals, batch runs, agent experiments against open models.
**The gist** Hugging Face Jobs can launch a vLLM server as an **OpenAI-compatible endpoint** with a single `hf jobs run` command on the stock **vllm/vllm-openai** Docker image — requires **huggingface_hub >= 1.20.0** and a payment method. Billing is **per second** by hardware flavor (**A10G Large at $1.50/hour**; paired H200s with tensor parallelism serve a 122B model), endpoints are gated behind HF tokens, and SSH debugging is available.
**Why it matters** When your agent stack needs a temporary open-model endpoint — a one-off eval, batch generation, or trying a model before committing — this is faster to wire than a managed Inference Endpoint, and anything speaking the **OpenAI client** protocol points at it unchanged.
**The gist** Hugging Face Jobs can launch a vLLM server as an **OpenAI-compatible endpoint** with a single `hf jobs run` command on the stock **vllm/vllm-openai** Docker image — requires **huggingface_hub >= 1.20.0** and a payment method. Billing is **per second** by hardware flavor (**A10G Large at $1.50/hour**; paired H200s with tensor parallelism serve a 122B model), endpoints are gated behind HF tokens, and SSH debugging is available. **Why it matters** When your agent stack needs a temporary open-model endpoint — a one-off eval, batch generation, or trying a model before committing — this is faster to wire than a managed Inference Endpoint, and anything speaking the **OpenAI client** protocol points at it unchanged. **Watch out** This is for **experiments, not production**: jobs stop at their **--timeout**, you pay until you cancel, and larger models need manual tuning of --max-model-len and --max-num-seqs to fit memory.