# Run a vLLM Server on HF Jobs in One Command

Source: [huggingface.co](https://huggingface.co/blog/vllm-jobs)  
Feed7 permalink: https://feed7.dev/p/vllm-jobs-0236m0i  
Published: Unknown  
Trust: Source Linked (source_linked)

## Why Included

HF Jobs now stands up an OpenAI-compatible vLLM endpoint in one command, billed per second (A10G at $1.50/hr). Useful for throwaway endpoints: one-off evals, batch runs, agent experiments against open models.

## Source Summary

**The gist** Hugging Face Jobs can launch a vLLM server as an **OpenAI-compatible endpoint** with a single `hf jobs run` command on the stock **vllm/vllm-openai** Docker image — requires **huggingface_hub >= 1.20.0** and a payment method. Billing is **per second** by hardware flavor (**A10G Large at $1.50/hour**; paired H200s with tensor parallelism serve a 122B model), endpoints are gated behind HF tokens, and SSH debugging is available.

## Practical Implication

**Why it matters** When your agent stack needs a temporary open-model endpoint — a one-off eval, batch generation, or trying a model before committing — this is faster to wire than a managed Inference Endpoint, and anything speaking the **OpenAI client** protocol points at it unchanged.

## Agent-Ready Context

**The gist** Hugging Face Jobs can launch a vLLM server as an **OpenAI-compatible endpoint** with a single `hf jobs run` command on the stock **vllm/vllm-openai** Docker image — requires **huggingface_hub >= 1.20.0** and a payment method. Billing is **per second** by hardware flavor (**A10G Large at $1.50/hour**; paired H200s with tensor parallelism serve a 122B model), endpoints are gated behind HF tokens, and SSH debugging is available.

**Why it matters** When your agent stack needs a temporary open-model endpoint — a one-off eval, batch generation, or trying a model before committing — this is faster to wire than a managed Inference Endpoint, and anything speaking the **OpenAI client** protocol points at it unchanged.

**Watch out** This is for **experiments, not production**: jobs stop at their **--timeout**, you pay until you cancel, and larger models need manual tuning of --max-model-len and --max-num-seqs to fit memory.

## Context Map

- Layer: infra
- Domains: None
- Topics: open-models

## Uncertainty

- This is for **experiments, not production**: jobs stop at their **--timeout**, you pay until you cancel, and larger models need manual tuning of --max-model-len and --max-num-seqs to fit memory.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.