# Serving 2 Million Models Without Melting: Scaling the Hugging Face Hub — Arek Borucki, Hugging Face

Source: [AI Engineer](https://www.youtube.com/watch?v=lyL5QhgIOxc)  
Feed7 permalink: https://feed7.dev/p/serving-2-million-models-without-melting-scaling-the-hugging-face-hub-ar-06hazr6  
Published: 2026-07-28T13:41:11.000Z  
Trust: Source Linked (source_linked)

## Why Included

Hugging Face’s Hub shows how to scale model search: precompute tokens, separate metadata from artifacts, isolate heavy reads, and autoscale from workload signals.

## Source Summary

Hugging Face reports **3 million public models**, **1 million datasets**, and more than **14 million users**. Model metadata lives in MongoDB Atlas, artifacts live in object storage, and search uses precomputed tokens plus Lucene-backed Atlas Search.

## Practical Implication

For large agent-facing catalogs, optimize the read path separately from writes and binary storage. Route stale-tolerant reads and heavy aggregation away from the primary, isolate reporting, and scale both application pods and cluster nodes.

## Agent-Ready Context

Hugging Face reports **3 million public models**, **1 million datasets**, and more than **14 million users**. Model metadata lives in MongoDB Atlas, artifacts live in object storage, and search uses precomputed tokens plus Lucene-backed Atlas Search.

For large agent-facing catalogs, optimize the read path separately from writes and binary storage. Route stale-tolerant reads and heavy aggregation away from the primary, isolate reporting, and scale both application pods and cluster nodes.

These choices reflect Hugging Face’s workload, not a universal blueprint. Sharding is still planned, and moving from CPU- and memory-based HPA to **KEDA workload metrics** is described as upcoming work.

## Connected Context

Feed7 judgment across 262 accumulated Signals:

This contributes a concrete read-heavy catalog architecture rather than another model-serving or retrieval technique: metadata, binaries, search, reporting, and stale-tolerant reads are scaled as distinct paths. It is useful as workload-specific evidence, not a finished blueprint, because sharding and workload-based autoscaling remain planned rather than demonstrated.

No material corpus connection was strong enough to record.

## Context Map

- Layer: infra
- Domains: data
- Topics: retrieval, open-models

## Uncertainty

- These choices reflect Hugging Face’s workload, not a universal blueprint. Sharding is still planned, and moving from CPU- and memory-based HPA to **KEDA workload metrics** is described as upcoming work.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.