# Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Training

Source: [arXiv](https://arxiv.org/abs/2608.14498v1)  
Feed7 permalink: https://feed7.dev/p/2608-14498v1-1a3lotd  
Published: 2026-08-14T17:13:34.000Z  
Trust: Needs Review (needs_review)

## Why Included

Rollplex overlaps VLM prefix work with rollout decoding and shares weights across training phases. On 32 H800 GPUs, it reports higher throughput without changing synchronous on-policy updates.

## Source Summary

Rollplex moves response-independent prefix computation into the rollout decode window. Phase-aware HBM management and parallelism-aware weight sharing address a naive Qwen2.5-VL-32B footprint of roughly **165 GiB per GPU**.

## Practical Implication

Teams post-training vision-language agents can inspect phase boundaries for independent prefix work and share compatible tensor storage across differing parallel layouts. The reported schedule preserves synchronous on-policy updates under the same GPU budget.

## Agent-Ready Context

Rollplex moves response-independent prefix computation into the rollout decode window. Phase-aware HBM management and parallelism-aware weight sharing address a naive Qwen2.5-VL-32B footprint of roughly **165 GiB per GPU**.

Teams post-training vision-language agents can inspect phase boundaries for independent prefix work and share compatible tensor storage across differing parallel layouts. The reported schedule preserves synchronous on-policy updates under the same GPU budget.

On **32 H800 GPUs**, Rollplex reports **1.23×–1.30×** speedup over serial colocation and **1.57×–2.24×** over disaggregation. The material provides results for this hardware and workload setup, not broader cost or scaling evidence.

## Connected Context

Feed7 judgment across 468 accumulated Signals:

Rollplex adds a concrete systems pattern for visual on-policy post-training: overlap response-independent prefix work with rollout decoding and share compatible tensor storage across phase-specific parallel layouts. It strengthens the case that phase scheduling can outperform both serial colocation and disaggregation under a fixed cluster budget, while narrowing the evidence to synchronous Qwen2.5-VL training on 32 H800s rather than general scaling or cost efficiency.

- [Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal](https://feed7.dev/p/taking-reinforcement-learning-cross-datacenter-nan-jiang-modal-05t7dwu) — Both reorganize rollout infrastructure, but at different boundaries: Rollplex spatially shares one GPU cluster while preserving synchronous updates, whereas the cross-datacenter proposal separates rollout workers and synchronizes sparse weight deltas.
- [Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel](https://feed7.dev/p/accelerating-fine-tuning-nvidia-nemo-automodel-0dj2ywc) — Both show that post-training throughput can improve through infrastructure changes without changing the model objective, but NeMo AutoModel addresses MoE fine-tuning APIs while Rollplex targets phase overlap and memory sharing in VLM rollouts, so their speedups are not directly comparable.
- [OPD-V: Visual On-Policy Self-Distillation with Modality Balance](https://feed7.dev/p/2608-05131v1-0m0x349) — OPD-V supplies an example of visual on-policy post-training whose rollout workload could benefit from Rollplex’s scheduling pattern; Rollplex addresses execution efficiency rather than OPD-V’s modality-balance objective.

## Context Map

- Layer: infra
- Domains: image, video
- Topics: None

## Uncertainty

- On **32 H800 GPUs**, Rollplex reports **1.23×–1.30×** speedup over serial colocation and **1.57×–2.24×** over disaggregation. The material provides results for this hardware and workload setup, not broader cost or scaling evidence.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.