Generative Media

DeepMind’s panel shows why generative-media evals need task-specific human review: broad preferences can miss repeated artifacts, exact sizing, text errors, and brand consistency.

Vercel is halving AI Gateway charges for MiniMax H3 and H3 Max through September 13; existing model IDs receive the discount without code changes.

Adobe’s prototype assembles intent-specific page blocks from existing site content in roughly a second, making model latency and per-site evaluation part of frontend architecture.

Muse Image is available through Vercel AI Gateway for both generation and instruction-based editing, including reference-image guidance through the AI SDK.

Gemini 3.5 Transcribe adds batch and live WebSocket transcription to AI Gateway, with automatic language detection, 85+ languages, and custom vocabulary.

Wan 3.0 gives AI Gateway one video model ID for text, image, frame, and reference workflows, with async renders up to 30 seconds at 1080p and synchronized audio.

Re³Cap uses multimodal retrieval to find caption omissions and hallucinations before refinement, offering a concrete retrieval-and-review pattern for vision agents.

DeepSeek V4 Flash Vision adds screenshot, image, and chart input to Vercel AI Gateway while retaining tool use, reasoning, and caching, but its experimental ID signals production risk.

Vercel AI Gateway added four Fish Audio models for speech generation and transcription, with AI SDK 7 support and a free window whose model naming determines later billing.

Reactor frames real-time video as a programmable session rather than a generated file, enabling interactive worlds and live editing but exposing hard state, latency, and evaluation problems.

Real-time video models are becoming cheap and responsive enough for agent interfaces, but builders still need global GPU routing, streaming infrastructure, and multi-model orchestration.

LemonSlice’s avatar stack treats long-running visual stability, audio-conditioned emotion, and deterministic action timing as the core engineering problems beyond lip sync.

Krea 2’s training notes put data curation and iteration speed ahead of architecture novelty, with explicit safeguards against filtering away unusual visual styles.

Vercel AI Gateway now exposes xAI’s image model through the AI SDK, including 1K/2K generation, batches, and targeted edits that aim to preserve untouched details.

SABRE turns a Markdown test design into generated VLM stress tests, then filters and repairs candidates. It offers a repeatable pattern for refreshing evals as models improve.

Seedance 2.5 adds multimodal video generation and local edits to AI Gateway, with clips up to 30 seconds and separate image and video references.

Kenton Varda argues that personal AI-generated apps need per-user code and strong isolation, not one server-owned version. The demo shows agents modifying app code inside a constrained local runtime.

Controlled text-to-MIDI tests find token representation matters more than a 34× model-size increase for distributional fidelity; performance timing also beats beat-grid tokenization.

MiniMax H3 brings short 2K video generation to Vercel AI Gateway, with text, keyframe, and multimodal reference inputs. Reference and keyframe modes cannot be combined.

Grok Voice Think Fast 2.0 brings speech-to-speech reasoning and earlier tool calls to Vercel’s realtime API, with server-minted tokens keeping gateway keys off clients.

ModernMOE applies efficient expert-routing patterns from LLMs to diffusion transformers, improving convergence and quality-cost balance without relying only on larger parameter counts.

Video evaluators can reward polish while missing frozen action, broken physics, or failed storytelling. Builders need time-aware criteria and human-calibrated data, not frame quality alone.

Uber’s image-editing agent uses routing, iterative QA, golden-set gates, and production feedback to avoid costly edits, hallucinated food, and quality regressions.

MedGame turns static clinical cases into executable decision stories with separate narrative and orchestration stages, a useful architecture pattern for case-grounded learning agents.

A frozen diffusion language model can transcribe speech by refining the full transcript in parallel. The prototype trains a small audio interface and reaches 6.6% WER in roughly eight steps.

Seedream 5.0 Pro adds image generation and editing to Vercel AI Gateway, targeting reliable text rendering and dense infographic layouts through the AI SDK.

E-VQA requires video answers to include tracked pixel-level evidence, revealing when good QA scores hide weak perception and supplying grounded training data.

OpenMontage turns coding agents into pipeline-driven video producers, covering research through rendering with approval gates, provider selection, and auditable costs.

IAAN boosts selected audio-encoder neurons at inference, improving fine-grained speech perception across three models without retraining or labels.

SearchGen-Bench shows open image generators score 21–28/100 on long-tail entities, and naive search retrieval only adds noise; a teach-then-search co-training recipe learns when to retrieve versus rely on weights.

Meta's Tech Podcast covers AssetGen, its foundation model for generating 3D assets from text, and the path toward AI-generated worlds in Horizon Studio. A podcast episode, so light on specifics.

Two new Gemini API models: Nano Banana 2 Lite generates 1K images in ~4s at $0.034 each, and Omni Flash does video at $0.10/sec in public preview — cheap enough to wire asset generation into agent pipelines.

Google's June roundup: Gemma 4 12B runs locally in 16GB of memory, Gemini 3.5 Flash adds computer use for desktop, mobile, and browser agents, and Nano Banana 2 Lite ships as a cheaper image model.

ComfyUI turns multimodal generation into reusable node graphs with API access, incremental execution, and offline operation. Pin stable releases if custom nodes matter to your workflow.