Cumulus Labs presents IonAttention, its fused prefill-plus-decode attention kernel for NVIDIA Grace. The kernel serves 30 to 50 percent more tokens per second than stock vLLM and SGLang on a GH200. The technical deep dive focuses on IonAttention’s custom attention kernels.
Cumulus Labs explains that hand-written failover code fails in production. It presents declared routing rules as a way to make dispatch deterministic, traceable, and resilient to provider outages.
Cumulus Labs presents a guide to inference platforms, framing inference as more than a model call. The guide covers routing, caching, observability, evaluation, fine-tuning, and runtime components, treating them as parts of the broader platform rather than reducing inference to a single model invocation.
Cumulus presents a one-line migration path for moving an existing application built with the OpenAI SDK to Cumulus without changing its code. The tutorial explains what Cumulus’s routing, caching, and observability layers provide as part of the migration.